A test method for measuring live oil solid deposition with a full view image gauge

Abstract

The application discloses a kind of full-view image metering live oil solid deposition test methods in the field of crude oil solid deposition, step 1, image pre-processing;Step 2, image detection and identification;Step 3, man-machine interaction;Compared with prior art, the beneficial effects of the present application are that the image metering and particle size test of the whole process of solid phase particle generation and growth in the process of crude oil high pressure physical property test are solved, the traditional solid deposition can only be qualitatively analyzed, the quantitative analysis of the whole process and full scale of solid deposition particles is achieved, the image metering and particle size test of the whole process of solid phase particle generation and growth in the process of crude oil high pressure physical property test are solved, the traditional solid deposition can only be qualitatively analyzed, the quantitative analysis of the whole process and full scale of solid deposition particles is achieved;The geometric properties of solid phase can be greatly reduced by using computer vision technology to automatically and quantitatively analyze the solid phase, and the work efficiency is improved.

Description

Technical Field

[0001] This invention relates to a full-view image metrology testing method for crude oil solid phase deposition in the field of crude oil solid phase deposition. Background Technology

[0002] Asphaltene precipitation in crude oil is a major challenge affecting normal oilfield production. Currently, the influencing factors and precipitation patterns are poorly understood, and corresponding prevention technologies are lacking. Existing research on the precipitation patterns of heavy solids in deep-well crude oil relies on qualitative analysis methods, such as optical projection methods. These methods receive light signals, convert them into voltage, and then use temperature and pressure as precipitation conditions when the voltage changes. While these methods rely on the naked eye and optical / electrical equipment to observe changes in crude oil phase state and solid deposition under different temperatures and pressures, and can obtain the temperature and pressure conditions for precipitation, they cannot determine the amount of solid deposition or its variation patterns under those conditions. Furthermore, quantitative measurements of solid precipitation still rely on manual labor and some basic image processing software, such as measuring solid area content and geometric dimensions. This work is labor-intensive, and inconsistent standards among personnel can easily lead to poor representativeness of the measurement results. Summary of the Invention

[0003] The purpose of this invention is to provide a full-view image metrology test method for active oil solid phase deposition, enabling more comprehensive automated analysis of solid phases.

[0004] To achieve the above objectives, the present invention provides a test method for full-view image metrology of active oil solid phase deposition, comprising: step 1, image preprocessing; step 2, image detection and recognition; and step 3, human-computer interaction.

[0005] Compared with existing technologies, the beneficial effects of this invention are that it solves the problem of image measurement and particle size testing of the entire process of solid particle generation and growth during high-pressure crude oil physical property testing, breaking through the traditional qualitative analysis of solid deposition and achieving quantitative analysis of solid deposition particles throughout the entire process and at all scales; the use of computer vision technology to automatically and quantitatively analyze the geometric properties of solid materials can greatly reduce labor costs and improve work efficiency, and the use of visual algorithms can ensure the consistency of detection results.

[0006] As a further improvement to the present invention, the specific content of step 1 is as follows.

[0007] Step 1.1, Input Image: The image is acquired and input via a host computer through a light source system and a CCD industrial camera system;

[0008] Step 1.2, Image Storage: Images are acquired in real time. The software can be set to save images to a specified directory at regular intervals. The image analysis software automatically processes the saved images in a timely manner to achieve real-time processing. Images are saved sequentially by time and number. The image analysis software continuously identifies the newly output image numbers in the specified directory and processes them in real time.

[0009] Step 1.3, Binarization: The Otus algorithm, also known as the maximum inter-class variance method, is used to automatically select the binarization threshold.

[0010] Step 1.4, Noise Preprocessing: The median filtering preprocessing algorithm is used, and the anisotropic diffusion smoothing filtering technique is used to smooth and denoise the sediment images to be processed.

[0011] As a further improvement to the present invention, the specific content of step 1.4 is as follows:

[0012] Step 1.4.1, targeting Median filtering first calculates the maximum, median, and minimum values ​​for each column within the window, resulting in three sets of data.

[0013] Maximum value group: Max0 = max[P0, P3, P6], Max1 = max[P1, P4, P7], Max2 = max[P2, P5, P8]

[0014] Median groups: Med0 = med[P0, P3, P6], Med1 = med[P1, P4, P7], Med2 = med[P2, P5, P8]

[0015] Minimum value set: Min0 = Min[P0, P3, P6], Min1 = Min[P1, P4, P7], Min2 = max[P2, P5, P8]

[0016] in for From top left to bottom right in the region 1 pixel;

[0017] In this comparison, the maximum value in the maximum value group and the minimum value in the minimum value group must be the maximum and minimum values ​​among the 9 elements, and cannot be the median, leaving 7 elements. In the median value group, the maximum value must be at least 5 pixels greater than the minimum value, and cannot be the median, leaving 5 elements. In the maximum value group, the median value must be at least 5 elements greater than the minimum value group, and cannot be the median, leaving 3 elements to compare: the minimum value in the maximum value group, the median value in the median value group, and the maximum value in the minimum value group. The median of these three values ​​is the median of the 9 elements.

[0018] Step 1.4.2: Considering that median filtering cannot effectively filter continuously generated noise, while anisotropic filtering can further smooth homogeneous regions, anisotropic filtering algorithm is introduced here.

[0019] Based on the heat conduction equation, an anisotropic diffusion equation can be established and applied to image smoothing in the medium-scale space of image processing. The diffusion equation for a two-dimensional image signal is:

[0020]

[0021] in, It is the original image. It is spatial information about the diffusion process. This indicates the step size, and also represents the time information of the diffusion process. and Let them represent the divergence operator and the gradient operator, respectively. It is the diffusion coefficient, which is defined as:

[0022]

[0023] The formula indicates that the diffusion coefficient is a monotonically decreasing function with respect to the gradient magnitude. When the gradient is small, the diffusion equation smooths the image like a low-pass filter, resulting in a strong smoothing effect. When the gradient is large, the diffusion behavior of the diffusion equation becomes less pronounced, minimizing or even eliminating smoothing, thereby preserving the edge information of the image.

[0024] As a further improvement to the present invention, the specific content of step 2 is as follows.

[0025] Step 2.1, Design of an instance segmentation network based on DCNN;

[0026] Step 2.2, customize the feature scale;

[0027] Step 2.3, Anchor frame dimension design;

[0028] Step 2.4, optimize network strategy design.

[0029] As a further improvement to the present invention, the specific content of step 2.1 is as follows:

[0030] Step 2.1.1, Backbone Architecture Design: Introduce Feature Pyramid Network, i.e., FPN to extend the backbone network, so that the target can be better represented at multiple scales. FPN improves the performance of the standard feature extraction pyramid by adding a second pyramid. The second pyramid can select high-level features from the first pyramid and pass them to the lower level. Through this process, it allows the features at each level to be combined with high-level and low-level features.

[0031] Step 2.2.2, Region Proposal Module Design: The Region Proposal Module, or RPN, is a lightweight neural network that uses a sliding window to scan the image and find regions where the target exists. The regions scanned by the RPN are called anchor points, which are rectangles distributed across the image region and overlap each other to cover the image as much as possible. The RPN scans these anchor points very quickly. The sliding window is implemented by the convolution process of the RPN, and all regions can be scanned in parallel using a GPU. The RPN does not scan the image directly, but scans the backbone feature map, which allows the RPN to effectively reuse the extracted features and avoid redundant computation.

[0032] Step 2.3.3, ROI Classifier and Bounding Box Regressor Design: This stage runs on the proposed regions, i.e., ROIs, proposed by the RPN. Like the RPN, it generates two outputs for each ROI: a. Class and b. Bounding Box Fine-tuning. a. Class is the class of the target within the ROI; unlike the RPN, this network is deeper and can classify regions into specific classes. b. Bounding Box Fine-tuning: Its goal is to further refine the position and size of the bounding boxes to encapsulate the target.

[0033] Step 2.4.4, Region Contour Design: The region contour branch is a convolutional network that takes the positive regions selected by the ROI classifier as input and generates their masks. The generated masks are low-resolution. Pixels, but these are soft masks represented by floating-point numbers, offering more detail compared to binary masks; the small size of the masks helps maintain the lightweight nature of the mask branch network, shrinking the real mask to a smaller size during training. The loss function is calculated, and during the inference process, the predicted mask is enlarged to the size of the ROI bounding box to give the final mask result so that each target has a region outline.

[0034] As a further improvement to the present invention, the specific content of step 2.2 is as follows:

[0035] Darknet53 with CSP was selected as the backbone network of the detection network. It adopts CSP structure, SPP and PAN structure to enhance the ability to describe features.

[0036] In the CSP structure , , , These represent the width, height, number of input channels, and number of output channels of the feature map, respectively. The input features are directly transformed using two 1x1 convolutions. Transforming all input features using two 1x1 convolutions further improves feature reusability compared to directly dividing channels, and effectively halves the number of channels before inputting to subsequent modules, reducing computational cost. The purpose of this channel separation design is to maximize the difference in gradient joints. It uses gradient flow truncation to prevent different layers from learning duplicate gradient information.

[0037] Adding the SPP structure after the convolutional layer transforms different feature maps into fixed feature maps, ignoring the input size and producing a fixed-length output. It uses spatial containers of multiple scales instead of a single-size sliding window for pooling. SPP extracts features at different sizes, and experiments show that this approach can improve recognition accuracy.

[0038] As a further improvement to the present invention, the specific content of step 2.3 is as follows:

[0039] The PAN structure is used as the feature fusion structure. FPN is top-down, passing down strong semantic features from higher layers to enhance the entire pyramid. However, it only enhances semantic information and does not pass on localization information. PAN addresses this by adding a bottom-up pyramid after FPN. This operation complements FPN by passing up strong localization features from lower layers. Deeper feature maps carry stronger semantic features and weaker localization information, while shallower feature maps carry stronger positional information and weaker semantic features.

[0040] As a further improvement to the present invention, the specific content of step 2.4 is as follows:

[0041] The design focuses on data augmentation and training loss. Data augmentation will employ the Mosaic approach. On the prediction side, the CIOU loss structure will be used. The loss function for the object detection task consists of two parts: a classification loss function and a regression loss function. The regression loss function is CIOU_Loss.

[0042] CIOU_Loss adds an impact factor that takes into account the aspect ratio of both the predicted bounding box and the target bounding box, as detailed below;

[0043]

[0044] in It is a parameter that measures the consistency of aspect ratio, and can be defined as:

[0045]

[0046] in, These are the width and height of the manually labeled bounding box and the predicted bounding box, respectively; thus, CIOU_Loss takes into account three important geometric factors that the target bounding box regression function should consider: overlap area, center point distance, and aspect ratio.

[0047] As a further improvement to the present invention, the specific content of step 3 is as follows.

[0048] Step 3.1, Algorithm Interface Design

[0049] The algorithm interface is used to transmit images acquired by the front-end acquisition device to the algorithm processing module. Considering that different sensors have different image output methods, the interface design can be compatible with input images of various encoding formats.

[0050] Step 3.2, Algorithm Communication Design

[0051] Algorithm communication determines how the interface and algorithm interact; the interface thread and algorithm thread are independent and interact through thread communication. Since the visual algorithm runs on both CPU and GPU, communication and synchronization between CPU thread and GPU thread should also be designed.

[0052] Step 3.3, Algorithm Input and Output Variable Design

[0053] The algorithm contains a large number of custom parameters to adapt to different sensors and sample types; therefore, a separate input area should be designed on the interface to provide staff with relevant information; the number, precision and data format of parameters should be agreed upon; at the same time, the variables output by the algorithm include numerical and image data, and corresponding areas should also be designed on the interface. Attached Figure Description

[0054] Figure 1 This is a flowchart of the present invention.

[0055] Figure 2 This is a segmentation framework diagram for an example of the present invention.

[0056] Figure 3 This is an example diagram of the backbone structure of the standard convolutional network in this paper.

[0057] Figure 4 This is a schematic diagram of the CSP structure of the present invention.

[0058] Figure 5 This is a schematic diagram of the SPP structure of the present invention.

[0059] Figure 6 This is an example diagram of the PAN structure of the present invention.

[0060] Figure 7 This is a schematic diagram illustrating the IOU loss definition of the present invention.

[0061] Figure 8 This is a schematic diagram of GIOU loss according to the present invention.

[0062] Figure 9 This is a schematic diagram of the DIOU loss of the present invention. Detailed Implementation

[0063] The present invention will be further described below with reference to the accompanying drawings:

[0064] like Figure 1-9 The method for testing live oil solid phase deposition using full-view image metrology includes the following steps:

[0065] Step 1, Image preprocessing;

[0066] Step 1.1, Input Image: The image is acquired and input via a host computer through a light source system and a CCD industrial camera system;

[0067] Step 1.2, Image Storage: Images are acquired in real time. The software can be set to save images to a specified directory at regular intervals. The image analysis software automatically processes the saved images in a timely manner to achieve real-time processing. Images are saved sequentially by time and number. The image analysis software continuously identifies the newly output image numbers in the specified directory and processes them in real time.

[0068] Step 1.3, Binarization: The Otus algorithm, also known as the maximum inter-class variance method, is used to automatically select the binarization threshold.

[0069] Step 1.4, Noise Preprocessing: The median filtering preprocessing algorithm is used, and the anisotropic diffusion smoothing filtering technique is used to smooth and denoise the sediment images to be processed.

[0070] Step 1.4.1, targeting Median filtering first calculates the maximum, median, and minimum values ​​for each column within the window, resulting in three sets of data.

[0071] Maximum value group: Max0 = max[P0, P3, P6], Max1 = max[P1, P4, P7], Max2 = max[P2, P5, P8]

[0072] Median groups: Med0 = med[P0, P3, P6], Med1 = med[P1, P4, P7], Med2 = med[P2, P5, P8]

[0073] Minimum value set: Min0 = Min[P0, P3, P6], Min1 = Min[P1, P4, P7], Min2 = max[P2, P5, P8]

[0074] in for From top left to bottom right in the region 1 pixel;

[0075] In this comparison, the maximum value in the maximum value group and the minimum value in the minimum value group must be the maximum and minimum values ​​among the 9 elements, and cannot be the median, leaving 7 elements. In the median value group, the maximum value must be at least 5 pixels greater than the minimum value, and cannot be the median, leaving 5 elements. In the maximum value group, the median value must be at least 5 elements greater than the minimum value group, and cannot be the median, leaving 3 elements to compare: the minimum value in the maximum value group, the median value in the median value group, and the maximum value in the minimum value group. The median of these three values ​​is the median of the 9 elements.

[0076] Step 1.4.2: Considering that median filtering cannot effectively filter continuously generated noise, while anisotropic filtering can further smooth homogeneous regions, anisotropic filtering algorithm is introduced here.

[0077] Based on the heat conduction equation, an anisotropic diffusion equation can be established and applied to image smoothing in the medium-scale space of image processing. The diffusion equation for a two-dimensional image signal is:

[0078]

[0079] in, It is the original image. It is spatial information about the diffusion process. This indicates the step size, and also represents the time information of the diffusion process. and Let them represent the divergence operator and the gradient operator, respectively. It is the diffusion coefficient, which is defined as:

[0080]

[0081] The formula indicates that the diffusion coefficient is a monotonically decreasing function with respect to the gradient magnitude. When the gradient is small, the diffusion equation smooths the image like a low-pass filter, resulting in a strong smoothing effect. When the gradient is large, the diffusion behavior of the diffusion equation becomes less pronounced, minimizing or even eliminating smoothing, thereby preserving the edge information of the image.

[0082] Step 2, image detection and recognition;

[0083] Step 2.1, Design of an instance segmentation network based on DCNN;

[0084] Step 2.1.1, Backbone Architecture Design: Introduce Feature Pyramid Network, i.e., FPN to extend the backbone network, so that the target can be better represented at multiple scales. FPN improves the performance of the standard feature extraction pyramid by adding a second pyramid. The second pyramid can select high-level features from the first pyramid and pass them to the lower level. Through this process, it allows the features at each level to be combined with high-level and low-level features.

[0085] Step 2.2.2, Region Proposal Module Design: The Region Proposal Module, or RPN, is a lightweight neural network that uses a sliding window to scan the image and find regions where the target exists. The regions scanned by the RPN are called anchor points, which are rectangles distributed across the image region and overlap each other to cover the image as much as possible. The RPN scans these anchor points very quickly. The sliding window is implemented by the convolution process of the RPN, and all regions can be scanned in parallel using a GPU. The RPN does not scan the image directly, but scans the backbone feature map, which allows the RPN to effectively reuse the extracted features and avoid redundant computation.

[0086] Step 2.3.3, ROI Classifier and Bounding Box Regressor Design: This stage runs on the proposed regions, i.e., ROIs, proposed by the RPN. Like the RPN, it generates two outputs for each ROI: a. Class and b. Bounding Box Fine-tuning. a. Class is the class of the target within the ROI; unlike the RPN, this network is deeper and can classify regions into specific classes. b. Bounding Box Fine-tuning: Its goal is to further refine the position and size of the bounding boxes to encapsulate the target.

[0087] Step 2.4.4, Region Contour Design: The region contour branch is a convolutional network that takes the positive regions selected by the ROI classifier as input and generates their masks. The generated masks are low-resolution. Pixels, but these are soft masks represented by floating-point numbers, offering more detail compared to binary masks; the small size of the masks helps maintain the lightweight nature of the mask branch network, shrinking the real mask to a smaller size during training. The loss function is calculated, and during the inference process, the predicted mask is enlarged to the size of the ROI bounding box to give the final mask result so that each target has a region outline.

[0088] Step 2.2, customize the feature scale;

[0089] Darknet53 with CSP was selected as the backbone network of the detection network. It adopts CSP structure, SPP and PAN structure to enhance the ability to describe features.

[0090] In the CSP structure , , , These represent the width, height, number of input channels, and number of output channels of the feature map, respectively. The input features are directly transformed using two 1x1 convolutions. Transforming all input features using two 1x1 convolutions further improves feature reusability compared to directly dividing channels, and effectively halves the number of channels before inputting to subsequent modules, reducing computational cost. The purpose of this channel separation design is to maximize the difference in gradient joints. It uses gradient flow truncation to prevent different layers from learning duplicate gradient information.

[0091] Adding the SPP structure after the convolutional layer transforms different feature maps into fixed feature maps, ignoring the input size and producing a fixed-length output. It uses spatial containers of multiple scales instead of a single-size sliding window for pooling. SPP extracts features at different sizes, and experiments show that this approach can improve recognition accuracy.

[0092] Step 2.3, Anchor frame dimension design;

[0093] The PAN structure is used as the feature fusion structure. FPN is top-down, passing down strong semantic features from higher layers to enhance the entire pyramid. However, it only enhances semantic information and does not pass on localization information. PAN addresses this by adding a bottom-up pyramid after FPN. This operation complements FPN by passing up strong localization features from lower layers. Deeper feature maps carry stronger semantic features and weaker localization information, while shallower feature maps carry stronger positional information and weaker semantic features.

[0094] Step 2.4, optimize network strategy design.

[0095] The design focuses on data augmentation and training loss. Data augmentation will employ the Mosaic approach. On the prediction side, the CIOU loss structure will be used. The loss function for the object detection task consists of two parts: a classification loss function and a regression loss function. The regression loss function is CIOU_Loss.

[0096] CIOU_Loss adds an impact factor that takes into account the aspect ratio of both the predicted bounding box and the target bounding box, as detailed below;

[0097]

[0098] in It is a parameter that measures the consistency of aspect ratio, and can be defined as:

[0099]

[0100] in, These are the width and height of the manually labeled bounding box and the predicted bounding box, respectively; thus, CIOU_Loss takes into account three important geometric factors that the target bounding box regression function should consider: overlap area, center point distance, and aspect ratio.

[0101] Step 3, Human-computer interaction.

[0102] Step 3.1, Algorithm Interface Design

[0103] The algorithm interface is used to transmit images acquired by the front-end acquisition device to the algorithm processing module. Considering that different sensors have different image output methods, the interface design can be compatible with input images of various encoding formats.

[0104] Step 3.2, Algorithm Communication Design

[0105] Algorithm communication determines how the interface and algorithm interact; the interface thread and algorithm thread are independent and interact through thread communication. Since the visual algorithm runs on both CPU and GPU, communication and synchronization between CPU thread and GPU thread should also be designed.

[0106] Step 3.3, Algorithm Input and Output Variable Design

[0107] The algorithm contains a large number of custom parameters to adapt to different sensors and sample types; therefore, a separate input area should be designed on the interface to provide staff with relevant information; the number, precision and data format of parameters should be agreed upon; at the same time, the variables output by the algorithm include numerical and image data, and corresponding areas should also be designed on the interface.

[0108] This invention mainly includes the following three parts:

[0109] Image preprocessing techniques.

[0110] Solid-state imaging suffers from high noise levels and low resolution, necessitating image preprocessing to facilitate subsequent target search. Previous studies explored various preprocessing algorithms, including nonlinear logarithmic expansion, histogram normalization, mean filtering, median filtering, and exponential filtering. Comparative testing revealed that the median filtering algorithm demonstrated superior performance in both preprocessing efficiency and speed.

[0111] Common template sizes for median filtering include , , and Several. This invention uses The template is as follows:

[0112] The key to calculating the median lies in sorting the pixels within the sliding window; the choice of sorting algorithm is a crucial factor influencing the median filtering algorithm. Traditional sorting algorithms are based on bubble sort. If the number of pixels in the window is... Then each window needs to be sorted The comparison operation for each pixel has a time complexity of O(n). Furthermore, conventional filtering algorithms require a sorting process every time the window is moved. When the size of an image is... When that happens, the entire calculation will require When the window is large, the computational workload is significant and time-consuming.

[0113] To improve the implementation speed of median filtering, for Median filtering: A fast parallel median filtering method is proposed.

[0114] First, calculate the maximum, median, and minimum values ​​for each column within the window, thus obtaining three sets of data:

[0115] Maximum value group: Max0 = max[P0, P3, P6], Max1 = max[P1, P4, P7], Max2 = max[P2, P5, P8]

[0116] Median groups: Med0 = med[P0, P3, P6], Med1 = med[P1, P4, P7], Med2 = med[P2, P5, P8]

[0117] Minimum value set: Min0 = Min[P0, P3, P6], Min1 = Min[P1, P4, P7], Min2 = max[P2, P5, P8]

[0118] in for From top left to bottom right in the region Each pixel represents a value. Therefore, we can observe that the maximum value in the maximum group and the minimum value in the minimum group must be the maximum and minimum values ​​among the nine elements, and cannot be the median, leaving seven. The maximum value in the median group is at least greater than five pixels, and the minimum value is at least less than five pixels, so it cannot be the median, leaving five. The median value in the maximum group is at least greater than five elements, and the median value in the minimum group is at least less than five elements, so it cannot be the median. Finally, three elements remain to be compared: the minimum value in the maximum group, the median value in the median group, and the maximum value in the minimum group. The median of these three values ​​is the median of the nine elements.

[0119] Considering that median filtering is not effective at filtering continuously generated noise, while anisotropic filtering can further smooth homogeneous regions, anisotropic filtering algorithms are introduced here.

[0120] Based on the heat conduction equation, an anisotropic diffusion equation can be established and applied to image smoothing in the medium-scale space of image processing. The diffusion equation for a two-dimensional image signal is:

[0121]

[0122] in, It is the original image. It is spatial information about the diffusion process. This indicates the step size, and also represents the time information of the diffusion process. and Let them represent the divergence operator and the gradient operator, respectively. It is the diffusion coefficient, which is defined as:

[0123]

[0124] The formula indicates that the diffusion coefficient is a monotonically decreasing function with respect to the gradient magnitude. When the gradient is small, the diffusion equation smooths the image like a low-pass filter, resulting in a strong smoothing effect. When the gradient is large, the diffusion behavior of the diffusion equation becomes less pronounced, minimizing or even eliminating smoothing, thereby preserving the edge information of the image.

[0125] Anisotropic diffusion smoothing filtering was used to smooth and denoise solid images. Anisotropic diffusion smoothing filtering can effectively remove noise while preserving image edges and important details.

[0126] II. Instance segmentation algorithm based on DCNN.

[0127] Measuring the geometric properties and location information of a single solid object in an image requires completely extracting the region where the solid object is located within the image. Considering the uneven color and grayscale intensity of the solid object itself, and the lack of sharp edges separating it from the background region, this invention employs an instance segmentation method based on deep convolutional networks to accurately extract the region where a single solid object is located in the image.

[0128] In the past two years, the field of deep learning computer vision has seen new developments. In existing technologies, deep neural networks are often single-task applications, such as image classification (AlexNet, VGG16, etc.), image segmentation (multiple papers represented by FCN), and object detection (R-CNN, Fast R-CNN, and Faster R-CNN, as well as later YOLO and SSD, etc.). These tasks are often integrated into a single framework, such as instance segmentation. Instance segmentation is similar to object detection, but object detection typically outputs the minimum bounding box of the object's region, while instance segmentation outputs a set of pixels / masks that accurately describe the object. Instance segmentation also differs from semantic segmentation; it doesn't require labeling each pixel. It only needs to find the edge contours of the object of interest. Since different objects have their own outer contours, instance segmentation can distinguish individual objects. As can be seen, instance segmentation has three main advantages:

[0129] a) Separating the smallest bounding rectangle region of an object from the background, i.e., object detection;

[0130] b) Extract the detected objects pixel by pixel, i.e., image segmentation;

[0131] c) Classify the detected objects into categories, i.e., image classification.

[0132] The instance segmentation framework used in this invention is as follows: Figure 2 As shown:

[0133] Template-based detection convolutional networks are a two-stage framework. The first stage scans the image and generates proposals (i.e., regions that may contain an object). The second stage classifies the proposals and generates bounding boxes and masks. The method proposed in this invention extends Faster R-CNN, proposed in 2016. Faster R-CNN is a popular object detection framework that includes the following main modules:

[0134] Backbone architecture, such as Figure 3 Example of a standard convolutional network backbone structure shown.

[0135] like Figure 3It uses a standard convolutional neural network (typically ResNet50 or ResNet101) as a feature extractor. The lower layers detect low-level features (edges and corners, etc.), while higher layers detect higher-level features. After forward propagation through the backbone network, features of size... Taking an image as an example, the tensor is transformed into a shape of The feature map will serve as input for the next stage. This project aims to further enhance the backbone network. A Feature Pyramid Network (FPN) is introduced to extend the backbone network, thus enabling better representation of the target at multiple scales. The FPN improves the performance of the standard feature extraction pyramid by adding a second pyramid that selects high-level features from the first pyramid and passes them down to the lower levels. This process allows features at each level to be combined with both high-level and low-level features.

[0136] Region Recommendation Module (RPN)

[0137] RPN is a lightweight neural network that uses a sliding window to scan an image and find regions where targets exist. The regions scanned by RPN are called anchors, which are rectangles distributed across the image area. In fact, with different sizes and aspect ratios, there are nearly 200,000 anchors on an image, overlapping each other to cover as much of the image as possible. RPN scans these anchors very quickly. The sliding window is implemented by the convolution process of RPN, allowing all regions to be scanned in parallel using a GPU. Furthermore, RPN does not directly scan the image but instead scans the backbone feature map. This allows RPN to efficiently reuse extracted features and avoid redundant computation. Through these optimizations, RPN can complete a scan within 10ms.

[0138] ROI classifier and bounding box regressor

[0139] This stage operates on the proposed Regions of Interest (ROIs) proposed by the RPN. Like the RPN, it generates two outputs for each ROI: a. Category: The category of the target within the ROI. Unlike the RPN (two categories, foreground or background), this network is deeper and can classify regions into specific categories (various defects, etc.). b. Boundary Fine-tuning: Similar to the RPN, its goal is to further refine the position and size of the bounding boxes to encapsulate the target.

[0140] Region outline

[0141] The region contour branch is a convolutional network that takes the positive regions selected by the ROI classifier as input and generates their masks. The generated masks are low-resolution. Pixels. But these are soft masks represented by floating-point numbers, offering more detail compared to binary masks. The small size of the mask helps maintain the lightweight nature of the mask branch network. During training, the real mask is scaled down to... The loss function is calculated, and during the inference process, the predicted mask is enlarged to the size of the ROI bounding box to give the final mask result so that each target has a region outline.

[0142] Backbone Network Design

[0143] This invention selects Darknet53, which features Cross-stage Partial Connections (CSP), as the backbone network for the detection network. Compared to the traditional Darknet53, it employs CSP, SPP, and PAN structures to enhance its feature description capabilities. These will be described in detail below.

[0144] Figure 4 CSP structure diagram

[0145] A schematic diagram of the CSP structure is shown below. Figure 4 As shown. Among them, , , , These represent the width, height, number of input channels, and number of output channels of the feature map, respectively. It directly transforms the input features using two 1x1 convolutions. Understandably, transforming all input features using two 1x1 convolutions improves feature reusability compared to directly dividing channels, and effectively halves the number of channels before inputting to subsequent modules, reducing computational cost. Theoretically, the purpose of this channel separation design is to maximize the difference in gradient joints. It uses gradient flow truncation to prevent different layers from learning duplicate gradient information. The principle is that if duplicate gradient learning can be effectively reduced, the network's learning ability can be greatly improved.

[0146] Figure 5 This is a schematic diagram of the SPP structure.

[0147] Traditional feature cropping involves cropping / transforming the original image and then directly feeding it into a convolutional layer before entering a fully connected classification layer. However, the position and size of target objects in each image are difficult to maintain consistently, and cropping / transformation inevitably affects the accuracy of image features. Since the connection between the convolutional layer and the fully connected layer must maintain a fixed dimension, if the image is not cropped / transformed to a uniform size, the extracted feature maps will be of different sizes, making them difficult to feed into the fully connected layer. Therefore, a new structure is needed to solve this problem, and SPP is the most suitable method. Figure 5As shown, this problem can be solved simply by adding it after the convolutional layer. Its function is to transform different feature maps into fixed feature maps, eliminating the need for pre-cropping / transforming the image. SPP has three main characteristics: (1) SPP can ignore the input size and produce a fixed-length output. (2) SPP uses spatial containers of multiple scales instead of using a sliding window of only one size for pooling operations. (3) SPP extracts features at different sizes, which can improve recognition accuracy.

[0148] Feature fusion method design

[0149] This invention uses a PAN structure as the feature fusion structure, such as Figure 7 The example of a PAN structure is shown.

[0150] FPN (Fixed-Pyramid Network) works top-down, passing down strong semantic features from higher layers to enhance the entire pyramid. However, it only enhances semantic information, not localization information. PAN (Programmable Node Network) addresses this by adding a bottom-up pyramid after FPN. This operation complements FPN by passing strong localization features from lower layers upwards. Deeper feature maps carry stronger semantic features but weaker localization information, while shallower feature maps carry stronger location information and weaker semantic features. FPN passes deep semantic features to shallower layers, thus enhancing semantic expression across multiple scales. PAN, on the other hand, transmits shallow localization information to deeper layers, enhancing localization capabilities across multiple scales.

[0151] Network optimization strategy design

[0152] Regarding network optimization strategies, this invention designs data augmentation and training loss. Data augmentation will adopt the Mosaic data augmentation method. It has several advantages: (1) Enriching the dataset: Randomly using 4 images, randomly scaling them, and then randomly distributing them for stitching greatly enriches the detection dataset. In particular, random scaling adds many small targets, making the network more robust. (2) Reducing GPU requirements: With only one GPU, during Mosaic augmentation training, the data of 4 images can be directly calculated, so the sub-batch size does not need to be very large, and one GPU can achieve a relatively good effect.

[0153] On the prediction side, this invention employs a CIOU loss structure. The loss function for object detection tasks generally consists of two parts: a classification loss function and a regression loss function. Starting with the most commonly used IOU_Loss, this loss is defined as the intersection-union ratio (IOU) between the predicted bounding box and the manually labeled bounding box, such as... Figure 7 The diagram illustrates the definition of IOU loss. .

[0154] The problems are: (1) In state 1, when the predicted box and the target box do not intersect, IOU=0, which cannot reflect the distance between the two boxes. At this time, the loss function is not differentiable, and IOU_Loss cannot optimize the case where the two boxes do not intersect. (2) In states 2 and 3, when the two predicted boxes are the same size, the two IOUs are also the same, and IOU_Loss cannot distinguish the different cases where they intersect. Therefore, GIOU_Loss was introduced in 2019 to improve this, such as Figure 8 The diagram shows the U-loss.

[0155] like Figure 8 As shown, GIOU_Loss adds a method for measuring intersection, alleviating the awkwardness of simple IOU_Loss. Its definition is:

[0156] However, a shortcoming exists: states 1 and 2 both involve the predicted bounding box being inside the target bounding box and having the same size. In this case, the difference between the predicted and target bounding boxes is the same, resulting in the same GIOU value for both states. GIOU degenerates into IOU, making it impossible to distinguish relative positional relationships. To address this issue, DIOU_Loss was proposed in 2020. Figure 9 The DIOU loss diagram is shown below.

[0157] DIOU_Loss is defined as: A good bounding box regression function should consider three important geometric factors: overlap area, center point distance, and aspect ratio. Addressing the problems of IOU and GIOU, DIOU considers two aspects: (1) How to minimize the normalized distance between the predicted box and the target box? (2) How to achieve more accurate regression when the predicted box and the target box overlap? For the first question, DIOU_Loss was proposed. DIOU_Loss considers the overlap area and center point distance; when the target box wraps around the predicted box, it directly measures the distance between the two boxes, thus DIOU_Loss converges faster. However, as mentioned earlier regarding good bounding box regression functions, it does not consider the aspect ratio. For example... Figure 9 In the two states where the target box wraps around the predicted box, DIOU_Loss should normally take effect. However, since the center point of the predicted box is always at the same position, according to the calculation formula of DIOU_Loss, both values ​​are the same. To address this issue, this invention proposes CIOU_Loss.

[0158] The formulas for CIOU_Loss and DIOU_Loss are the same, but an additional influence factor is added to take into account the aspect ratios of both the predicted and target boxes.

[0159]

[0160] in It is a parameter that measures the consistency of aspect ratio, and can be defined as:

[0161]

[0162] in, These are the width and height of the manually labeled bounding box and the predicted bounding box, respectively. Thus, CIOU_Loss takes into account three important geometric factors that the target bounding box regression function should consider: overlap area, center point distance, and aspect ratio.

[0163] III. Research and development of human-computer interaction systems.

[0164] A good human-computer interaction system enables staff to quickly familiarize themselves with the system's operation and easily observe the algorithm processing results. This research project will focus on several aspects, including algorithm interface design, algorithm communication design, and algorithm input and output variable design.

[0165] Algorithm Interface Design

[0166] The algorithm interface is used to transmit images acquired by the front-end acquisition device to the algorithm processing module. Considering that different sensors have different image output methods, the interface design should be designed to be compatible with input images of various encoding formats.

[0167] Algorithm Communication Design

[0168] Algorithm communication determines how the interface and the algorithm interact. Considering the relatively complex convolutional network-based visual algorithm used in this R&D project, a separate approach for the interface and algorithm threads will be adopted, with interaction facilitated by thread communication. Since the visual algorithm runs simultaneously on both the CPU and GPU, methods for communication and synchronization between the CPU and GPU threads must also be designed. Proper thread allocation and synchronization are crucial for efficient system operation.

[0169] Algorithm input and output variable design

[0170] The algorithm contains numerous custom parameters to adapt to different sensors and sample types. Therefore, a dedicated input area should be designed on the interface to allow staff to input relevant information. This requires thorough discussion and agreement on the number of parameters, accuracy, and data format during the early stages of the development plan. Furthermore, since the algorithm's output variables include both numerical and image-type data, corresponding areas should be designed within the interface accordingly.

[0171] This invention aims to reduce labor costs, improve work efficiency, enhance the accuracy of quantitative analysis, and promote the consistency of quantitative analysis results through research on its key technologies. It fully considers high-performance, stable, and engineering-practice-suitable algorithms in current computer vision technology and proposes a reasonable architecture for their integration. Ultimately, it achieves an end-to-end solution for "input image - output solid phase geometric properties and position." Specific innovations are as follows:

[0172] Promoting the application of artificial intelligence in the petroleum industry can achieve a win-win situation of improving the industry's technological level and implementing AI technology. Computer vision technology can be used to automate the quantitative analysis of heavy solids precipitation in deep-well crude oil. Currently, solid geometry measurements rely solely on manual methods using basic image processing software. Automating the quantitative analysis of solid geometry using computer vision technology can significantly reduce labor costs and improve efficiency. More importantly, subjective differences in perception among personnel can compromise the consistency of solid matter identification. Visual algorithms can ensure the consistency of detection results.

[0173] This invention improves the frequency of quantitative analysis, enabling full-process observation of the spatiotemporal process of solid phase precipitation. With the support of GPUs, the algorithm of this invention can complete the analysis with near real-time efficiency. Therefore, continuous quantitative analysis of the precipitation process is possible. This provides the possibility of observing the flow state and distribution of crude oil fluids under high temperature and high pressure conditions, and revealing the intrinsic relationship between the timing of crude oil solid phase deposition, deposition conditions, sediment particle size, and total deposition amount.

[0174] This invention is not limited to the above embodiments. Based on the technical solutions disclosed herein, those skilled in the art can make some substitutions and modifications to some of the technical features without creative effort, and all such substitutions and modifications are within the protection scope of this invention.

Claims

1. A method for testing live oil solid phase deposition using full-field image metrology, characterized in that, Includes the following steps: Step 1, Image preprocessing; Step 1.1, Input Image: The image is acquired and input via a host computer through a light source system and a CCD industrial camera system; Step 1.2, Image Storage: Images are acquired in real time. The software can be set to save images to a specified directory at regular intervals. The image analysis software automatically processes the saved images in a timely manner to achieve real-time processing. Images are saved sequentially by time and number. The image analysis software continuously identifies the newly output image numbers in the specified directory and processes them in real time. Step 1.3, Binarization: The Otus algorithm, also known as the maximum inter-class variance method, is used to automatically select the binarization threshold. Step 1.4, Noise Preprocessing: The median filtering preprocessing algorithm is used, and the anisotropic diffusion smoothing filtering technique is used to smooth and denoise the sediment images to be processed. Step 1.4.1, targeting Median filtering first calculates the maximum, median, and minimum values ​​for each column within the window, resulting in three sets of data. Maximum value group: Max0 = max[P0, P3, P6], Max1 = max[P1, P4, P7], Max2 = max[P2, P5, P8]; Median groups: Med0 = med[P0, P3, P6], Med1 = med[P1, P4, P7], Med2 = med[P2, P5, P8]; Minimum value set: Min0 = Min[P0, P3, P6], Min1 = Min[P1, P4, P7], Min2 = max[P2, P5, P8]; in for From top left to bottom right in the region 1 pixel; In this comparison, the maximum value in the maximum value group and the minimum value in the minimum value group must be the maximum and minimum values ​​among the 9 elements, and cannot be the median, leaving 7 elements. In the median value group, the maximum value must be at least 5 pixels greater than the minimum value, and cannot be the median, leaving 5 elements. In the maximum value group, the median value must be at least 5 elements greater than the minimum value group, and cannot be the median, leaving 3 elements to compare: the minimum value in the maximum value group, the median value in the median value group, and the maximum value in the minimum value group. The median of these three values ​​is the median of the 9 elements. Step 1.4.2: Considering that median filtering cannot effectively filter continuously generated noise, while anisotropic filtering can further smooth homogeneous regions, anisotropic filtering algorithm is introduced here. Based on the heat conduction equation, an anisotropic diffusion equation can be established and applied to image smoothing in the medium-scale space of image processing. The diffusion equation for a two-dimensional image signal is: ； in, It is the original image. It is spatial information about the diffusion process. This indicates the step size, and also represents the time information of the diffusion process. and Let them represent the divergence operator and the gradient operator, respectively. It is the diffusion coefficient, which is defined as: ； The formula shows that the diffusion coefficient is a monotonically decreasing function with respect to the gradient magnitude. When the gradient is small, the diffusion equation smooths the image like a low-pass filter, and the smoothing effect is very strong. When the gradient is large, the diffusion behavior of the diffusion equation becomes very inconspicuous, minimizing or even eliminating smoothing, thereby preserving the edge information of the image. Step 2, image detection and recognition; Step 3, Human-computer interaction.

2. The method for testing live oil solid phase deposition using full-field image metrology according to claim 1, characterized in that: The specific details of step 2 are as follows: Step 2.1, Design of an instance segmentation network based on DCNN; Step 2.2, customize the feature scale; Step 2.3, Anchor frame dimension design; Step 2.4, optimize network strategy design.

3. The method for testing live oil solid phase deposition using full-field image metrology according to claim 2, characterized in that: The specific details of step 2.1 are as follows: Step 2.1.1, Backbone Architecture Design: Introduce Feature Pyramid Network, i.e., FPN to extend the backbone network, so that the target can be better represented at multiple scales. FPN improves the performance of the standard feature extraction pyramid by adding a second pyramid. The second pyramid can select high-level features from the first pyramid and pass them to the lower level. Through this process, it allows the features at each level to be combined with high-level and low-level features. Step 2.2.2, Region Proposal Module Design: The Region Proposal Module, or RPN, is a lightweight neural network that uses a sliding window to scan the image and find regions where the target exists. The regions scanned by the RPN are called anchor points, which are rectangles distributed across the image region and overlap each other to cover the image as much as possible. The RPN scans these anchor points very quickly. The sliding window is implemented by the convolution process of the RPN, and all regions can be scanned in parallel using a GPU. The RPN does not scan the image directly, but scans the backbone feature map, which allows the RPN to effectively reuse the extracted features and avoid redundant computation. Step 2.3.3, ROI Classifier and Bounding Box Regressor Design: This stage runs on the proposed regions, i.e., ROIs, proposed by the RPN. Like the RPN, it generates two outputs for each ROI: a. Class and b. Bounding Box Fine-tuning. a. Class is the class of the target within the ROI; unlike the RPN, this network is deeper and can classify regions into specific classes. b. Bounding Box Fine-tuning: Its goal is to further refine the position and size of the bounding boxes to encapsulate the target. Step 2.4.4, Region Contour Design: The region contour branch is a convolutional network that takes the positive regions selected by the ROI classifier as input and generates their masks. The generated masks are low-resolution. Pixels, but these are soft masks represented by floating-point numbers, offering more detail compared to binary masks; the small size of the masks helps maintain the lightweight nature of the mask branch network, shrinking the real mask to a smaller size during training. The loss function is calculated, and during the inference process, the predicted mask is enlarged to the size of the ROI bounding box to give the final mask result so that each target has a region outline.

4. The test method for full-field image metrology of active oil solid phase deposition according to claim 3, characterized in that: The specific content of step 2.2 is as follows: Darknet53 with CSP was selected as the backbone network of the detection network. It adopts CSP structure, SPP and PAN structure to enhance the ability to describe features. In the CSP structure , , , These represent the width, height, number of input channels, and number of output channels of the feature map, respectively. The input features are directly transformed using two-way 1x1 convolutions. Transforming all input features using two-way 1x1 convolutions further improves feature reusability compared to directly dividing channels, and also halves the number of channels before inputting to subsequent modules, reducing computation. The purpose of designing this type of channel separation is to maximize the difference in gradient joint. It uses gradient flow truncation to prevent different layers from learning duplicate gradient information; Adding the SPP structure after the convolutional layer transforms different feature maps into fixed feature maps, ignoring the input size and producing a fixed-length output. It uses spatial containers of multiple scales instead of a single-size sliding window for pooling. SPP extracts features at different sizes, and experiments show that this approach can improve recognition accuracy.

5. The test method for full-field image metrology of active oil solid phase deposition according to claim 4, characterized in that: The specific details of step 2.3 are as follows: The PAN structure is used as the feature fusion structure. FPN is top-down, passing down strong semantic features from higher layers to enhance the entire pyramid. However, it only enhances semantic information and does not pass on localization information. PAN addresses this by adding a bottom-up pyramid after FPN. This operation complements FPN by passing up strong localization features from lower layers. Deeper feature maps carry stronger semantic features and weaker localization information, while shallower feature maps carry stronger positional information and weaker semantic features.

6. The test method for full-field image metrology of active oil solid phase deposition according to claim 5, characterized in that: The specific content of step 2.4 is as follows: The design focuses on data augmentation and training loss. Data augmentation will employ the Mosaic approach. On the prediction side, the CIOU loss structure will be used. The loss function for the object detection task consists of two parts: a classification loss function and a regression loss function. The regression loss function is CIOU_Loss. CIOU_Loss adds an impact factor that takes into account the aspect ratio of both the predicted bounding box and the target bounding box, as detailed below; ； in It is a parameter that measures the consistency of aspect ratio, and can be defined as: ； in, These are the width and height of the manually labeled bounding box and the predicted bounding box, respectively; thus, CIOU_Loss takes into account three important geometric factors that the target bounding box regression function should consider: overlap area, center point distance, and aspect ratio.

7. The method for testing live oil solid phase deposition using full-field image metrology according to claim 6, characterized in that: The specific details of step 3 are as follows: Step 3.1, Algorithm Interface Design; The algorithm interface is used to transmit images acquired by the front-end acquisition device to the algorithm processing module. Considering that different sensors have different image output methods, the interface design can be compatible with input images of various encoding formats. Step 3.2, Algorithm Communication Design; Algorithm communication determines how the interface and algorithm interact; the interface thread and algorithm thread are independent and interact through thread communication. Since the visual algorithm runs on both CPU and GPU, communication and synchronization between CPU thread and GPU thread should also be designed. Step 3.3, Algorithm input and output variable design; The algorithm contains a large number of custom parameters to adapt to different sensors and sample types; therefore, a separate input area should be designed on the interface to allow staff to input relevant information. The number, precision, and data format of parameters should be agreed upon. In addition, the variables output by the algorithm include both numerical and image data, and the corresponding areas should be designed in the interface accordingly.

Images