A generative data augmentation method for petrochemical equipment defect detection, device and medium

Abstract

The application discloses a generative data enhancement method for petrochemical equipment defect detection, equipment and medium, relates to the field of image generation. The method acquires a real defect mask and extracts a defect content image Transformer The network extracts content features and style features; the features are fused by using a cross attention mechanism, and a generated image with a defect structure is reconstructed through a decoder; a fusion energy function is constructed based on a gradient relationship, a defect region is embedded into a target background, and a fusion defect image with continuous boundaries is obtained; finally, a comprehensive score is calculated through a confidence degree output by a pre-training model and a Jaccard index, and high-quality samples are reserved for constructing a final training set. The application solves the problems of poor data authenticity, inconsistent distribution and difficult quality control in the prior art, and greatly improves the defect recognition accuracy and generalization ability of the model in a complex industrial scene.

Description

Technical Field

[0001] This invention relates to the field of image generation, and more specifically to a generative data augmentation method, device, and medium for defect detection in petrochemical equipment. Background Technology

[0002] The statements in this section are provided only as background information in connection with this disclosure and may not constitute prior art.

[0003] With the rapid development of industrial visual inspection, intelligent manufacturing, and petrochemical equipment safety monitoring, deep learning technology has shown great application potential in industrial equipment defect detection (such as tank corrosion, pipeline cracks, and reactor structural damage). During the training process of deep learning models, the scale, realism, and consistency of data distribution of the training data are crucial to the accuracy of the model in ultimately performing defect identification and localization tasks.

[0004] However, in real-world petrochemical industrial settings, severe defects occur infrequently, resulting in a significant shortage of real defect samples that can be collected. Existing data augmentation techniques (such as cropping and flipping based on traditional geometric transformations, or simple image stitching algorithms) often struggle to handle complex industrial scenarios. Specifically, existing technologies suffer from the following main technical shortcomings: 1. Poor visual realism of synthetic samples: Simple image stitching cannot handle the differences in lighting and texture between defective areas and the background of the device, and is prone to producing obvious boundary abrupt changes and artifacts.

[0005] 2. The structural consistency of defect morphology is difficult to maintain: using conventional generative adversarial networks (GANs) GAN When performing style transfer, the topological structure of the original defects is often destroyed while transferring device textures, leading to the generation of defects. " distortion ” .

[0006] 3. The quality of generated samples is difficult to control and the data distribution is inconsistent: Existing generative augmentation methods lack targeted quality screening mechanisms, resulting in a large number of low-quality samples or samples whose distribution is inconsistent with the real scene. " Pseudo data ” Mixing into the training set not only fails to improve model performance, but also causes a decrease in the model's detection accuracy. Summary of the Invention

[0007] The purpose of this invention is to address the technical problems in existing technologies, such as insufficient number of real defect samples, inconsistency between the distribution of synthetic data and real-world scenarios, and difficulty in controlling the quality of generated samples. This invention aims to provide a generative data augmentation method, device, and medium for defect detection in petrochemical equipment. This invention enables the controlled migration and generation of defect structures in different equipment backgrounds while ensuring extremely high visual realism, thereby providing high-quality, consistently distributed training data support for industrial defect detection models.

[0008] The technical solution of the present invention is as follows: A generative data augmentation method for defect detection in petrochemical equipment includes: Acquire real defect images and their corresponding defect masks, as well as images of petrochemical equipment; perform pixel-level separation processing on the real defect images and the defect masks to extract defect content images; The defect content image is input into a content encoder for feature extraction to obtain defect content features; and the petrochemical equipment image is input into a style encoder for feature extraction to obtain equipment style features. By using a cross-attention mechanism, the defective content features are used as query vectors, and the device style features are used as key vectors and value vectors for multi-level feature fusion to obtain fused features; The fused features are input into the image decoder for reconstruction representation to generate a generated defective image with defective structure; wherein, the content encoder, style encoder and image decoder are pre-trained based on a joint loss function composed of content loss and style loss; Using the petrochemical equipment image as a background image, a fusion energy function is constructed based on the gradient relationship between the generated defect image and the background image. By minimizing the fusion energy function, the defect region in the generated defect image is embedded into the background image to obtain a fused defect image that maintains boundary continuity. The fused defect image is input into a pre-trained detection model, which outputs a predicted bounding box and its corresponding detection confidence. The cross-union ratio (CUP) between the predicted bounding box and the ground truth bounding box generated based on the defect mask is calculated, and the comprehensive score of the fused defect image is calculated by combining the detection confidence and the CUP. When the overall score is greater than the preset screening threshold, the corresponding fusion defect image will be retained as the target generated sample. All the retained target generated samples are merged with the real defect dataset to construct the final training dataset, and the defect detection model for petrochemical equipment is trained based on the final training dataset.

[0009] Further, the step of inputting the defect content image into a content encoder for feature extraction to obtain defect content features; and inputting the petrochemical equipment image into a style encoder for feature extraction to obtain equipment style features, includes: The defect content image is divided into several image blocks and mapped to a feature space to obtain a content image block sequence. This content image block sequence is then input into a multi-layer... Transformer The defective content features are output from the content encoder, which is composed of structural components. The petrochemical equipment image is divided into several image blocks and mapped to a feature space to obtain an equipment image block sequence. This equipment image block sequence is then input into a multi-layer... Transformer The style encoder, composed of structural components, outputs the style characteristics of the device.

[0010] Furthermore, the method utilizes a cross-attention mechanism, using the defective content features as a query vector and the device style features as key and value vectors for multi-level feature fusion to obtain fused features, including: The defect content features are mapped and calculated to obtain a query matrix; and the device style features are mapped and calculated respectively to obtain the corresponding key matrix and value matrix. The cross-attention weight matrix is ​​calculated by combining the query matrix, the key matrix, and the preset feature dimensions; The product of the cross-attention weight matrix and the value matrix is ​​calculated, and the above fusion process is iterated in the multi-layer cross-attention module to obtain the final fusion feature.

[0011] Furthermore, the joint loss function is constructed by calculating the following formula:

[0012] in: This is the total loss function; This results in a loss of content consistency. For loss of style consistency; The preset first weighting coefficient; This is the preset second weighting coefficient.

[0013] Furthermore, the formula for calculating the content consistency loss is as follows:

[0014] in: The generated content features are extracted after the generated defective image is re-inputted into the content encoder; The defect content features are as described above; The formula for calculating the style consistency loss is as follows:

[0015] in: The generated style features are extracted after the generated defective image is re-inputted into the style encoder; This refers to the style characteristics of the device.

[0016] Further, the step of using the petrochemical equipment image as a background image and constructing a fusion energy function based on the gradient relationship between the generated defect image and the background image includes: Extract the defect region indicated by the defect mask, and construct the fusion energy function using the gradient of the generated defect image within the defect region and the gradient of the fused defect image within the defect region:

[0017] in: For the fusion energy function; The set of pixels in the defective region; For pixel position; The gradient of the fusion defect image to be optimized; The gradient for generating the defect image; Using the boundary pixel values ​​of the corresponding defect region in the background image as boundary conditions, the fusion energy function is minimized iteratively to obtain the fused defect image that maintains boundary continuity.

[0018] Further, the step of calculating the comprehensive score of the fused defect image by combining the detection confidence score and the cross-union ratio includes: The maximum detection confidence is extracted from the prediction bounding box output by the pre-trained detection model as the overall confidence. Based on the defect mask, a true defect bounding box is generated, and the intersection-union ratio (IUU) between the predicted bounding box and the true defect bounding box is calculated. The overall score is calculated using the following formula:

[0019] in: For comprehensive scoring; For overall confidence level; For intersection, union, and comparison; This is the preset third weighting coefficient; This is the preset fourth weighting coefficient.

[0020] Furthermore, the defect detection model for petrochemical equipment is trained based on the final training dataset, and its optimization objective satisfies the following formula:

[0021] in: These are the optimal parameters for the final trained defect detection model; These are the parameters to be optimized in the model; For parameters Defect detection model; For detection loss function; For the final training dataset Sample images in the database.

[0022] The present invention also proposes an electronic device, comprising: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor executes the instructions stored in the memory to perform the method described above.

[0023] The present invention also proposes a computer-readable storage medium for storing instructions that, when executed, cause the method described above to be implemented.

[0024] Compared with existing technologies, the advantages of this invention are: 1. Stable migration and consistency maintenance of defect structures. This invention achieves stable migration and consistency maintenance of defect structures by masking and separating the actual defect areas using multiple layers. Transformer The network extracts defect content features and device style features separately, and innovatively utilizes a cross-attention mechanism (using defect content as the query vector and device style as the key). / Feature fusion is achieved using value vectors. This mechanism enables the robust maintenance of the original defect topology during the transfer of textures from the target device, avoiding the structural distortion problems caused by traditional generative networks.

[0025] 2. Extremely high visual realism and seamless boundary transitions. After generating the initial defect image, this invention does not directly replace pixels. Instead, it uses the petrochemical equipment image as the background and constructs a fusion energy function by constraining the gradient relationship between the generated defect region and the equipment background. Through iterative solving in the Poisson gradient domain, the defect region maintains excellent boundary continuity and lighting texture consistency when embedded in the target background, completely eliminating stitching artifacts.

[0026] 3. Highly reliable closed-loop control of sample quality. This invention introduces a pre-trained detection model as a sample selection engine, utilizing the maximum detection confidence score and the intersection-union ratio of the true bounding boxes (IU / U). IoU The dual-indicator calculation of the comprehensive score ensures that all generated samples that deviate from the true defect distribution are of high quality, thus guaranteeing that all data ultimately added to the training set are of high quality. " High value, high realism ” Valid samples.

[0027] 4. Significantly improves the robustness of detection in real-world industrial scenarios. This invention effectively expands the scale of training data in complex scenarios, compensating for the shortcomings of small sample data at petrochemical equipment sites. By training the final model using the enhanced dataset, the recall and accuracy of the detection system can be significantly improved under different equipment materials, varying lighting conditions, and complex environments. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0029] Figure 1 This is a flowchart of a generative data augmentation method for defect detection in petrochemical equipment. Figure 2 A schematic diagram of the microstructure of the cross-attention module provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of the micro-network structure of the image decoder provided in an embodiment of the present invention.

[0030] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0031] It should be noted that the terminology " First ” and " second ”Relational terms like "etc." are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any actual relationship or order between these entities or operations. Furthermore, the term... " include ” , " Include ” Or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, the statement... " Including one ……” The definition of a specific element does not preclude the presence of other identical elements in a process, method, article, or apparatus that includes the element.

[0032] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0033] Example 1 This embodiment provides a generative data augmentation method for defect detection in petrochemical equipment. In actual petrochemical industrial scenarios (such as the inspection of equipment like oil and gas pipelines, storage tanks, and reactors), the low frequency of actual defects (such as corrosion, cracks, and deformation) results in a severe shortage of data samples available for training detection models. Furthermore, the texture, lighting, and background of different equipment surfaces vary greatly, making it difficult to guarantee the visual realism and feature distribution consistency of the synthesized data by directly using traditional data augmentation methods (such as flipping, cropping, and simple image stitching).

[0034] Please see Figure 1 To solve the above problems, the overall process of the method in this embodiment is as follows: Step S101: Obtain the real defect image and its corresponding defect mask, as well as the petrochemical equipment image; perform pixel-level separation processing on the real defect image and the defect mask to extract the defect content image; In practical applications, a real-world defect dataset is first constructed. This dataset primarily consists of images captured by petrochemical equipment inspection drones or industrial cameras, as well as some publicly available industrial defect data. Each image in the dataset contains pixel-level annotation information (i.e., a defect mask) for the defect region, represented as follows:

[0035] in: It is a real defect dataset; It is the first Zhang's original, authentic defect image; It is the first Binarized defect mask in Zhang image; It represents the number of images.

[0036] To avoid interference from the original background in subsequent feature transfer, this step extracts a clean defect content image through pixel-by-pixel multiplication. :

[0037] in: Indicates pixel-by-pixel multiplication; Simultaneously, the pixel set of the defective region is extracted:

[0038] in: It is the first A set of pixels in a defective region; It is a mask function; By using mask separation, it can be ensured that subsequent networks target only... " The topology of the defect itself ” Learn by discarding the existing, ineffective background information.

[0039] Step S102: Input the defect content image into the content encoder for feature extraction to obtain defect content features; and input the petrochemical equipment image into the style encoder for feature extraction to obtain equipment style features; Specifically, the defect content diagram The image is divided into several image blocks and mapped to a feature space to obtain a content image block sequence. This content image block sequence is then input into a multi-layer... Transformer In the content encoder composed of the structure, the defective content features are output; the petrochemical equipment image is divided into several image blocks and mapped to a feature space to obtain an equipment image block sequence, and the equipment image block sequence is input into a multi-layer... Transformer The style encoder, composed of structural components, outputs the style characteristics of the device.

[0040] In this embodiment, it should be noted that traditional convolutional neural networks ( CNN When extracting features, many methods focus only on local receptive fields, which can easily lead to the loss of the global continuous structure of long, strip-shaped defects such as cracks during the migration process. Therefore, this embodiment preferably uses 12 layers. Transformer The structure, as an encoder, has its output content features represented as follows:

[0041] in, It is a representation of content features; It consists of 12 floors Transformer The content encoder constitutes the structure; This is a defective content image.

[0042] Taking the content encoding process as an example, in order to adapt Transformer The input requirements are as follows: first, draw the defect content diagram. Divided into Non-overlapping Patch (Image patch), then Patch Embedded representation is:

[0043] in: It is a content image Patch sequence; It is the first indivual Patch Feature vector; yes Patch quantity.

[0044] After 12 floors Transformer After encoding, the output contains content features with global dependencies:

[0045] in: It is the first indivual Patch Content features are represented.

[0046] Similarly, images of defect-free petrochemical equipment. (The material used to provide the target background texture) was also processed as Patch The sequence, through a style encoder with the same structure, extracts the device style features.

[0047] The style encoder also consists of 12 layers. Transformer Composition, its output style characteristics are represented as

[0048] in: It represents stylistic features; It consists of 12 floors Transformer The style encoder is composed of elements; It is a device image; Similarly, device images are divided into Patch sequence:

[0049] in: It is a device image. Patch sequence; It is the first indivual Patch Feature vector; yes Patch quantity; go through Transformer After encoding, the style features are obtained:

[0050] in: It is the first indivual Patch Its stylistic characteristics.

[0051] This is based on Patch The self-attention extraction method lays the foundation for subsequent precise pixel-level style injection.

[0052] Step S103: Using the cross-attention mechanism, the defect content features are used as query vectors, and the device style features are used as key vectors and value vectors for multi-level feature fusion to obtain fused features; Specifically, the defective content features are mapped and calculated to obtain a query matrix; the device style features are mapped and calculated to obtain corresponding key matrices and value matrices; the query matrix, the key matrix, and the preset feature dimensions are combined to calculate a cross-attention weight matrix; the cross-attention weight matrix and the value matrix are multiplied and calculated, and the above fusion process is iterated in the multi-layer cross-attention module to obtain the final fused feature.

[0053] This step is to improve the generated image. " Structural stability ” The core mechanism. Existing GAN Generative adversarial networks (GANs) often lead to distortion of defective morphologies during style transfer. This invention cleverly introduces cross-attention (CPA). Cross-Attention )mechanism: Content features Multiply by the mapping matrix to obtain the query matrix:

[0054] in: It is a query matrix; It is a content feature; It is a query mapping matrix.

[0055] style characteristics Multiply by the mapping matrix to obtain the key matrix Sum matrix ;in, It is a key matrix; It is a stylistic feature; It is a key mapping matrix; It is a value matrix; It is a stylistic feature; It is a value mapping matrix.

[0056] The formula for calculating cross-attention weights is:

[0057] in: It is the attention weight matrix; It is a query matrix; It is a key matrix; It is the feature dimension.

[0058] Then, the single-layer fusion features are calculated:

[0059] in: It is the feature representation after fusion; It is the attention weight matrix; It is a value matrix; To achieve a more thorough fusion, this embodiment preferably stacks 12 layers of cross-attention modules, calculating the final fused features layer by layer:

[0060] in: It is the feature representation after fusion; This represents a fusion module consisting of 12 layers of cross-attention.

[0061] Under this mechanism (Defective structure) Actively go " Query ” And capture k Extract the corresponding features from the (device texture) matching features. V The process involves merging the components. This ensures the quality of the generated result." skeleton ” It still has the shape of the original defect, but " epidermis ” It has been replaced with the textured material of the target petrochemical equipment (such as rusty iron sheets, reflective steel pipes, etc.).

[0062] Furthermore, combined Figure 2 It can be seen that the multi-layer cross-attention module is preferably a stacked 12-layer structure. For each layer of the cross-attention module, the internal data flow and processing procedure are as follows: The input features are first normalized by layers ( Layer Normalization ) processing; then entering the cross-attention layer (where settings are configured) Q= Content characteristics K=V= The style features are fused and calculated, and the result is added to the residual of the original features input to the module. Residual Addition Next, the sum of the residuals is input into the feedforward network. Feedforward Network The feedforward network consists of a linear layer, a linear layer, and a linear layer. ReLU The system consists of activation layers and linear layers; finally, the outputs of the feedforward network are summed again using residuals to obtain the final output of this layer. This design, which includes a dual residual structure, effectively avoids the gradient vanishing problem during the training of 12-layer deep networks and ensures the stable propagation of defective topological features during multi-layer style injection.

[0063] Step S104: Input the fused features into the image decoder for reconstruction representation to generate a generated defective image with defective structure; wherein, the content encoder, style encoder and image decoder are pre-trained based on a joint loss function composed of content loss and style loss; Specifically, the joint loss function is constructed by calculating the following formula:

[0064] in: This is the total loss function; This results in a loss of content consistency. For loss of style consistency; The preset first weighting coefficient; This is the preset second weighting coefficient; The formula for calculating the content consistency loss is as follows:

[0065] in: The generated content features are extracted after the generated defective image is re-inputted into the content encoder; The defect content features are as described above; The formula for calculating the style consistency loss is as follows:

[0066] in: The generated style features are extracted after the generated defective image is re-inputted into the style encoder; This refers to the style characteristics of the device.

[0067] In this embodiment, it should be noted that the decoder typically consists of multiple deconvolutional layers and upsampling modules, used to fuse features. Restore the image to its spatial representation and output the initial generated defect image. :

[0068] in: It generates defect images; It is an image decoder; It is a feature of fusion.

[0069] In order to guide the network to convergence during the training phase, a trade-off must be made. " How much of the original defective content should be retained? ” and " How many new equipment styles to migrate ” This scheme constructs a joint loss function during training. This is achieved by making the generated image... Features are extracted again by the encoder and compared with the true features. A and B Performing Euclidean distance (norm 2) calculations can enforce constraints on network behavior. Adjusting the ratio of the first and second weight coefficients (e.g., dynamically fine-tuning based on the surface complexity of petrochemical equipment) can achieve synergistic optimization of defect structure preservation and equipment style transfer.

[0070] Furthermore, combined Figure 3 It can be seen that the process of inputting the fused features into the image decoder for reconstructing the representation specifically includes: First, the process Transformer The fusion features of the output one-dimensional sequence are reshaped into a two-dimensional spatial feature map, and the dimensionality transformation process is represented as follows: .in, The number of image patches, For feature dimension, and These represent the height and width of the original image, respectively, and 512 represents the number of feature channels after reshaping.

[0071] Subsequently, the reshaped spatial feature map is input into an upsampling module consisting of four stages, with each stage executing... The upsampling operation. Specifically, in each upsampling stage, the feature map undergoes the following processing in sequence: (1) First, it undergoes bilinear upsampling ( Bilinear Upsampling , To increase spatial resolution; (2) Then, it is filled with reflective material. Reflection Padding ), 3 3 convolutional blocks and ReLU Activation layer; (3) After being filled with reflection again, 3 3 convolutional blocks and ReLU Activation layer; (4) Finally, perform instance normalization ( Instance Normalization ).

[0072] After completing four stages of upsampling in sequence, the feature map is finally filled by one reflection and output with 3 channels. 3 convolutional blocks, output dimension is The final generated defect image.

[0073] In the image decoding process of this embodiment, a special technique is employed. " Reflective fill ( Reflection Padding ) ” Instead of regular zero-padding ( Zero Padding Because defect edges often have complex transitions with the background of petrochemical equipment, zero-fill can lead to black borders or artifacts in the generated image, while reflection filling utilizes the image's own edge pixels for mirror extension, resulting in a very natural transition of defect image boundaries; simultaneously, using " Instance normalization ( Instance Normalization ) ” Instead of traditional batch normalization ( BN The reason is that instance normalization only calculates the mean and variance for the channels of a single image, which can preserve the unique texture and contrast style of each petrochemical equipment image to the greatest extent, thereby greatly improving the visual realism of the generated image.

[0074] Step S105: Using the petrochemical equipment image as the background image, construct a fusion energy function based on the gradient relationship between the generated defect image and the background image, and embed the defect region in the generated defect image into the background image by minimizing the fusion energy function to obtain a fused defect image that maintains boundary continuity; Specifically, the defect region indicated by the defect mask is extracted, and the fusion energy function is constructed using the gradient of the generated defect image within the defect region and the gradient of the fused defect image within the defect region.

[0075] in: For the fusion energy function; The set of pixels in the defective region; For pixel position; The gradient of the fusion defect image to be optimized; The gradient for generating the defect image; Using the boundary pixel values ​​of the corresponding defect region in the background image as boundary conditions, the fusion energy function is minimized iteratively to obtain the fused defect image that maintains boundary continuity.

[0076] In this embodiment, it should be noted that after obtaining the image generated by the network, if a simple image stitching method is directly adopted (such as...) CutMix (Or simply mask replacement) to attach the defect into the background of the new device, and obvious edge abruptness will appear at the seam (i.e. " artifacts ” This will severely damage the quality of training data.

[0077] Therefore, this embodiment introduces a method based on the Poisson equation ( Poisson Equation The gradient domain fusion technique is used. The working principle of this step is that the human eye and the detection model perceive the image... " Gradient (i.e., high-frequency edge changes) ” Comparison " Absolute pixel intensity ” More sensitive. Therefore, while keeping the gradient inside the defect constant, we solve for a completely new fused image.

[0078] In the specific implementation of this step, the background image is set as follows: The defective area is fused images .

[0079] The energy function is defined as:

[0080] in, It is the fusion energy function; It is the pixel position; It is to fuse image gradients; It generates image gradients.

[0081] The discrete update formula is:

[0082] in, It is a pixel p The fusion value; It is a pixel p The neighborhood set; It is the value of the neighboring pixels; It is the gradient guiding term. The neighborhood set is defined as:

[0083] in, It is a pixel p The neighborhood set.

[0084] The gradient-guided term is:

[0085] in, It generates the gradient difference of the image.

[0086] The boundary conditions are:

[0087] in, It is the boundary of the defect area; It's the background image.

[0088] Final generated image:

[0089] in, It generates images; It is a fusion operation.

[0090] Step S106: Input the fused defect image into the pre-trained detection model, output the predicted bounding box and its corresponding detection confidence; calculate the intersection-union ratio (IU) between the predicted bounding box and the ground truth bounding box generated based on the defect mask, and calculate the comprehensive score of the fused defect image by combining the detection confidence and the IU; When the overall score is greater than a preset screening threshold, the corresponding fused defect image is retained as a target generated sample; specifically, the maximum detection confidence is extracted as the overall confidence from the predicted bounding box output by the pre-trained detection model; a ground truth bounding box for the defect is generated based on the defect mask, and the intersection-union ratio (IUU) between the predicted bounding box and the ground truth bounding box is calculated; using the formula... Calculate the comprehensive score; where: For comprehensive scoring; For overall confidence level; For intersection, union, and comparison; This is the preset third weighting coefficient; This is the preset fourth weighting coefficient.

[0091] In this embodiment, it should be noted that data augmentation should not blindly pursue quantity. Generating low-quality samples (such as those with completely lost defect features or those with pseudo-defects that resemble the background) and mixing them into the training set can actually cause problems. " Catastrophic forgetting ” Or misleading models.

[0092] Therefore, this step innovatively introduces... " Detection model backdrive ” The sample selection mechanism utilizes a pre-trained basic defect detection network (such as...). YOLOv 8 or Faster R-CNN Inference is performed on the newly generated fusion defect image: 1 . Detection confidence level ( ,in, It is the maximum detection confidence; It is the first k The confidence score of each prediction box reflects whether the generated defect matches the data distribution of the real defect (if it does not, the network confidence score will be extremely low). 2 . Intersection and Union (IoU) ,in, The intersection-over-union (IoU) ratio of the predicted bounding box to the ground truth bounding box reflects whether the defect generation location has shifted. Ground truth bounding box From the aforementioned mask The circumscribed rectangle is calculated to obtain ( ,in, It is a defective true bounding box; (It is a defect mask), prediction box Obtained from the generated image input detection network ( The detection network is represented as: ,in, It is a detection network; It is the input image; These are model parameters.

[0093] Assign appropriate weights to confidence level and cross-union ratio. , Receive a comprehensive score Only when Only when the (screening threshold) is the enhanced sample considered to be... " High quality and consistent distribution ” Those that are valid should be kept; otherwise, they should be discarded. discard ), which is represented as follows:

[0094] in, These are the filtering results; It is a rating value; It is the filtering threshold.

[0095] Step S107: Merge all the retained target generated samples with the real defect dataset to construct the final training dataset, and train the defect detection model for petrochemical equipment based on the final training dataset.

[0096] Specifically, the optimization objective satisfies the following formula:

[0097] in: These are the optimal parameters for the final trained defect detection model; These are the parameters to be optimized in the model; For parameters Defect detection model; For detection loss function; For the final training dataset Sample images in the database.

[0098] In this embodiment, specifically, let all the generated images that pass the screening form a dataset. (in, It is the generated sample set after filtering; It is the first k One filtered image is generated; The number of generated samples is determined by filtering, and this is ultimately mixed with the initial set of real defects. (in, This is the final training dataset; It is a real defect dataset; It is the generated sample set after filtering; (This indicates a dataset merging operation).

[0099] By introducing filtered generated samples, the scale of training data can be effectively expanded while maintaining the true characteristics of defect distribution, thereby improving the adaptability of the detection model to different equipment backgrounds, defect morphologies, and complex scenarios. Based on the final training set... The detection model is retrained, and its optimization objective can be expressed as:

[0100] in: These are the optimal parameters for the final trained defect detection model; These are the parameters to be optimized in the model; For parameters Defect detection model; For detection loss function; For the final training dataset Sample images in the database.

[0101] On this enhanced massive dataset, the network parameters are updated using the detection loss function (which typically includes classification loss and bounding box regression loss) to finally obtain the optimal model parameters.

[0102] This embodiment incorporates a large number of high-quality defect samples generated under various complex lighting conditions and different equipment textures. The resulting defect detection model not only overcomes the problem of overfitting when training with small sample data from petrochemical equipment sites, but also significantly improves the system's robustness, generalization ability, and detection recall under complex backgrounds such as corrosion, cracks, and structural damage.

[0103] Based on the same technical concept, embodiments of the present invention also provide an electronic device that can implement the generative data augmentation method flow for defect detection in petrochemical equipment provided in the above embodiments of the present invention. In one embodiment, the electronic device can be a server, a terminal device, or other electronic equipment. Figure 4 As shown, the electronic device may include: At least one processor and a memory connected to the at least one processor. In this embodiment of the invention, the specific connection medium between the processor and the memory is not limited. Figure 4 The example used is the connection between the processor and memory via a bus. The bus... Figure 4 The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. Buses can be divided into address buses, data buses, control buses, etc., but for ease of representation, [the specific bus type is not shown here]. Figure 4The processor is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, a processor can also be called a controller; there are no restrictions on the name.

[0104] In this embodiment of the invention, the memory stores instructions executable by at least one processor. By executing the instructions stored in the memory, the at least one processor can perform the generative data augmentation method for defect detection in petrochemical equipment discussed above. The processor can implement... Figure 4 The functions of each module in the device shown.

[0105] The processor is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory and calling data stored in memory, it can monitor the device's various functions and process data, thereby enabling overall monitoring of the device.

[0106] In an alternative design, the processor may include one or more processing units. The processor may integrate an application processor and a modem processor, wherein the application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may also not be integrated into the processor. In some embodiments, the processor and memory may be implemented on the same chip; in some embodiments, they may also be implemented separately on separate chips.

[0107] The processor can be a general-purpose processor, such as CPU Digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the generative data augmentation method for defect detection in petrochemical equipment disclosed in the embodiments of this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0108] Memory, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, and random access memory. (Random Access Memory , RAM) Static Random Access Memory (Static Random Access Memory , SRAM Programmable Read-Only Memory (Programmable Read Only Memory , PROM) Read-only memory (Read Only Memory , ROM) Electrically erasable programmable read-only memory (Electrically Erasable Programmable Read Only Memory , EEPROM) Magnetic storage devices, disks, optical discs, etc. A memory is any medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited thereto. The memory in this embodiment can also be a circuit or any other device capable of implementing storage functions, used to store program instructions and... / Or data.

[0109] By designing and programming the processor, the code corresponding to the generative data augmentation method for defect detection in petrochemical equipment described in the foregoing embodiments can be embedded into the chip, enabling the chip to execute the steps of the method described in the foregoing embodiments during runtime. How to design and program the processor is a technique well-known to those skilled in the art and will not be elaborated upon here.

[0110] Based on the same inventive concept, embodiments of the present invention also provide a storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the generative data augmentation method for defect detection in petrochemical equipment described above.

[0111] In some alternative embodiments, the present invention also provides that various aspects of the generative data augmentation method for defect detection in petrochemical equipment can also be implemented as a program product comprising program code that, when the program product is run on a device, causes the control device to perform the steps in the generative data augmentation method for defect detection in petrochemical equipment described above according to various exemplary embodiments of the present invention.

[0112] It should be noted that although several units or sub-units of the apparatus have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the invention, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units. Furthermore, although the operation of the method of the invention is described in a specific order in the drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and... / Alternatively, a single step can be broken down into multiple steps for execution.

[0113] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of one or more computer-usable storage media containing computer-usable program code. ( Including but not limited to disk storage, CD ROM Optical memory, etc. ) The form of computer program products implemented on [the platform].

[0114] This invention relates to methods and apparatus according to embodiments of the invention. ( system ) Flowcharts and computer program products / Or a block diagram. It should be understood that flowcharts and... can be implemented by computer program instructions. / Or each process in the block diagram and / Or boxes, and flowcharts and / Or the process in the block diagram and / Or a combination of boxes. These computer program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a server, such that the instructions, which execute via the processor of the computer or other programmable data processing device, produce instructions for implementing the process. Figure 1 One or more processes and / or box Figure 1 A device that provides the functions specified in one or more boxes.

[0115] Program code for performing the operations of this invention can be written using any combination of one or more programming languages, including object-oriented programming languages ​​such as... Java , C++ And so on, including conventional procedural programming languages, such as "C” A programming language or similar. The program code can be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0116] In cases involving remote computing devices, the remote computing device can be connected to any type of network, including a local area network. (LAN) or wide area network (WAN) It can be connected to a user computing device, or it can be connected to an external computing device. ( For example, using an Internet service provider to connect to the Internet.) .

[0117] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or box Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or box Figure 1 The steps of the function specified in one or more boxes.

[0119] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

Claims

1. A generative data augmentation method for defect detection in petrochemical equipment, characterized in that, include: Acquire real defect images and their corresponding defect masks, as well as images of petrochemical equipment; The real defect image and the defect mask are separated at the pixel level to extract the defect content image. The defective content image is input into the content encoder for feature extraction to obtain the defective content features; The image of the petrochemical equipment is then input into a style encoder for feature extraction to obtain the equipment style features. By using a cross-attention mechanism, the defective content features are used as query vectors, and the device style features are used as key vectors and value vectors for multi-level feature fusion to obtain fused features; The fused features are input into the image decoder for reconstruction representation to generate a generated defective image with defective structure; wherein, the content encoder, style encoder and image decoder are pre-trained based on a joint loss function composed of content loss and style loss; Using the petrochemical equipment image as a background image, a fusion energy function is constructed based on the gradient relationship between the generated defect image and the background image. By minimizing the fusion energy function, the defect region in the generated defect image is embedded into the background image to obtain a fused defect image that maintains boundary continuity. The fused defect image is input into a pre-trained detection model, which outputs a predicted bounding box and its corresponding detection confidence. The cross-union ratio (CUP) between the predicted bounding box and the ground truth bounding box generated based on the defect mask is calculated, and the comprehensive score of the fused defect image is calculated by combining the detection confidence and the CUP. When the overall score is greater than the preset screening threshold, the corresponding fusion defect image will be retained as the target generated sample. All the retained target generated samples are merged with the real defect dataset to construct the final training dataset, and the defect detection model for petrochemical equipment is trained based on the final training dataset. The joint loss function is constructed by calculating the following formula: in: This is the total loss function; This results in a loss of content consistency. For loss of style consistency; The preset first weighting coefficient; This is the preset second weighting coefficient; The formula for calculating the content consistency loss is as follows: in: The generated content features are extracted after the generated defective image is re-inputted into the content encoder; The defect content features are as described above; The formula for calculating the style consistency loss is as follows: in: The generated style features are extracted after the generated defective image is re-inputted into the style encoder; The device's style characteristics; The step of using the petrochemical equipment image as a background image and constructing a fusion energy function based on the gradient relationship between the generated defect image and the background image includes: Extract the defect region indicated by the defect mask, and construct the fusion energy function using the gradient of the generated defect image within the defect region and the gradient of the fused defect image within the defect region: in: For the fusion energy function; The set of pixels in the defective region; For pixel position; The gradient of the fusion defect image to be optimized; The gradient for generating the defect image; Using the boundary pixel values ​​of the corresponding defect region in the background image as boundary conditions, the fusion energy function is minimized iteratively to obtain the fused defect image that maintains boundary continuity.

2. The generative data augmentation method for defect detection in petrochemical equipment according to claim 1, characterized in that, The defective content image is input into a content encoder for feature extraction to obtain defective content features; The image of the petrochemical equipment is then input into a style encoder for feature extraction to obtain equipment style features, including: The defect content image is divided into several image blocks and mapped to a feature space to obtain a content image block sequence. This content image block sequence is then input into a multi-layer... Transformer The defective content features are output from the content encoder, which is composed of structural components. The petrochemical equipment image is divided into several image blocks and mapped to a feature space to obtain an equipment image block sequence. This equipment image block sequence is then input into a multi-layer... Transformer The style encoder, composed of structural components, outputs the style characteristics of the device.

3. The generative data augmentation method for defect detection in petrochemical equipment according to claim 2, characterized in that, The method utilizes a cross-attention mechanism, using the defective content features as a query vector and the device style features as key and value vectors for multi-layer feature fusion to obtain fused features, including: The defect content features are mapped and calculated to obtain a query matrix; and the device style features are mapped and calculated respectively to obtain the corresponding key matrix and value matrix. The cross-attention weight matrix is ​​calculated by combining the query matrix, the key matrix, and the preset feature dimensions; The product of the cross-attention weight matrix and the value matrix is ​​calculated, and the above fusion process is iterated in the multi-layer cross-attention module to obtain the final fusion feature.

4. The generative data augmentation method for defect detection in petrochemical equipment according to claim 1, characterized in that, The calculation of the comprehensive score for the fused defect image by combining the detection confidence score and the cross-union ratio includes: The maximum detection confidence is extracted from the prediction bounding box output by the pre-trained detection model as the overall confidence. Based on the defect mask, a true defect bounding box is generated, and the intersection-union ratio (IUU) between the predicted bounding box and the true defect bounding box is calculated. The overall score is calculated using the following formula: in: For comprehensive scoring; For overall confidence level; For intersection, union, and comparison; This is the preset third weighting coefficient; This is the preset fourth weighting coefficient.

5. The generative data augmentation method for defect detection in petrochemical equipment according to claim 1, characterized in that, The defect detection model for petrochemical equipment is trained based on the final training dataset, and its optimization objective satisfies the following formula: in: These are the optimal parameters for the final trained defect detection model; These are the parameters to be optimized in the model; For parameters Defect detection model; For detection loss function; For the final training dataset Sample images in the database.

6. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, and the at least one processor executes the instructions stored in the memory to perform the operation as described in claim 1. - The method as described in any one of the 5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store instructions that, when executed, cause as claimed in claim 1. - The method described in any one of the 5 is implemented.

Images