Positive sample filtering method and device combining image comparison and sensitive defect detection

Abstract

The invention relates to the technical field of image processing and defect detection, in particular to a positive sample filtering method and device combining image comparison and sensitive defect detection. The method comprises the following steps: setting a reference image, and obtaining a current inspection image; constructing a twin network change detection algorithm for detecting whether the current inspection image and the reference image have pixel-level changes or not, and marking a change region as a potential defect region; constructing a defect detection model for performing low-confidence slice reasoning on the current inspection image and the reference image, retaining defects only detected on the current inspection image, judging whether the defects are overlapped with a potential defect area or not, if yes, judging that the current inspection image is abnormal, and if not, judging that the current inspection image is abnormal. And otherwise, determining the sample as a normal sample, storing and updating the reference image, and pushing an abnormal image for subsequent analysis. No-defect samples are screened in advance through a dual verification mechanism, and only suspected defect samples are pushed for subsequent analysis, so that the defect false alarm rate is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and defect detection, and in particular to a positive sample filtering method and device combining image comparison and sensitive defect detection. Background Art

[0002] In the existing defect detection technologies, usually a single image comparison or defect detection method is adopted for anomaly recognition.

[0003] For example, Chinese Patent Document CN119068352A discloses a remote sensing image change detection method based on a PAP-UNet++ network. First, the original data set is preprocessed, and the training set images are uniformly scaled into image blocks of 512×512 size as inputs. In the encoder stage, a hybrid dilated convolution module is used to gradually extract multi-scale features and perform downsampling operations. In the decoder stage, a parallel dilated pyramid module is inserted to enhance the model's ability to capture global context features. In addition, a CBAM attention mechanism is introduced in the skip connection process at the same level to improve the recognition and discrimination ability of the changed regions. Finally, a pixel-level classification prediction result consistent with the input size is generated. Through the above method, the problem of insufficient utilization of multi-scale feature information by the network can be effectively solved, and the changed detail information can be accurately detected.

[0004] Chinese Patent Document CN119152194A discloses a defect small target detection method and system for high-resolution images in a power scenario. When training a small defect detection model, a progressive training method is adopted, which has a better detection effect on small targets in common power scenarios. Sampling is performed through a scoring network, which avoids complicated and redundant calculations while ensuring the model accuracy, thus ensuring the timeliness and accuracy of the entire model, and enabling end-to-end detection of power scenario targets. In the solution of the present invention, the current power device image is cropped using the target resolution to obtain multiple frames of current power image slices at the target resolution. The current power image slices are detected based on the trained small defect detection model, and after obtaining the defect inspection results, the box coordinates of the defect detection results are mapped to the current power device image, which has a high detection accuracy for small targets in power images and can meet the requirements of the power grid intelligent inspection task.

[0005] However, the existing methods have the following problems:

[0006] 1. Image comparison method: Only relying on pixel-level change detection is easily affected by environmental factors such as illumination and noise, resulting in a high false alarm rate.

[0007] 2. Defect detection method: Only relying on the defect detection model may not be able to effectively distinguish normal samples from defect samples, especially when the defect features are not obvious.

[0008] 3. Lack of dual verification mechanism in the prior art: Existing methods usually only use a single technology for defect identification and lack multiple verifications of the detection results, resulting in a relatively high false alarm rate.

[0009] Therefore, there is an urgent need for a dual verification method that can combine image comparison and defect detection to improve the accuracy and robustness of defect detection. Summary of the Invention

[0010] The present invention aims to overcome at least one defect of the above prior art and provides a positive sample filtering method that combines image comparison and sensitive defect detection. By using a dual verification mechanism, defect-free samples are screened in advance, and only suspected defect samples are pushed for subsequent analysis, thereby reducing the defect false alarm rate.

[0011] The present invention also discloses a device loaded with a positive sample filtering method that combines image comparison and sensitive defect detection.

[0012] The detailed technical solution of the present invention is as follows:

[0013] A positive sample filtering method that combines image comparison and sensitive defect detection, the method comprising:

[0014] S1. Set a reference image, and the reference image is a pure defect-free image;

[0015] S2. Obtain the current inspection image;

[0016] S3. Construct a siamese network change detection algorithm, input the current inspection image and the reference image into the siamese network change detection algorithm for comparison to determine whether there is a pixel-level change in the current inspection image compared with the reference image. If a changed area is detected, mark it as a potential defect area;

[0017] S4. Construct a defect detection model, and use the defect detection model to perform low-confidence slice inference on the current inspection image and the reference image respectively. If a defect area is detected in the current inspection image, retain the defect area; if defect areas are detected in both the current inspection image and the reference image and the positions overlap, discard the defect area;

[0018] S5. Merge the potential defect areas obtained by the siamese network change detection algorithm and the defect areas retained by the defect detection model. If the potential defect area overlaps with the defect area, it is determined that the current inspection image is abnormal; if the potential defect area does not overlap with the defect area, it is determined that the current inspection image is a normal sample, and the reference image is updated and stored in the database;

[0019] S6. Push all inspection images with abnormalities for subsequent analysis.

[0020] Preferably according to the present invention, in the step S3, a twin network change detection algorithm is constructed, specifically including: based on the UNet network, dilated convolution is used to increase the receptive field, and the CBAM attention mechanism is added to enhance the feature extraction ability.

[0021] Preferably according to the present invention, the CBAM attention mechanism includes a channel attention map M c (F), and:

[0022] M c (F) = σ(MLP(AvgPool(F)) + MLP(MaxPool(F))) (1);

[0023] In formula (1): σ(·) represents the sigmoid function, MLP represents a multi-layer perceptron, AvgPool(F) and MaxPool(F) respectively represent the average pooling feature and the maximum pooling feature; AvgPool(F) and MaxPool(F) are forwarded to a shared network composed of a multi-layer perceptron MLP, and the channel attention map M c (F) is generated through the sigmoid function σ.

[0024] Preferably according to the present invention, the CBAM attention mechanism further includes a spatial attention map M s (F), and:

[0025] M s (F) = σ(f 7×7 ([AvgPool(F)); MaxPool(F)])) (2);

[0026] In formula (2): f 7×7 represents a convolution operation with a filter size of 7×7; AvgPool(F) and MaxPool(F) are aggregated to generate two 2D maps, and the spatial attention map M s (F) is generated through the sigmoid function σ.

[0027] Preferably according to the present invention, in the step S3, constructing the twin network change detection algorithm specifically further includes: adopting a mixed loss method of Dice Loss and Focal Loss, and adjusting the weights of the twin network through error backpropagation; the loss function Loss of the twin network change detection algorithm is:

[0028] Loss = FL + DL (3);

[0029] In formula (3): FL is the Focal Loss function, and DL is the Dice Loss function;

[0030] Among them, the Focal Loss function is:

[0031] FL(p t ) = -α t (1 - p t ) γ log(p t ) (4);

[0032]

[0033] In equations (4)-(5): p is the probability of judging the positive sample y = 1, p t The function unifies p and 1 - p, and the parameter α t is used to suppress the imbalance in the number of positive and negative samples, and the parameter γ is used to control the imbalance in the number of easy / hard-to-distinguish samples;

[0034] The Dice Loss function is as follows:

[0035]

[0036] In equation (6): X is the prediction map, Y is the label map, where |X ∩ Y| is the intersection between X and Y, and |X| and |Y| respectively represent the number of elements in X and Y. Among them, the coefficient of the numerator is 2, indicating that there is duplicate calculation of the common elements between X and Y in the denominator.

[0037] Preferably according to the present invention, S4 specifically includes:

[0038] S41: Both the current inspection image and the reference image are cut into multiple slices, each slice having a partially overlapping area, and the size, position, and relative coordinate range of each slice in the image before slicing are recorded;

[0039] S42: The image training samples obtained by manual annotation based on historical inspection images are pre - cropped according to the size of the slices, and the defect detection model is obtained by training with the cropped image training samples to ensure that any slice of the current inspection image and the reference image has the same data scale as the image training samples;

[0040] S43: Target detection inference is independently performed on each slice of the current inspection image and the reference image. The defect detection model trained in S42 is used to detect each slice, and a low - confidence inference method is adopted to generate the detection results of each slice, including the bounding box coordinates, class labels, and confidence scores of each slice;

[0041] S44: The bounding box coordinates of each slice of the current inspection image and the reference image are converted from their local coordinate systems to the global coordinate system of the corresponding image before slicing to obtain the converted bounding box coordinates of each slice;

[0042] S45: Calculate the intersection over union (IoU) between the transformed bounding box coordinates corresponding to every two slices of the current inspection image and every two slices of the reference image, compare the IoU result with a preset IoU threshold. If the IoU result exceeds the preset IoU threshold, it is determined that the transformed bounding box coordinates of these two slices are repeated. According to the confidence scores of the slices, retain the bounding box coordinates corresponding to the slice with the higher confidence score, and the region formed by these bounding box coordinates is the defect region.

[0043] Preferably according to the present invention, in S5, the reference image storage update strategy is as follows:

[0044] If the current inspection image is a normal sample and the total number of reference image samples in the reference image library is less than n (where n is the capacity of the reference image library), directly store the current inspection image in the library;

[0045] If the current inspection image is a normal sample and the total number of reference image samples in the reference image library is greater than or equal to n, calculate the SSIM similarity between the current inspection image and each reference image sample in the reference image library one by one, and replace the reference image sample with the highest similarity with the current inspection image to maintain the diversity of the reference image library.

[0046] In another aspect of the present invention, there is provided an apparatus for implementing a positive sample filtering method that combines image comparison and sensitive defect detection. The apparatus includes:

[0047] An input module for setting a reference image, where the reference image is a pure defect-free image;

[0048] An acquisition module for acquiring the current inspection image;

[0049] A change detection module for constructing a twin network change detection algorithm, inputting the current inspection image and the reference image into the twin network change detection algorithm for comparison to determine whether there are pixel-level changes in the current inspection image compared with the reference image. If a changed area is detected, mark it as a potential defect area;

[0050] A defect detection module for constructing a defect detection model, using the defect detection model to perform low-confidence slice inference on the current inspection image and the reference image respectively. If a defect area is detected in the current inspection image, retain the defect area; if defect areas are detected in both the current inspection image and the reference image and the positions overlap, discard the defect area;

[0051] A result judgment module is used to merge the potential defect areas obtained by the twin network change detection algorithm with the remaining defect areas obtained by the defect detection model. If the potential defect areas overlap with the defect areas, it is determined that the current inspection image is abnormal; if the potential defect areas do not overlap with the defect areas, it is determined that the current inspection image is a normal sample, and the reference image is updated and stored in the database.

[0052] An output module is used to push all inspection images with abnormalities for subsequent analysis.

[0053] In another aspect of the present invention, an electronic device is further provided, including:

[0054] At least one processor; and

[0055] A memory, where the memory stores instructions. When the instructions are executed by the at least one processor, the at least one processor executes the positive sample filtering method combining image comparison and sensitive defect detection as described above.

[0056] In another aspect of the present invention, a machine-readable storage medium is further provided, which stores executable instructions. When the instructions are executed, the machine executes the positive sample filtering method combining image comparison and sensitive defect detection as described above.

[0057] Compared with the prior art, the beneficial effects of the present invention are:

[0058] (1) Dual verification mechanism: Combining image change detection and sensitive defect detection, the accuracy of defect detection is improved through the dual verification mechanism; only when the changed area overlaps with the defect detection area, it is determined as abnormal, effectively reducing false alarms.

[0059] (2) Pixel-level change detection algorithm: Design a twin network structure, use dilated convolution to increase the receptive field, combine the CBAM attention mechanism to improve the feature extraction ability, learn from the data of the front and back changes, realize pixel-level recognition of several changes such as the disappearance, appearance, and movement of the target, and at the same time, combining large-scale data training can effectively avoid the influence of environmental factors such as light and noise, and has higher robustness than the traditional image subtraction method.

[0060] (3) Low-confidence slice inference: Improve the training process of the Yolo series object detection algorithm, perform data augmentation in the form of slices in advance at the data input stage, and the trained model performs low-confidence slice inference on both the reference image and the inspection image at the same time, improving the defect detection rate and avoiding misjudging normal features as defects; only the defects detected on the inspection image are retained, further reducing the false alarm rate.

[0061] (4) Dynamic reference image update: The reference image can be dynamically updated according to the actual scenario, and a reference image update strategy is designed to ensure that the reference image library covers diverse scenarios to adapt to environmental changes such as lighting and equipment aging.

[0062] (5) Efficient positive sample filtering: By jointly applying change detection and defect detection, defect-free samples are screened in advance, reducing the computational burden of subsequent analysis and the false alarm rate of defects, thereby reducing the workload of manual confirmation. Description of the Drawings

[0063] Figure 1 is a flowchart of the positive sample filtering method combining image comparison and sensitive defect detection according to the present invention.

[0064] Figure 2 is a schematic diagram of change detection in Embodiment 1 of the present invention.

[0065] Figure 3 is a network structure diagram of the CBAM attention mechanism in Embodiment 1 of the present invention.

[0066] Figure 4 is a schematic diagram of sensitive defect detection in Embodiment 1 of the present invention. Detailed Embodiments

[0067] The present disclosure will be further described below in conjunction with the drawings and embodiments.

[0068] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present disclosure belongs.

[0069] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0070] In the case of no conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

[0071] Embodiment 1

[0072] Refer Figure 1 , this embodiment provides a positive sample filtering method combining image comparison and sensitive defect detection, and the method includes:

[0073] S1. Set a reference image, and the reference image is a pure defect-free image.

[0074] In this embodiment, a point position reference image is preset, and it is required that the reference image is a pure and defect-free image sample. The reference image serves as a reference standard for subsequent image comparison.

[0075] The described reference image can be dynamically updated according to the actual scenario.

[0076] S2. Obtain the current inspection image.

[0077] Specifically, during each inspection process, the current image is captured, that is, the current inspection image is obtained.

[0078] S3. Construct a Siamese network change detection algorithm, input the current inspection image and the reference image into the Siamese network change detection algorithm for comparison to determine whether there are pixel-level changes in the current inspection image compared with the reference image. If a changed area is detected, it is marked as a potential defect area.

[0079] In this embodiment, a change detection algorithm is designed in the form of a Siamese network, and the inference result is pixel-level change; at the same time, dilated convolution is used to increase the receptive field, combined with the CBAM attention mechanism to improve the feature extraction ability. By learning the data with changes before and after, pixel-level recognition of several changes such as the disappearance, appearance, and movement of the target is realized. At the same time, combined with large-scale data training, the influence of environmental factors such as illumination and noise can be effectively avoided.

[0080] Specifically, design a Siamese network structure, as Figure 2 shown, based on the UNet network, use dilated convolution to increase the receptive field to solve the problem that downsampling in semantic segmentation will reduce the image resolution and lose information; use CBAM to add attention mechanisms in the channel and spatial dimensions to enhance the feature extraction ability. Adopt a mixed loss method of Dice Loss and Focal Loss, and adjust the weights of the Siamese network through error backpropagation.

[0081] Furthermore, as Figure 3 shown, the CBAM attention mechanism includes a channel attention map M c (F), and:

[0082] M c (F) = σ(MLP(AvgPool(F)) + MLP(MaxPool(F))) (1);

[0083] In formula (1): σ(·) represents the sigmoid function, and MLP represents a multi-layer perceptron. AvgPool(F) and MaxPool(F) represent the average pooling feature and the maximum pooling feature respectively; AvgPool(F) and MaxPool(F) are forwarded to a shared network composed of a multi-layer perceptron MLP, and a channel attention map M is generated through the sigmoid function σ c (F).

[0084] The CBAM attention mechanism further includes a spatial attention map M s (F), and:

[0085] M s (F) = σ(f 7×7 ([AvgPool(F)); MaxPool(F)])) (2);

[0086] In formula (2): f 7×7 represents a convolution operation with a filter size of 7×7; AvgPool(F) and MaxPool(F) are aggregated to generate two 2D maps, and a spatial attention map M s (F) is generated through the sigmoid function σ

[0087] The loss function Loss of the twin network change detection algorithm is:

[0088] Loss = FL + DL (3);

[0089] In formula (3): FL is the Focal Loss function, and DL is the Dice Loss function;

[0090] Among them, the Focal Loss function is:

[0091] FL(p t ) = -α t (1 - p t ) γ log(p t ) (4);

[0092]

[0093] In formulas (4)-(5): p is the probability of judging the positive sample y = 1, p t The function unifies p and 1 - p, and the parameter α t is used to suppress the imbalance in the number of positive and negative samples, and the parameter γ is used to control the imbalance in the number of easy / difficult-to-distinguish samples;

[0094] The Dice Loss function is:

[0095]

[0096] In Equation (6): X is the prediction map, and Y is the label map. Here, |X∩Y| is the intersection between X and Y, and |X| and |Y| represent the number of elements in X and Y respectively. The coefficient of the numerator is 2 because the denominator double-counts the common elements between X and Y.

[0097] Based on the above, by comparing the current inspection image with the reference image, pixel-level recognition of the changed object is achieved. If a changed area is detected, it is marked as a potential defect area.

[0098] S4. Construct a defect detection model, and use the defect detection model to perform low-confidence slice inference on the current inspection image and the reference image respectively. If a defect area is detected in the current inspection image, retain the defect area; if defect areas are detected in both the current inspection image and the reference image and their positions overlap, discard the defect area.

[0099] In this embodiment, a defect detection model is designed based on the improvement of the object detection model (such as the yolo series). The trained defect detection model is used to perform low-confidence slice inference on the reference image and the inspection image respectively to improve the detection rate. If a defect is only detected in the inspection image, retain the defect; if a defect is detected in both the reference image and the inspection image and their positions overlap, discard the defect, that is, consider it as a normal feature.

[0100] Specifically, as Figure 4 shown, the large image is segmented into multiple small blocks (slices) for object detection to improve the detection effect on small objects and high-resolution images and increase the detection rate.

[0101] In this embodiment, this step is implemented as follows:

[0102] S41: Cut both the current inspection image and the reference image into multiple slices. Each slice has a partially overlapping area, and record the size, position, and relative coordinate range in the image before slicing for each slice.

[0103] Specifically, the original large images of the current inspection image and the reference image are cut into multiple small blocks, usually rectangular image blocks, such as 512x512 in size. Each slice has a 20% overlapping area to ensure that the object will not be split or lost due to the slice boundary. For areas smaller than 512x512 in size, gray filling is used.

[0104] Moreover, each slice has its starting position (such as the upper left corner coordinates) relative to the original image before slicing. When dividing the slices, it is necessary to record the size, position (offset), and relative coordinate range in the original image before slicing for each slice.

[0105] S42: Pre - crop the image training samples obtained by manual annotation based on historical inspection images according to the size of the slices, and use the cropped image training samples for model training to obtain a defect detection model, ensuring that any slice of the current inspection image and the reference image has the same data scale as the image training samples.

[0106] To ensure the effect during inference, during model training, improve the training process of the original object detection. Pre - crop the image training samples in advance according to the size of the slices, and use the cropped training samples for model training to obtain a defect detection model, ensuring that the inference data (i.e., the slice data of the current inspection image and the reference image) has the same scale as the training data.

[0107] S43: Independently perform object detection inference on each slice of the current inspection image and the reference image. Use the defect detection model trained in S42 to detect each slice, and adopt a low - confidence inference method to generate the detection results of each slice, including the bounding box coordinates, class labels, and confidence scores of each slice.

[0108] In this embodiment, to ensure that no defects are missed, a low - confidence inference method is adopted here to generate the detection results of each slice. The detection results include information such as the bounding box coordinates, class labels, and confidence scores of each slice.

[0109] S44: Convert the bounding box coordinates of each slice of the current inspection image and the reference image from their local coordinate systems to the global coordinate system of the corresponding un - sliced image, obtaining the converted bounding box coordinates of each slice.

[0110] This step is to map the detection results back to the coordinate system of the original un - sliced image, that is, to convert the coordinates of each bounding box from the local coordinate system of the slice to the global coordinate system of the original un - sliced image. Assume that the coordinates of the upper - left corner of the slice in the original image are (x_offset, y_offset), then the coordinates (x_min, y_min, x_max, y_max) of the bounding box in the slice are converted to:

[0111] x_min_global = x_min + x_offset;

[0112] y_min_global = y_min + y_offset;

[0113] x_max_global = x_max + x_offset;

[0114] y_max_global = y_max + y_offset.

[0115] Store the converted bounding box coordinates along with their corresponding categories and confidence scores in a global list.

[0116] S45: Calculate the intersection over union (IoU) between the converted bounding box coordinates of each pair of slices of the current inspection image and each pair of slices of the reference image. Compare the IoU result with a preset IoU threshold. If the IoU result exceeds the preset IoU threshold, it is determined that the converted bounding box coordinates of these two slices are duplicates. Based on the confidence scores of the slices, retain the bounding box coordinates corresponding to the slice with the higher confidence score, and the region formed by these bounding box coordinates is the defect region.

[0117] Since there may be overlapping regions between slices and the same target may be detected by multiple slices, it is necessary to merge the duplicate detection results.

[0118] For the detection results of all slices, calculate the intersection over union (IoU) between the converted bounding boxes of each pair of slices. Define an IoU threshold, such as 0.5. If the IoU result of the converted bounding boxes of two slices exceeds the set IoU threshold, they are considered duplicate detection results. Among the duplicate bounding boxes, retain the bounding box with the highest confidence score and delete the other bounding boxes. The region formed by the finally retained bounding boxes is the defect region.

[0119] Based on the above, complete the low-confidence slice inference on the current inspection image and the reference image to obtain the defect detection result. If the defect detection result is that a defect region is only detected on the current inspection image, retain the defect region; if the defect detection result is that defect regions are detected on both the current inspection image and the reference image and the positions overlap, discard the defect region and consider it a normal feature.

[0120] S5. Merge the potential defect region obtained by the Siamese network change detection algorithm with the retained defect region obtained by the defect detection model. If the potential defect region overlaps with the defect region, it is determined that the current inspection image is abnormal; if the potential defect region does not overlap with the defect region, it is determined that the current inspection image is a normal sample, and the reference image is updated in the database without pushing.

[0121] In this embodiment, the reference image database update strategy is as follows:

[0122] If the current inspection image is a normal sample and the total number of reference image samples in the reference image database is less than n (n is the capacity of the reference image database, which can be customized), directly store the current inspection image in the database;

[0123] If the current inspection image is a normal sample and the total number of reference image samples in the reference image library is greater than or equal to n, then calculate the SSIM similarity between the current inspection image and each reference image sample in the reference image library one by one, and replace the reference image sample with the highest similarity with the current inspection image to maintain the diversity of the reference image library, such as covering different lighting conditions, seasonal changes, etc.

[0124] S6. Push all inspection images with abnormalities for subsequent analysis.

[0125] That is, output the result, only push the images with abnormalities for subsequent detailed analysis, and reduce the false alarm rate.

[0126] In summary, the positive sample filtering method combining image comparison and sensitive defect detection of the present invention is applicable to pre-screening defect-free samples in scenarios such as industrial inspection and security monitoring, reducing the burden of subsequent analysis and reducing the false alarm rate.

[0127] Embodiment 2

[0128] This embodiment provides a device for implementing the positive sample filtering method combining image comparison and sensitive defect detection. The device includes:

[0129] An input module for setting a reference image, where the reference image is a pure defect-free image;

[0130] An acquisition module for acquiring the current inspection image;

[0131] A change detection module for constructing a twin network change detection algorithm, inputting the current inspection image and the reference image into the twin network change detection algorithm for comparison to determine whether there is a pixel-level change in the current inspection image compared with the reference image. If a changed area is detected, mark it as a potential defect area;

[0132] A defect detection module for constructing a defect detection model, and performing low-confidence slice inference on the current inspection image and the reference image respectively by using the defect detection model. If a defect area is detected on the current inspection image, retain the defect area; if defect areas are detected on both the current inspection image and the reference image and the positions overlap, discard the defect area;

[0133] A result judgment module for merging the potential defect area obtained by the twin network change detection algorithm and the defect area retained by the defect detection model. If the potential defect area overlaps with the defect area, it is determined that the current inspection image has an abnormality; if the potential defect area does not overlap with the defect area, it is determined that the current inspection image is a normal sample, and the reference image is updated for storage in the library;

[0134] An output module for pushing all the inspection images with anomalies for subsequent analysis.

[0135] Embodiment 3

[0136] This embodiment also provides an electronic device, including:

[0137] At least one processor; and

[0138] A memory storing instructions that, when executed by the at least one processor, cause the at least one processor to execute the positive sample filtering method combining image comparison and sensitive defect detection as described above.

[0139] In this embodiment, the electronic device may include, but is not limited to: personal computers, server computers, workstations, desktop computers, laptop computers, notebook computers, mobile computing devices, smart phones, tablet computers, cellular phones, personal digital assistants (PDAs), handheld devices, messaging devices, wearable computing devices, consumer electronic devices, and the like.

[0140] Embodiment 4

[0141] This embodiment also provides a machine-readable storage medium storing executable instructions that, when executed, cause the machine to execute the positive sample filtering method combining image comparison and sensitive defect detection as described above.

[0142] Specifically, a system or device equipped with a readable storage medium may be provided, on which software program codes for implementing the functions of any one of the above embodiments are stored, and the computer or processor of the system or device is caused to read and execute the instructions stored in the readable storage medium.

[0143] In this case, the program code read from the readable medium itself can implement the functions of any one of the above embodiments, so the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of this specification.

[0144] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Optionally, the program code may be downloaded from a server computer or a cloud via a communication network.

[0145] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0146] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0147] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing devices to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that realizes the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing devices, such that a series of operation steps are executed on the computer or other programmable devices to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable devices provide steps for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0149] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solutions of the present invention, rather than limitations on the specific implementation manners of the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the claims of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A positive sample filtering method combining image comparison and sensitive defect detection, characterized in that, The method includes: S1. Set a reference image, which is a pure and defect-free image; S2. Obtain the current inspection image; S3. Construct a siamese network change detection algorithm, input the current inspection image and the reference image into the siamese network change detection algorithm for comparison to determine whether there are pixel-level changes in the current inspection image compared with the reference image. If a changed area is detected, mark it as a potential defect area; S4. Construct a defect detection model, and use the defect detection model to perform low-confidence slice inference on the current inspection image and the reference image respectively. If a defect area is detected in the current inspection image, retain the defect area; if defect areas are detected in both the current inspection image and the reference image and the positions overlap, discard the defect area; S5. Merge the potential defect area obtained by the siamese network change detection algorithm and the retained defect area obtained by the defect detection model. If the potential defect area overlaps with the defect area, it is determined that the current inspection image is abnormal; if the potential defect area does not overlap with the defect area, it is determined that the current inspection image is a normal sample, and the reference image is updated and stored in the database; S6. Push all inspection images with abnormalities for subsequent analysis.

2. The positive sample filtering method combining image comparison and sensitive defect detection according to claim 1, characterized in that In S3, constructing the siamese network change detection algorithm specifically includes: using UNet as the network basis, adopting dilated convolution to increase the receptive field, and adding a CBAM attention mechanism to enhance the feature extraction ability.

3. The positive sample filtering method combining image comparison and sensitive defect detection according to claim 2, wherein The CBAM attention mechanism includes a channel attention map M c (F), and: M c (F) = σ(MLP(AvgPool(F)) + MLP(MaxPool(F))) (1); In Equation (1): σ(·) represents the sigmoid function, MLP represents a multi-layer perceptron, AvgPool(F) and MaxPool(F) represent the average pooling feature and the maximum pooling feature respectively; AvgPool(F) and MaxPool(F) are forwarded to a shared network composed of a multi-layer perceptron MLP, and after passing through the sigmoid function σ, a channel attention map M c (F) is generated.

4. The positive sample filtering method combining image comparison and sensitive defect detection according to claim 3, characterized in that The CBAM attention mechanism further includes a spatial attention map M s (F), and: M s (F) = σ(f 7×7 ([AvgPool(F)); MaxPool(F)])) (2); In formula (2): f 7×7 represents a convolution operation with a filter size of 7×7; AvgPool(F) and MaxPool(F) are aggregated to generate two 2D graphs, and through the sigmoid function σ, a spatial attention map M s (F) is generated.

5. The positive sample filtering method combining image comparison and sensitive defect detection according to claim 1, characterized in that In S3, constructing the siamese network change detection algorithm specifically further includes: adopting a mixed loss method of Dice Loss and Focal Loss, and adjusting the weights of the siamese network through error backpropagation; the loss function Loss of the siamese network change detection algorithm is: Loss = FL + DL (3); In formula (3): FL is the Focal Loss function, and DL is the Dice Loss function; Among them, the Focal Loss function is: FL(p t ) = -α t (1 - p t ) γ log(p t ) (4); In formulas (4)-(5): p is the probability of judging the positive sample y = 1, p t The function unifies p and 1-p, and the parameter α t is used to suppress the imbalance in the number of positive and negative samples, and the parameter γ is used to control the imbalance in the number of easy / difficult-to-distinguish samples; The Dice Loss function is: In formula (6): X is the prediction map, and Y is the label map, where |X∩Y| is the intersection between X and Y, and |X| and |Y| respectively represent the number of elements of X and Y. Among them, the coefficient of the numerator is 2, indicating that there is duplicate calculation of the common elements between X and Y in the denominator.

6. The positive sample filtering method combining image comparison and sensitive defect detection according to claim 1, characterized in that S4 specifically includes: S41: Cut both the current inspection image and the reference image into multiple slices, each slice having a partial overlapping area, and record the size, position of each slice, and its relative coordinate range in the image before slicing; S42: Pre-cut the image training samples obtained through manual annotation based on historical inspection images according to the size of the slices, and use the cut image training samples for model training to obtain a defect detection model, ensuring that any slice of the current inspection image and the reference image is consistent with the data scale of the image training samples; S43: Independently perform object detection inference on each slice of the current inspection image and the reference image. Use the defect detection model trained in S42 to detect each slice, and adopt a low-confidence inference method to generate the detection results of each slice, including the bounding box coordinates, class labels, and confidence scores of each slice. S44: Convert the bounding box coordinates of each slice of the current inspection image and the reference image from their local coordinate systems to the global coordinate system of the corresponding pre-sliced image to obtain the converted bounding box coordinates of each slice. S45: Calculate the intersection over union (IoU) between the converted bounding box coordinates of every two slices of the current inspection image and every two slices of the reference image. Compare the IoU result with a preset IoU threshold. If the IoU result exceeds the preset IoU threshold, it is determined that the converted bounding box coordinates of these two slices are repeated. According to the confidence scores of the slices, retain the bounding box coordinates corresponding to the slice with the higher confidence score, and the region formed by these bounding box coordinates is the defect region.

7. The positive sample filtering method combining image comparison and sensitive defect detection according to claim 1, characterized in that, In S5, the reference image library update strategy is as follows: If the current inspection image is a normal sample and the total number of reference image samples in the reference image library is less than n (where n is the capacity of the reference image library), directly store the current inspection image in the library. If the current inspection image is a normal sample and the total number of reference image samples in the reference image library is greater than or equal to n, calculate the SSIM similarity between the current inspection image and each reference image sample in the reference image library one by one, and replace the reference image sample with the highest similarity with the current inspection image to maintain the diversity of the reference image library.

8. An apparatus for implementing a positive sample filtering method that combines image comparison and sensitive defect detection, characterized in that, The device includes: An input module for setting a reference image, where the reference image is a pure defect-free image. An acquisition module for acquiring the current inspection image. A change detection module for constructing a siamese network change detection algorithm, inputting the current inspection image and the reference image into the siamese network change detection algorithm for comparison to determine whether there are pixel-level changes in the current inspection image compared with the reference image. If a changed area is detected, mark it as a potential defect area. A defect detection module for constructing a defect detection model, using the defect detection model to perform low-confidence slice inference on the current inspection image and the reference image respectively. If a defect area is detected in the current inspection image, retain the defect area. If defect areas are detected in both the current inspection image and the reference image and their positions overlap, discard the defect area. A result judgment module for merging the potential defect area obtained through the siamese network change detection algorithm and the retained defect area obtained through the defect detection model. If the potential defect area overlaps with the defect area, it is determined that the current inspection image is abnormal. If the potential defect area does not overlap with the defect area, it is determined that the current inspection image is a normal sample, and the reference image library is updated. An output module for pushing all inspection images with abnormalities for subsequent analysis.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory that stores instructions that, when executed by the at least one processor, cause the at least one processor to perform the positive sample filtering method combining image comparison and sensitive defect detection according to any one of claims 1 to 7.

10. A machine-readable storage medium, characterized in that, Executable instructions are stored on the machine-readable storage medium, and when the instructions are executed, they cause the machine to perform the positive sample filtering method combining image comparison and sensitive defect detection according to any one of claims 1 to 7.

Images