Industrial defect segmentation system based on deterministic perception and propagation

Abstract

The invention discloses an industrial defect segmentation system based on deterministic perception and propagation. The system comprises a feature extraction part, a deterministic perception part, a deterministic information enhancement part and a deterministic propagation part. Wherein the feature extraction part is used for extracting high-dimensional features of four levels of different scales; in the deterministic perception part, a deterministic perception module CAM obtains a deterministic attention graph and sends the deterministic attention graph to the feature enhancement part; the deterministic information enhancement part obtains space and channel attention improved target deterministic region features through four deterministic information enhancement modules CEM and sends the target deterministic region features to the deterministic propagation part; according to the deterministic propagation part, industrial defect prediction segmentation maps of different scales are obtained through three deterministic propagation modules CPM. According to the method, misleading caused by an uncertain area in the background can be avoided, interference of noise and redundant information in the background is avoided, industrial hidden defects can be detected and segmented more accurately, and a reliable quality control means is provided for industrial production.

Description

Technical Field

[0001] The present invention relates to an industrial defect detection system, in particular to an industrial defect segmentation system based on deterministic perception and propagation. Background Art

[0002] Industrial defects refer to various defects that can be observed and detected on the surface of industrial products due to the influence of various adverse factors such as mechanical impact, workpiece friction, oxidation corrosion, material defects, etc. in the industrial production process. Industrial defects may not only cause product performance failure, but also lead to production safety hazards. Therefore, industrial defect detection is of indispensable importance in industrial production.

[0003] With the rapid development of industrial intelligence, higher requirements are placed on the accuracy and efficiency of defect detection. At present, most defect recognition technologies still rely on machine-assisted manual detection, which has problems such as high error rate, poor timeliness, susceptibility to interference from human factors, lack of flexibility and applicability. With the rapid development of machine vision, its high accuracy, speed and high stability make it show great advantages in defect recognition and become a key means to achieve detection automation and intelligence.

[0004] In recent years, industrial defect detection based on deep learning methods has made significant research progress. From the perspective of data, in order to solve the problem of insufficient defect data, the small sample learning paradigm is used to distinguish different categories by learning the distribution differences between images, so that the model can better generalize and identify unknown categories; in order to solve the problem of difficult defect data labeling, weak supervision, self-supervision and other methods are used to use the feature information mined by the model itself as pseudo labels, and only pseudo labels can be used to achieve weak positioning of defects. From the perspective of model structure design, defect detection models based on convolutional neural networks (CNN), Transformer, and hybrid models of Transformer and CNN are designed, which provide a network foundation for extracting target features from complex image information and achieving more accurate defect detection. From the perspective of model optimization and improvement, researchers are committed to designing more efficient multi-scale feature fusion methods, and realizing adaptive extraction of different defect features through various attention mechanisms. In addition, considering the actual application needs, model pruning, model lightweighting and other technologies are also applied to model compression design to achieve real-time detection of defects in the production process.

[0005] Although existing methods can detect most easily identifiable industrial defects, it is difficult to accurately segment some defects that are highly concealed and have unclear edge contours. This is because existing methods often ignore the deterministic information generated by the model during the prediction process, which makes the model easily misled and confused by background areas similar to the target, and it is difficult to distinguish this uncertain area in the image. Therefore, the deterministic information predicted by the existing technical model is crucial for the complete identification and accurate segmentation of defects. Summary of the invention

[0006] The present invention provides an industrial defect segmentation system based on deterministic perception and propagation to solve the problem that the existing industrial defect detection and segmentation methods ignore the deterministic information generated by the model in the prediction process, causing the model to be easily misled by background areas similar to the target and making it difficult to completely segment the target part in the uncertain area in the image.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] An industrial defect segmentation system based on deterministic perception and propagation includes a feature extraction part, a deterministic perception part, a deterministic information enhancement part, and a deterministic propagation part; wherein the feature extraction part extracts high-dimensional features of four different scales from an industrial defect to be detected image, and sends the high-dimensional features of the fourth scale to the deterministic perception part, and sends the high-dimensional features of a total of four scales to the deterministic information enhancement part;

[0009] The deterministic perception part includes a deterministic perception module CAM, which first quantifies uncertainty based on high-dimensional features of the fourth-level scale by using the Monte Carlo Dropout method, thereby obtaining four predicted segmentation maps; then the deterministic perception module CAM obtains an average predicted map and an inferred uncertainty map based on the four predicted segmentation maps; then the deterministic perception module CAM obtains an inferred deterministic map based on the inferred uncertainty map, and obtains a predicted deterministic map based on the average predicted map; finally, the deterministic perception module CAM splices the inferred deterministic map and the predicted deterministic map, and then sequentially performs convolution and Min-Max normalization and multiplies the resultant map with the average predicted map, thereby obtaining a deterministic attention map and sending it to the feature enhancement part;

[0010] The deterministic information enhancement part includes four deterministic information enhancement modules CEM, and the four deterministic information enhancement modules CEM input high-dimensional features of four levels of scales one by one, and each deterministic information enhancement module CEM also inputs the deterministic attention map respectively; in each deterministic information enhancement module CEM, the size of the deterministic attention map is first adjusted to be consistent with the input high-dimensional features of the corresponding level scale, and then the target deterministic region in the high-dimensional features of the corresponding level scale is focused on based on the adjusted deterministic attention map through the spatial attention mechanism, thereby obtaining the spatial attention improved features; finally, each deterministic information enhancement module CEM focuses on the important channels related to the defect segmentation task in the spatial attention improved features through the channel attention mechanism, thereby obtaining the target deterministic region features of the spatial and channel attention improvements and sending them to the deterministic propagation part;

[0011] The deterministic propagation part includes three deterministic propagation modules CPM, among which the first deterministic propagation module CPM inputs the target deterministic regional features output by the deterministic information enhancement module CEM corresponding to the fourth-level scale high-dimensional features, and the target deterministic regional features output by the deterministic information enhancement module CEM corresponding to the third-level scale high-dimensional features, the second deterministic propagation module CPM inputs the target deterministic regional features output by the deterministic information enhancement module CEM corresponding to the second-level scale high-dimensional features and the improved features output by the first deterministic propagation module CPM, and the third deterministic propagation module inputs the target deterministic regional features output by the deterministic information enhancement module CEM corresponding to the first-level scale high-dimensional features and the improved features output by the second deterministic propagation module CPM;

[0012] In each deterministic propagation module CPM, the input higher-level scale features and lower-level scale features are firstly spliced ​​through channels, and then CBR processing and convolution adjustment channels are performed to obtain fused features; then each deterministic propagation module CPM separates the fused features into four groups of separated features by channel, and performs CBR processing through four branches, wherein the first group of separated features is processed by the first branch to obtain the first branch output features, the first branch output features are added to the second group of separated features, and then processed by the second branch to obtain the second branch output features, and the second branch output features are added to the third group of separated features. The features of the first branch are added and then processed by the third branch for CBR to obtain the output features of the third branch. The output features of the third branch are added to the separated features of the fourth group and then processed by the fourth branch for CBR to obtain the output features of the fourth branch. The output features of the first to fourth branches are spliced ​​by channel to obtain the spliced ​​features; then, each deterministic propagation module CPM adjusts the channel of the spliced ​​features by convolution and performs residual connection with the fusion features, and then performs CBR processing to integrate context information, thereby obtaining improved features; finally, each deterministic propagation module CPM performs convolution processing on the improved features to obtain an industrial defect prediction segmentation map;

[0013] Therefore, through the three deterministic propagation modules CPM in the deterministic propagation part, three industrial defect prediction segmentation maps of different scales are obtained, and the industrial defect prediction segmentation map output by the third deterministic propagation module CPM is used as the final prediction result.

[0014] Furthermore, the feature extraction part is a PVTv2 network.

[0015] Furthermore, the deterministic perception module CAM calculates the mean of the elements in the four predicted segmentation maps, thereby obtaining an average prediction map;

[0016] The deterministic perception module CAM calculates the standard deviation of the elements in the four predicted segmentation maps to obtain the inferred uncertainty map.

[0017] Furthermore, the deterministic perception module CAM uses Min-Max normalization to invert and add 1 to the inferred uncertainty graph, thereby obtaining the inferred deterministic graph;

[0018] The deterministic perception module CAM uses a Sigmoid function to activate the average prediction map, thereby obtaining a prediction deterministic map.

[0019] Furthermore, when focusing through the spatial attention mechanism in each deterministic information enhancement module CEM, the adjusted deterministic attention map is multiplied by the high-dimensional features of the corresponding level scale and then added to the high-dimensional features of the corresponding level scale, and then CBR processing is performed to obtain the spatial attention improved features.

[0020] Furthermore, when focusing through the channel attention mechanism in each deterministic information enhancement module CEM, the spatial attention improvement feature is first globally average pooled to obtain compressed information, and then the compressed information is linearly mapped and transformed, and then normalized through the Sigmoid function to obtain the channel attention, and then the channel attention is multiplied by the spatial attention improvement feature, and then CBR processing is performed to obtain the target deterministic regional features of spatial and channel attention improvement.

[0021] Furthermore, in each deterministic propagation module CPM of the deterministic propagation part, the CBR processing of the first branch includes 3×3 convolution, batch normalization, and ReLU activation, and the CBR processing of the second branch to the fourth branch includes 3×3 dilated convolution, batch normalization, and ReLU activation, respectively, and the dilation rate of the dilated convolution in the CBR processing from the second branch to the fourth branch increases gradually.

[0022] The present invention provides an industrial defect segmentation system CGNet based on deterministic perception and propagation, which consists of a feature extraction part, a deterministic perception part, a deterministic information enhancement part, and a deterministic propagation part. The CGNet of the present invention is a multi-stage model of perception-enhancement-refinement as a whole. In the process of defect segmentation by the CGNet of the present invention, recognition starts from the determined area and gradually propagates to the uncertain area, following a processing process from known to unknown, from simple to complex and the human visual recognition mechanism. This strategy helps to gradually and accurately identify and segment targets in complex images.

[0023] First, the present invention uses PVTv2 as the feature extraction part to extract image features. Subsequently, the deterministic perception part will identify the determined areas in the image and mine the deterministic clues about the target, such as clear boundaries, color or texture features, etc. After the deterministic perception part determines these areas, the deterministic information enhancement part enhances the model's attention to the determined areas in the target, while avoiding the interference of noise and redundant information in the background. Finally, the deterministic propagation part analyzes the relationship between these areas and the surrounding pixels, thereby inferring the properties of the uncertain areas and expanding to the surrounding uncertain areas, thereby gradually and accurately segmenting the target area to obtain the final segmentation result.

[0024] Compared with the prior art, the present invention has the following advantages:

[0025] The present invention proposes an industrial defect segmentation system based on deterministic perception and propagation. By introducing the deterministic perception part composed of the deterministic perception module CAM, the uncertainty existing in the inference process and results is quantified, so as to further mine the deterministic clues about the target in the image, better help the model to accurately learn the deterministic information about the target, and avoid being misled by the uncertain area in the background. By introducing the deterministic information enhancement part composed of the deterministic information enhancement module CEM, the features are enhanced using deterministic clues, and the model's attention to the determined area in the target is enhanced to avoid the interference of noise and redundant information in the background. By introducing the deterministic propagation part composed of the deterministic propagation module CPM, the surrounding uncertain areas are gradually inferred using known deterministic clues, so as to gradually improve the defect segmentation effect. Through the synergistic effect of these key modules, the present invention can more accurately detect and segment industrial hidden defects, and provide a reliable quality control method for industrial production. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the overall structure of the first embodiment of the present invention.

[0027] Figure 2 It is a schematic diagram of the structure of the deterministic perception module CAM in the first embodiment of the present invention.

[0028] Figure 3 It is a schematic diagram of the structure of each deterministic information enhancement module CEM in the first embodiment of the present invention.

[0029] Figure 4 It is a schematic diagram of the structure of each deterministic propagation module CPM in the first embodiment of the present invention.

[0030] Figure 5 It is a diagram of the experimental results of the second embodiment of the present invention. DETAILED DESCRIPTION

[0031] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0032] Embodiment 1

[0033] like Figure 1 As shown, this embodiment discloses an industrial defect segmentation system based on deterministic perception and propagation, named CGNet, which includes a feature extraction part, a deterministic perception part, a deterministic information enhancement part, and a deterministic propagation part. The specific description is as follows:

[0034] 1. Feature extraction part.

[0035] Due to its powerful feature extraction capability and high robustness, the PVTv2 network has become one of the mainstream visual Transformer benchmark models. Therefore, this embodiment CGNet uses the PVTv2-B5 network as the feature extraction part to extract multi-scale features of the image.

[0036] Assume that the size of the input industrial defect image I is W×H×3, where W and H represent the width and height of the image, respectively. In this embodiment, CGNet extracts four levels of high-dimensional features of different scales from the industrial defect image I through the PVTv2-B5 network. k ,k∈{1,2,3,4}, the resolution of each feature is The number of feature channels are 64, 128, 320 and 512. This set of features, from high resolution and weak semantics to low resolution and strong semantics, realizes all-round and multi-scale feature extraction of images.

[0037] The fourth-level high-dimensional feature f4 obtained by the PVTv2-B5 network is sent to the deterministic perception part, and the four-level high-dimensional features f1, f2, f3, and f4 obtained by the PVTv2-B5 network are sent to the deterministic information enhancement part.

[0038] 2. Deterministic perception part.

[0039] In this embodiment of CGNet, the deterministic perception part includes a deterministic perception module CAM. The deterministic perception module CAM is used to perceive the uncertainty in the inference process and results of the model, so as to further explore the deterministic clues about the target in the image, which is conducive to the model to accurately learn the deterministic information about the target and avoid being misled by the uncertain areas in the background, resulting in incorrect segmentation or confusion of the target. The structural diagram of the deterministic perception module CAM is shown in the figure. Figure 2 As shown in the figure, given the input features, the deterministic perception module performs preliminary segmentation of the target. During this process, the Monte Carlo Dropout method is used to quantify the uncertainty generated in the model inference process, and then the prediction uncertainty of the model is calculated through the classification probability in the segmentation map. The uncertainty is normalized and scaled between 0 and 1, and the certainty is obtained by taking the inverse and adding one. The certainty map represents the model's confidence in the classification of pixels in the corresponding area of ​​the image during inference and prediction. The prediction result map is multiplied by the certainty map to obtain the certainty map of the target area, that is, the deterministic clues about the target. The output of the deterministic perception module will be input into the deterministic information enhancement module to enhance the information of the features.

[0040] In this embodiment of CGNet, the data processing process of the deterministic perception module CAM is as follows:

[0041] (2.1) The fourth-level high-dimensional feature f4 extracted by the PVTv2-B5 network is input into the deterministic perception module CAM. The deterministic perception module CAM first uses the Monte Carlo Dropout method to quantify the uncertainty generated in the model inference process. Specifically, the deterministic perception module CAM uses a 1×1 convolution to perform a preliminary segmentation of the defect target in the fourth-level high-dimensional feature f4, sets the Dropout rate of the 1×1 convolution to 0.25, and performs 4 forward propagations, thereby obtaining 4 predicted segmentation maps M. i ,i∈{1,2,3,4}.

[0042] (2.2), then the deterministic perception module CAM is based on the four predicted segmentation maps M i Get the average prediction graph and the inferred uncertainty graph, then the deterministic perception module CAM obtains the inferred deterministic graph based on the inferred uncertainty graph, and obtains the predicted deterministic graph based on the average predicted graph. The specific description is as follows:

[0043] The deterministic perception module CAM calculates four predicted segmentation maps M i The mean of the elements in , thus obtaining the average prediction graph As shown in formula (1):

[0044]

[0045] The deterministic perception module CAM then uses the Sigmoid function to average the prediction graph Activate and obtain the classification probability p of each pixel being predicted as background or target i,j , thus obtaining the prediction certainty graph Classification probability p i,j The closer its value is to 0, the greater the probability that the pixel is classified as background; the classification probability p i,j The closer its value is to 1, the greater the probability that the pixel is classified as a target. When the classification probability is close to 0.5, the more difficult it is for the model to distinguish it as a target or background, the lower the prediction certainty of the point; when the classification probability is close to 0 or 1, the easier it is for the model to distinguish it as a target or background, the higher the prediction certainty of the point. Use the Sigmoid function to average the prediction graph The activation is shown in formula (2):

[0046]

[0047] In formula (2), The average prediction graph σ(·) is the Sigmoid activation function.

[0048] The deterministic perception module CAM calculates four predicted segmentation maps M i The standard deviation of the elements in , thus obtaining the inferred uncertainty map, and the inferred uncertainty map is normalized using Min-Max and then negated and added 1 to obtain the inferred certainty map As shown in formula (3):

[0049]

[0050] In formula (3), std(·) is the standard deviation function; Norm Min-Max (·) is Min-Max normalization.

[0051] (2.3) Finally, the deterministic perception module CAM infers the deterministic graph and the predicted certainty graph After concatenation by channel, 1×1 convolution is used for deterministic adaptive adjustment and selection, followed by Min-Max normalization and average prediction map Multiply them together to get the deterministic attention map MAP conf And sent to the feature enhancement part. Deterministic attention map MAP conf The closer the element value is to 1, the higher the certainty of the pixel in the target area; deterministic attention map MAP conf The closer the element value is to 0, the lower the certainty of the pixel in the target area. The processing process is shown in formula (4):

[0052]

[0053] In formula (4), is the average predicted segmentation map; To infer the deterministic graph; is the predicted deterministic graph; [·] is the channel concatenation operation; Conv(·) is a 1×1 convolution; Norm Min-Max (·) is Min-Max normalization.

[0054] 3. Deterministic information enhancement part.

[0055] In the present embodiment of CGNet, the deterministic information enhancement part includes four deterministic information enhancement modules CEM, which correspond to input high-dimensional features of four scales one by one, and each deterministic information enhancement module CEM also inputs the deterministic attention map respectively. The outputs of the four deterministic information enhancement modules CEM are sent to the deterministic propagation part for deterministic propagation.

[0056] Each deterministic information enhancement module CEM uses the spatial attention and channel attention mechanisms to enhance features using deterministic cues perceived by CAM, enhancing the model's attention to certain areas in the target and avoiding interference from background noise and redundant information. The structure of each deterministic information enhancement module CEM is as follows: Figure 3 As shown, the data processing process of each deterministic information enhancement module CEM is as follows:

[0057] (3.1) In each deterministic information enhancement module CEM, the deterministic attention map MAP is first interpolated by bilinear interpolation. conf The size sampling is adjusted to be consistent with the high-dimensional features of the corresponding level of the input, and the adjusted deterministic attention map MAP′ is obtained conf .

[0058] (3.2) In each deterministic information enhancement module CEM, the spatial attention mechanism is used based on the adjusted deterministic attention map MAP′ conf Focus on the target deterministic area in the high-dimensional features of the corresponding level, thereby obtaining the spatial attention improved feature F SA .

[0059] Specifically, when focusing through the spatial attention mechanism, the adjusted deterministic attention map MAP′ is first conf After multiplying with the high-dimensional features of the corresponding level scale, it is added to the high-dimensional features of the corresponding level scale to retain the detail information, and then CBR processing is performed. This CBR processing includes 3×3 convolution, batch normalization and ReLU activation operations, thereby obtaining the spatial attention improved feature F SA , as shown in formula (5):

[0060] F SA = CBR(MAP′ conf ×F+F) (5)

[0061] In formula (5), F is the high-dimensional feature of the corresponding level of the input; CBR is a 3×3 convolution, batch normalization and ReLU activation operation.

[0062] (3.3) In each deterministic information enhancement module CEM, the channel attention mechanism is used to focus on the important channels related to the defect segmentation task in the spatial attention improvement feature, thereby obtaining the target deterministic region feature F of the spatial and channel attention improvement SA+CA And sent to the deterministic propagation part.

[0063] Specifically, when focusing through the channel attention mechanism, the spatial attention feature F is first improved SAPerform global average pooling on each channel to obtain the maximum pixel value on each channel to compress the feature information and obtain compressed information. Then, use a one-dimensional 1×1 convolution to linearly map and transform the compressed information, and then use the Sigmoid function to normalize it to between 0 and 1, thereby obtaining channel attention. Then, the channel attention and spatial attention are combined to improve the feature F. SA After multiplication, CBR processing is performed, which includes 1×1 convolution, batch normalization and ReLU activation operations, thereby obtaining the target deterministic regional features F with improved spatial and channel attention. SA+CA , as shown in formula (6):

[0064] F SA+CA =CBR(Conv1d(GlobalAvgPool(F SA ))*F SA ) (6)

[0065] In formula (6), F SA is the spatial attention improvement feature of the input; GlobalAvgPool(·) is the global average pooling; Conv1d is the one-dimensional convolution; CBR is the 1×1 convolution, batch normalization and ReLU activation operation.

[0066] Finally, the target deterministic region features F SA+CA , and sent to the deterministic propagation part for deterministic propagation.

[0067] 4. Deterministic propagation part.

[0068] In the present embodiment CGNet, the deterministic propagation part includes three deterministic propagation modules CPM. Among the three deterministic propagation modules CPM, the first deterministic propagation module CPM inputs the target deterministic region feature F output by the deterministic information enhancement module CEM corresponding to the fourth-level high-dimensional feature SA+CA , and the target deterministic regional feature F output by the deterministic information enhancement module CEM corresponding to the third-level high-dimensional feature SA+CA The second deterministic propagation module CPM inputs the target deterministic region feature F output by the deterministic information enhancement module CEM corresponding to the second-level scale high-dimensional feature SA+CA As well as the improved features output by the first deterministic propagation module CPM, the third deterministic propagation module inputs the target deterministic region features F output by the deterministic information enhancement module CEM corresponding to the first-level scale high-dimensional features SA+CA and improved features of the output of the second deterministic propagation module CPM.

[0069] Each deterministic propagation module CPM is responsible for analyzing the relationship between these target deterministic regions and the surrounding pixels, thereby inferring the properties of the uncertain regions and expanding to the surrounding uncertain regions, thereby gradually and accurately segmenting the target region to obtain the final segmentation result. This propagation mechanism can avoid misleading uncertain regions and reduce the erroneous recognition and segmentation of non-target regions. The structure of each deterministic propagation module CPM is as follows: Figure 4 As shown, each deterministic propagation module CPM outputs the improved features and industrial defect prediction segmentation map of the corresponding level scale. The specific data processing process of each deterministic propagation module CPM is as follows:

[0070] (4.1) In order to maintain the spatial consistency of features, each deterministic propagation module CPM first uses bilinear interpolation to convert the input higher-level scale features F h Upsampling is adjusted to the same scale as the lower level feature F l The same size, and the adjusted higher-level scale feature F h With the lower-level scale feature F l After channel splicing, the spliced ​​features are obtained.

[0071] For the first deterministic propagation module CPM, its input is the target deterministic region feature F output by the deterministic information enhancement module CEM corresponding to the fourth-level high-dimensional feature SA+CA , and the target deterministic regional feature F output by the deterministic information enhancement module CEM corresponding to the third-level high-dimensional feature SA+CA , where the fourth-level scale corresponds to the target deterministic regional feature F SA+CA Considered as a higher-level scale feature F h , the target deterministic regional feature F corresponding to the third level scale SA+CA Considered as a lower-level scale feature F l Therefore, in the first deterministic propagation module CPM, the target deterministic regional feature F corresponding to the fourth level scale is SA+CA Adjusted to the target deterministic regional feature F corresponding to the third level scale SA+CA After the sizes are consistent, splicing can be carried out.

[0072] For the second deterministic propagation module CPM, its input is the target deterministic region feature F output by the deterministic information enhancement module CEM corresponding to the second-level scale high-dimensional feature SA+CA and the improved features output by the first deterministic propagation module CPM, where the improved features output by the first deterministic propagation module CPM are regarded as higher-level scale features F h , the target deterministic regional feature F corresponding to the second level scale SA+CA Considered as a lower-level scale feature F lTherefore, in the second deterministic propagation module CPM, the improved features output by the first deterministic propagation module CPM are adjusted to the target deterministic regional features F corresponding to the second level scale SA+CA After the sizes are consistent, splicing can be carried out.

[0073] For the third deterministic propagation module CPM, its input is the target deterministic region feature F output by the deterministic information enhancement module CEM corresponding to the first-level scale high-dimensional feature SA+CA and the improved features output by the second deterministic propagation module CPM, where the improved features output by the second deterministic propagation module CPM are regarded as higher-level scale features F h , the target deterministic region feature F output by the deterministic information enhancement module CEM corresponding to the first-level scale high-dimensional feature SA+CA Considered as a lower-level scale feature F l Therefore, in the third deterministic propagation module CPM, the improved features output by the second deterministic propagation module CPM are adjusted to the target deterministic regional features F corresponding to the first level scale SA+CA After the sizes are consistent, splicing can be carried out.

[0074] Then, in each deterministic propagation module CPM, CBR processing is performed on the concatenated features to extract the fused features. This CBR processing includes 3×3 convolution, batch normalization, and ReLU activation. Next, the channel is adjusted through 1×1 convolution to obtain the fused feature C, as shown in formula (7):

[0075] C=Conv(CBR([U(F h ),F l ])) (7)

[0076] In formula (7), U(·) represents the target deterministic regional feature F corresponding to the higher-level high-dimensional feature SA+CA [·] represents channel concatenation; CBR represents 3×3 convolution, batch normalization, and ReLU activation operations; Conv represents 1×1 convolution.

[0077] (4.2) In order to retain image detail information during feature extraction, each deterministic propagation module CPM separates the fused feature C into four groups of separated features c according to the channel. i ,i∈{1,2,3,4}, and CBR processing is performed through four parallel branches.

[0078] In each deterministic propagation module CPM, the CBR processing of the second to fourth branches of the four parallel branches includes convolutions with different dilation rates to extract local and global features of the image under different receptive fields. Among them, the CBR processing of the first branch includes 3×3 convolution, batch normalization, and ReLU activation, and the CBR processing of the second to fourth branches includes 3×3 dilated convolution, batch normalization, and ReLU activation, respectively, and the dilation rate of the dilated convolution in the CBR processing from the second branch to the fourth branch increases.

[0079] The first branch output feature c′1 is obtained after the first group of separated features c1 is processed by CBR in the first branch. The first branch output feature c′1 is added to the second group of separated features c2 element by element and then processed by CBR in the second branch to obtain the second branch output feature c′2. The second branch output feature c′2 is added to the third group of separated features c3 element by element and then processed by CBR in the third branch to obtain the third branch output feature c′3. The third branch output feature c′3 is added to the fourth group of separated features c4 element by element and then processed by CBR in the fourth branch to obtain the fourth branch output feature c′4. The semantic information is enhanced by element-by-element addition. The enhancement of existing semantic information and the expansion of the receptive field from local to global are achieved by adding feature elements across branches and gradually expanding convolution, and the gradual propagation of current deterministic information to the surrounding uncertain areas are achieved. Then, the output features c′1, c′2, c′3, and c′4 of the first to fourth branches are spliced ​​by channel to obtain the spliced ​​feature C′. The specific process is shown in formulas (8)-(10).

[0080] c′1=CBR(c1) (8)

[0081] c′ k =DCBR d＝k (c′ k-1 +c k ),k∈{2,3,4} (9)

[0082] C′=[c′1,c′2,c′3,c′4] (10)

[0083] In formulas (8)-(10), CBR is a 3×3 convolution, batch normalization, and ReLU activation operation; DCBR d＝k is the 3×3 dilated convolution with dilation rate k, batch normalization and ReLU activation operation, i.e., the CBR processing of the corresponding branch; [·] is channel concatenation.

[0084] (4.3) In each deterministic propagation module CPM, the concatenated feature C′ is scaled by 1×1 convolution, and then residually connected with the fused feature C. Then, CBR processing is performed to integrate context information, thereby obtaining the improved feature F′ of the corresponding scale of the deterministic propagation module CPM. h , this CBR processing includes 3×3 convolution, batch normalization and ReLU activation. Finally, the improved feature F′ obtained by each deterministic propagation module CPM h Perform 1×1 convolution processing to obtain the industrial defect prediction segmentation map P of the corresponding scale. Specifically, it is shown in formula (11):

[0085] P = Conv(F′ h )=Conv(CBR(Conv(C′)+C)) (11)

[0086] In formula (11), CBR is a 3×3 convolution, batch normalization and ReLU activation operation; Conv is a 1×1 convolution.

[0087] In this embodiment, the improved feature F′ obtained by the first deterministic propagation module CPM h Input to the second deterministic propagation module CPM as a higher-level scale feature F h , the improved feature F′ obtained by the second deterministic propagation module CPM h Input to the third deterministic propagation module CPM as a higher-level scale feature F h , thereby combining low-level features to further refine predictions.

[0088] Thus, through the three deterministic propagation modules CPM in the deterministic propagation part, three industrial defect prediction segmentation maps of different scales are obtained, and the industrial defect prediction segmentation map output by the third deterministic propagation module CPM is used as the final prediction result. And when CGNet in this embodiment is trained, the three industrial defect prediction segmentation maps all participate in supervised training as part of the loss function, but the final prediction result is the industrial defect prediction segmentation map output by the third deterministic propagation module CPM.

[0089] The loss function of CGNet training in this embodiment is described as follows:

[0090] Different from the standard IoU and BCE losses for segmentation tasks, and The weighted intersection-over-union (IOU) and binary cross-entropy (BCE) losses represent global constraints and local (pixel-level) constraints, which increase the weights of indistinguishable pixels and can highlight the importance of difficult-to-distinguish regions. and The loss is calculated as shown in formulas (12) and (13):

[0091]

[0092] In formulas (12) and (13), I() is the indicator function; γ is an adjustable hyperparameter; l∈{0,1} represents two labels; G ij and p ij Respectively represent the predicted value and true value of the pixel at position (i, j) in the image; Pr(p ij =l|ψ) represents the predicted probability.

[0093] α ij Represents the element at position (i, j) of the weight α matrix. and In the example, each pixel will be assigned a weight α. Pixels that are difficult to distinguish correspond to larger α, and pixels that are easy to distinguish will be assigned smaller α. α can be regarded as an indicator of pixel importance, which is calculated based on the difference between the center pixel and its surrounding pixels. α ij As shown in formula (14):

[0094]

[0095] In formula (14), A ij represents the area around the pixel (i, j) in the truth map G (usually an 8-neighborhood); (m, n) represents the area belonging to A ij For a point in the region, m is the horizontal coordinate of the point and n is the vertical coordinate; G mn Represents the predicted value of the position of the true value map (m, n); in formula (14) This means area A ij The sum of the true values ​​of all points in . For all pixels, α ij ∈[0,1]. If α ij If α is large, then the pixel at (i, j) is very different from its surrounding pixels. Therefore, it is an important pixel (e.g., an edge or a hole) and deserves more attention. On the contrary, if α ij If it is very small, it is considered an ordinary pixel and not worthy of attention.

[0096] Based on the above theory, the output prediction graph is upsampled to the same size as the true value graph G. The loss function L used in the final training of CGNet in this embodiment is shown in formula (15):

[0097]

[0098] In formula (15), G is the target truth map; is the average prediction map in CAM; P iis the prediction graph output by the i-th CPM.

[0099] Embodiment 2

[0100] This embodiment is a comparative experimental example of CGNet described in Embodiment 1 and other sota methods in the field of industrial camouflage defect segmentation, and the specific description is as follows:

[0101] (A) Experimental setup

[0102] This experimental example uses NVIDIA RTX 3090GPU (24G memory), and the CGNet model code is built based on Python and Pytorch framework. In addition, CGNet described in Example 1 uses Adam optimizer to update model parameters, the learning rate is initially set to 1e-4, 150 epochs are trained, the batch size is set to 8, and the input image size is uniformly adjusted to 384×384 pixels.

[0103] (B) Datasets and evaluation metrics

[0104] This experimental example uses CDS2K as the CGNet training and test dataset described in Example 1. CDS2K was released by Fan Deng et al. in 2023. It is the first camouflaged industrial defect segmentation benchmark dataset for application scenarios, compiled from various established industrial defect libraries. The dataset has a total of 2492 samples, including 1330 positive samples and 1162 negative samples. The training set and test set are divided into 8:2 ratios, and 1064 of the positive samples are randomly selected as training sets and 266 as test sets for algorithm performance verification. Figure 5 The segmentation results of some test data are shown.

[0105] This experimental example is used to evaluate the prediction effect of the CGNet model described in Example 1. The experimental indicators used mainly include mean absolute error (M), F measure (F β ), Enhanced Matching Index (E φ ), structural measurement (S α ). Among them, the smaller the MAE value and the larger the other index values, the better the model performance.

[0106] (C) Experimental performance comparison

[0107] In this experimental example, the CGNet described in Example 1 and other sota methods in the field of industrial camouflage defect segmentation (from the benchmark method introduced in the data set related paper Advances in Deep Concealed Scene Understanding) were compared in terms of quantitative indicators. The data set division and experimental hyperparameter settings of other methods are the same as those of this experiment. The comparative experimental results are shown in Table 1.

[0108] Table 1 Comparative experimental results

[0109]

[0110] from Figure 5 It can be seen that the CGNet described in Example 1 shows good segmentation effect when detecting various types of industrial defects and achieves accurate hidden defect segmentation. As can be seen from Table 1, in comparison with the existing cutting-edge hidden detection methods, the CGNet described in Example 1 has achieved significant improvements in various indicators, demonstrating its superiority and feasibility in actual industrial applications.

[0111] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. The embodiments described in the present invention are merely descriptions of the preferred embodiments of the present invention, and do not limit the concept and scope of the present invention. The various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction, and such combinations should also be regarded as the contents disclosed in the present disclosure as long as they do not violate the concept of the present invention. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

[0112] The present invention is not limited to the specific details of the above-mentioned embodiments. Within the technical concept of the present invention and without departing from the design concept of the present invention, various modifications and improvements made to the technical solution of the present invention by technical personnel in this field should fall within the protection scope of the present invention. The technical contents for which protection is sought in the present invention have been fully recorded in the claims.

Claims

1. An industrial defect segmentation system based on deterministic perception and propagation, characterized in that: It includes feature extraction part, deterministic perception part, deterministic information enhancement part and deterministic propagation part; The feature extraction part extracts high-dimensional features of four different scales from the industrial defect to be detected image, and sends the high-dimensional features of the fourth scale to the deterministic perception part, and sends the high-dimensional features of a total of four scales to the deterministic information enhancement part; The deterministic perception part includes a deterministic perception module CAM, which first quantifies uncertainty based on high-dimensional features of the fourth-level scale by using the Monte Carlo Dropout method, thereby obtaining four predicted segmentation maps; then the deterministic perception module CAM obtains an average predicted map and an inferred uncertainty map based on the four predicted segmentation maps; then the deterministic perception module CAM obtains an inferred deterministic map based on the inferred uncertainty map, and obtains a predicted deterministic map based on the average predicted map; finally, the deterministic perception module CAM splices the inferred deterministic map and the predicted deterministic map, and then sequentially performs convolution and Min-Max normalization and multiplies the resultant map with the average predicted map, thereby obtaining a deterministic attention map and sending it to the feature enhancement part; The deterministic information enhancement part includes four deterministic information enhancement modules CEM, and the four deterministic information enhancement modules CEM input high-dimensional features of four levels of scales one by one, and each deterministic information enhancement module CEM also inputs the deterministic attention map respectively; each deterministic information enhancement module CEM first adjusts the size of the deterministic attention map to be consistent with the input high-dimensional features of the corresponding level scale, and then focuses on the target deterministic area in the high-dimensional features of the corresponding level scale based on the adjusted deterministic attention map through the spatial attention mechanism, thereby obtaining the spatial attention improved features; Finally, each deterministic information enhancement module CEM focuses on the important channels related to the defect segmentation task in the spatial attention improvement features through the channel attention mechanism, thereby obtaining the target deterministic regional features of the spatial and channel attention improvements and sending them to the deterministic propagation part; The deterministic propagation part includes three deterministic propagation modules CPM, among which the first deterministic propagation module CPM inputs the target deterministic regional features output by the deterministic information enhancement module CEM corresponding to the fourth-level scale high-dimensional features, and the target deterministic regional features output by the deterministic information enhancement module CEM corresponding to the third-level scale high-dimensional features, the second deterministic propagation module CPM inputs the target deterministic regional features output by the deterministic information enhancement module CEM corresponding to the second-level scale high-dimensional features and the improved features output by the first deterministic propagation module CPM, and the third deterministic propagation module inputs the target deterministic regional features output by the deterministic information enhancement module CEM corresponding to the first-level scale high-dimensional features and the improved features output by the second deterministic propagation module CPM; In each deterministic propagation module CPM, first, the input higher-level scale features and lower-level scale features are spliced ​​through channels, and then CBR processing and convolution adjustment channels are performed to obtain fused features; then, each deterministic propagation module CPM separates the fused features into four groups of separated features by channels, and performs CBR processing through four branches, wherein the first group of separated features is processed by the first branch to obtain the first branch output feature, the first branch output feature is added to the second group of separated features and then processed by the second branch to obtain the second branch output feature, the second branch output feature is added to the third group of separated features and then processed by the third branch to obtain the third branch output feature, the third branch output feature is added to the fourth group of separated features and then processed by the fourth branch to obtain the fourth branch output feature, and the first to fourth branch output features are spliced ​​by channels to obtain spliced ​​features; then, each deterministic propagation module CPM adjusts the channels of the spliced ​​features through convolution, performs residual connection with the fused features, and then performs CBR processing to integrate context information, thereby obtaining improved features; Finally, each deterministic propagation module CPM performs convolution processing on the improved features to obtain the industrial defect prediction segmentation map; Therefore, through the three deterministic propagation modules CPM in the deterministic propagation part, three industrial defect prediction segmentation maps of different scales are obtained, and the industrial defect prediction segmentation map output by the third deterministic propagation module CPM is used as the final prediction result.

2. According to claim 1, an industrial defect segmentation system based on deterministic perception and propagation is characterized in that: The feature extraction part is a PVTv2 network.

3. The industrial defect segmentation system based on deterministic perception and propagation according to claim 1 is characterized in that: The deterministic perception module CAM calculates the mean of the elements in the four prediction segmentation maps, thereby obtaining an average prediction map; The deterministic perception module CAM calculates the standard deviation of the elements in the four predicted segmentation maps to obtain the inferred uncertainty map.

4. The industrial defect segmentation system based on deterministic perception and propagation according to claim 1 is characterized in that: The deterministic perception module CAM uses Min-Max normalization to infer the uncertainty graph and then inverts and adds 1 to obtain the inferred deterministic graph; The deterministic perception module CAM uses a Sigmoid function to activate the average prediction map, thereby obtaining a prediction deterministic map.

5. The industrial defect segmentation system based on deterministic perception and propagation according to claim 1 is characterized in that: When focusing through the spatial attention mechanism in each deterministic information enhancement module CEM, the adjusted deterministic attention map is multiplied by the high-dimensional features of the corresponding level scale and then added to the high-dimensional features of the corresponding level scale, and then CBR processing is performed to obtain the spatial attention improved features.

6. The industrial defect segmentation system based on deterministic perception and propagation according to claim 1 is characterized in that: When focusing through the channel attention mechanism in each deterministic information enhancement module CEM, the spatial attention improvement feature is first globally averaged pooled to obtain compressed information, and then the compressed information is linearly mapped and transformed, and then normalized by the Sigmoid function to obtain the channel attention. The channel attention is then multiplied by the spatial attention improvement feature and then CBR processing is performed to obtain the target deterministic regional features of spatial and channel attention improvement.

7. The industrial defect segmentation system based on deterministic perception and propagation according to claim 1 is characterized in that: In each deterministic propagation module CPM of the deterministic propagation part, the CBR processing of the first branch includes 3×3 convolution, batch normalization, and ReLU activation, and the CBR processing of the second branch to the fourth branch includes 3×3 dilated convolution, batch normalization, and ReLU activation, respectively, and the dilation rate of the dilated convolution in the CBR processing from the second branch to the fourth branch increases gradually.

Images