Weak scratch detection method and device, electronic equipment and storage medium

Abstract

The invention provides a weak scratch detection method and device, electronic equipment and a storage medium, and effectively solves the problems that an existing unsupervised learning method is insufficient in weak scratch detection and positioning precision of an aluminum plate and has a significant difference from a real situation. The method comprises the following steps: inputting an image to be tested into a pre-trained weak scratch detection model; extracting image features in the to-be-tested image to obtain a feature map set, and selecting a plurality of target feature maps from the feature map set; performing multi-scale fusion on the plurality of target feature maps to obtain a set feature map, and expanding the set feature map to obtain a plurality of feature image blocks and embedded feature vectors; and performing calculation based on a target embedded feature vector selected from the plurality of embedded feature vectors and a normal embedded feature vector to obtain an abnormal score graph, obtaining a scratch detection result based on the abnormal score graph, and evaluating the scratch detection result to complete weak scratch detection on the to-be-tested aluminum plate.

Description

Technical Field

[0001] The present application relates to the technical field of metal material surface detection, and specifically to a method, device, electronic device and storage medium for detecting weak scratches. Background Art

[0002] Industrial aluminum sheet is a metal material with excellent corrosion resistance, high strength, and hardness. Currently, in industrial manufacturing, minor surface defects on aluminum sheet, especially faint scratches, can affect the material's mechanical properties and structural integrity, leading to equipment failure or product quality issues. These rare scratches are difficult to detect through manual inspection, resulting in high detection costs and low efficiency.

[0003] Therefore, machine vision-based scratch detection technology is crucial for aluminum sheet quality control. Scratches are typically generated during the aluminum sheet manufacturing process, appearing as irregular straight or curved lines. These scratches are sensitive to optical interference and have fuzzy edges, significantly increasing the difficulty of detection and localization. Currently, deep learning methods based on feature extraction are widely used in computer vision inspection tasks due to their excellent performance in handling complex backgrounds and detecting faint defects. However, these methods, primarily based on supervised learning, have several unavoidable limitations: 1) high labeling costs; 2) scarcity and unknown number of defect samples; and 3) the potential for inadvertent introduction of label noise during data labeling. The detection quality of supervised learning methods relies heavily on the quality of the labeled data and the availability of diverse labels. While these supervised methods have achieved significant results, they still suffer from numerous limitations, such as high labeling costs and the susceptibility to label noise. Furthermore, the scarcity of faint scratch samples poses significant challenges for detection. Consequently, both research and industry are investing significant effort in the development of unsupervised detection technologies, which are becoming a new trend in defect detection.

[0004] In the context of deep learning methods, unsupervised learning methods only use normal image samples for training during the training phase. There are no defect samples and no need for any annotation information. This eliminates the annotation cost and avoids the influence of annotation bias and noise. Therefore, it is suitable for scenarios where defect samples are scarce in the field of industrial inspection. In addition, in actual industrial scenarios, the changes in local pixels of defect images are close to the texture background, and unsupervised learning methods still face significant challenges. First, there is the problem of training data distribution. The lack of specific normal training samples will lead to false detections. At the same time, scratches and cracks are usually subtle, with blurred edges, and only occupy a small area. Due to the scarcity of pixel-level supervision labels, accurate defect contour segmentation is challenging. Therefore, most unsupervised learning methods are not accurate enough for detecting and locating weak scratches, which is significantly different from the actual situation. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a method, device, electronic device and storage medium for detecting weak scratches. This method, device, electronic device and storage medium for detecting weak scratches effectively solve the problem that the current unsupervised learning method has insufficient accuracy in detecting and locating weak scratches on aluminum plates, and there is a significant difference from the actual situation.

[0006] In a first aspect, an embodiment of the present application provides a method for detecting weak scratches, the method comprising:

[0007] Obtaining a test image of the aluminum plate to be tested, and inputting the test image into a weak scratch detection model pre-trained based on multiple normal images of sample aluminum plates; the weak scratch detection model is established based on a deep convolutional neural network;

[0008] Extracting image features from the image to be tested to obtain a feature map set, and selecting multiple target feature maps from the feature map set according to a preset selection method; different target feature maps have different scales; target feature maps of different scales have different image features;

[0009] Multi-scale fusing the multiple target feature maps to obtain a set feature map, expanding the set feature map to obtain multiple feature image blocks, and reducing the dimensions of the multiple feature image blocks to obtain corresponding embedded feature vectors;

[0010] Calculation is performed based on a target embedded feature vector selected from a plurality of embedded feature vectors and a normal embedded feature vector to obtain an anomaly score map, a scratch detection result is obtained based on the anomaly score map, and the scratch detection result is evaluated to complete the weak scratch detection of the aluminum plate to be tested.

[0011] In combination with the first aspect, an embodiment of the present application provides a first possible implementation of the first aspect, wherein the calculation based on the target embedded feature vector selected from the multiple embedded feature vectors and the preset normal embedded feature vector to obtain the anomaly score map includes:

[0012] calling a normal embedding feature vector generated when training the faint scratch detection model, and inputting the normal embedding feature vector and the target embedding feature vector into a preset scratch calculation network;

[0013] The normal embedded feature vector and the target embedded feature vector are processed based on the scratch calculation network to obtain an anomaly score map, and a scratch detection result is determined based on a value in the anomaly score map.

[0014] In combination with the first aspect, the embodiment of the present application provides a second possible implementation of the first aspect, wherein determining the scratch detection result based on the values in the anomaly score map includes:

[0015] Determining whether the value corresponding to the abnormal score map meets the preset weak scratch existence condition;

[0016] If so, position information of the faint scratch is located based on the anomaly score map to generate a scratch detection result based on the position information.

[0017] In combination with the first aspect, the embodiment of the present application provides a third possible implementation of the first aspect, wherein locating the position information of the faint scratch based on the anomaly score map includes:

[0018] Setting different first preprocessing thresholds to perform corresponding processing on the anomaly score map based on the different first preprocessing thresholds;

[0019] Matching is performed based on the preprocessed anomaly score map with the image to be tested of the aluminum plate to locate the position information of the weak scratch on the image to be tested based on the position information of the weak scratch on the anomaly score map.

[0020] In combination with the first aspect, the embodiment of the present application provides a fourth possible implementation of the first aspect, wherein the evaluating the scratch detection result includes:

[0021] Evaluate the scratch detection result based on multiple dimensions to obtain evaluation results of corresponding dimensions;

[0022] The evaluation results of the multiple dimensions are combined to obtain a final evaluation result, so as to determine the result of the scratch detection based on the final evaluation result.

[0023] In combination with the first aspect, the embodiment of the present application provides a fifth possible implementation of the first aspect, wherein the multi-scale fusion of the multiple target feature maps to obtain the aggregate feature map includes:

[0024] Determining a first fusion method and a second fusion method for the target feature map according to different types of the target feature map;

[0025] The target feature map is fused according to the first fusion method and the second fusion method to obtain a set feature map.

[0026] In combination with the first aspect, an embodiment of the present application provides a sixth possible implementation of the first aspect, wherein calculating a target embedded feature vector selected from multiple embedded feature vectors includes:

[0027] Randomly generate a plurality of sequence numbers and bind them one-to-one with the embedded feature vectors so that the sequence numbers correspond one-to-one with the embedded feature vectors;

[0028] A preset number of sequence numbers are randomly selected, and the corresponding embedded feature vectors are determined as the target embedded feature vectors.

[0029] In a second aspect, an embodiment of the present application provides a weak scratch detection device, comprising:

[0030] An acquisition module is used to acquire a test image of the aluminum plate to be tested, and input the test image into a weak scratch detection model pre-trained based on multiple normal images of sample aluminum plates; the weak scratch detection model is established based on a deep convolutional neural network;

[0031] An extraction module is used to extract image features from the image to be tested to obtain a feature map set, and select multiple target feature maps from the feature map set according to a preset selection method; different target feature maps have different scales; target feature maps of different scales have different image features;

[0032] an expansion module for fusing the multiple target feature maps at multiple scales to obtain a set feature map, expanding the set feature map to obtain multiple feature image blocks, and reducing the dimensions of the multiple feature image blocks to obtain corresponding embedded feature vectors;

[0033] A calculation module is used to perform calculations based on a target embedded feature vector selected from a plurality of embedded feature vectors and a normal embedded feature vector to obtain an anomaly score map, obtain a scratch detection result based on the anomaly score map, and evaluate the scratch detection result to complete the weak scratch detection of the aluminum plate to be tested.

[0034] In a third aspect, an embodiment of the present application provides an electronic device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of any one of the methods for detecting weak scratches are performed.

[0035] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program executes any one of the steps of the faint scratch detection method.

[0036] The embodiment of the present application provides a weak scratch detection method, which first obtains a test image of an aluminum plate to be tested, and inputs the test image into a weak scratch detection model pre-trained based on multiple normal images of sample aluminum plates; the weak scratch detection model is established based on a deep convolutional neural network; secondly, the image features in the test image are extracted to obtain a feature map set, and multiple target feature maps are selected from the feature map set according to a preset selection method; different target feature maps have different scales; target feature maps of different scales have different image features; then the multiple target feature maps are multi-scale fused to obtain a set feature map, the set feature map is expanded to obtain multiple feature image blocks, and the multiple feature image blocks are reduced. The corresponding embedded feature vector is obtained based on the dimension of the target embedded feature vector selected from the multiple embedded feature vectors and the normal embedded feature vector. Finally, calculation is performed to obtain an abnormal score map, a scratch detection result is obtained based on the abnormal score map, and the scratch detection result is evaluated to complete the detection of weak scratches on the aluminum plate to be tested. The scratch detection result includes the position information of the scratch, thereby achieving the effect of weak scratch detection and positioning based on unsupervised learning, and ensuring the effectiveness and accuracy of detection and positioning, thereby ensuring that the detected weak scratches are consistent with the actual situation, effectively solving the problem that the current unsupervised learning method has insufficient accuracy in the detection and positioning of weak scratches on aluminum plates, and there is a significant difference from the actual situation. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 A schematic diagram of the process of a first weak scratch detection method provided in an embodiment of the present application is shown;

[0039] Figure 2 A schematic diagram of image feature extraction provided by an embodiment of the present application is shown;

[0040] Figure 3 A schematic diagram of multi-scale feature fusion provided by an embodiment of the present application is shown;

[0041] Figure 4 A schematic diagram of feature splicing provided by an embodiment of the present application is shown;

[0042] Figure 5 A schematic diagram of Gaussian distribution learning of embedded feature vectors provided in an embodiment of the present application is shown;

[0043] Figure 6 A schematic diagram of random sampling provided by an embodiment of the present application is shown;

[0044] Figure 7 A schematic diagram of a test data set provided in an embodiment of the present application is shown;

[0045] Figure 8 A schematic diagram of scratch detection provided by an embodiment of the present application is shown;

[0046] Figure 9 A schematic diagram of scratch positioning provided in an embodiment of the present application is shown;

[0047] Figure 10 The following is a structural block diagram of a first weak scratch detection device provided by an embodiment of the present application;

[0048] Figure 11 A structural block diagram of a first electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of illustration and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can be implemented out of sequence, and steps without logical context can be reversed or implemented simultaneously. In addition, those skilled in the art, under the guidance of the contents of this application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0050] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0051] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0052] In real-world industrial scenarios, the variations in local pixels in defect images closely resemble the textured background, posing significant challenges for unsupervised learning methods. First, there's the issue of training data distribution. The lack of specific, normal training samples can lead to false detections. Furthermore, scratches and cracks are often subtle, with blurred edges and occupying only a small area. Due to the scarcity of pixel-level supervised labels, accurate defect contour segmentation is challenging. Consequently, most unsupervised learning methods lack sufficient accuracy for detecting and locating faint scratches, significantly differing from real-world conditions.

[0053] Based on this, the embodiments of the present application provide a method, device, electronic device and storage medium for detecting weak scratches, which are described below through embodiments.

[0054] Example 1

[0055] To facilitate understanding of this embodiment, a weak scratch detection method disclosed in the embodiment of this application is first introduced in detail. Figure 1 The flowchart of a weak scratch detection method shown in FIG. 1 is a flowchart of a weak scratch detection method provided by the present application, and the method includes:

[0056] S101. Obtain a test image of an aluminum plate to be tested, and input the test image into a weak scratch detection model pre-trained based on multiple normal images of sample aluminum plates; the weak scratch detection model is established based on a deep convolutional neural network;

[0057] S102, extracting image features from the image to be tested to obtain a feature map set, and selecting multiple target feature maps from the feature map set according to a preset selection method; different target feature maps have different scales; target feature maps of different scales have different image features;

[0058] S103, multi-scale fusing the multiple target feature maps to obtain a collective feature map, expanding the collective feature map to obtain multiple feature image blocks, and reducing the dimensions of the multiple feature image blocks to obtain corresponding embedded feature vectors;

[0059] S104. Calculate a target embedded feature vector selected from a plurality of embedded feature vectors and a normal embedded feature vector to obtain an abnormal score map, obtain a scratch detection result based on the abnormal score map, and evaluate the scratch detection result to complete weak scratch detection on the aluminum plate to be tested.

[0060] In step S101, the weak scratch detection method provided in the present application is implemented based on the aluminum plate surface scratch detection device, which includes software, signal control equipment, data acquisition and mechanical execution devices. The mechanical execution part is composed of a precision motor axis XYZ and a 3D motion module, and is equipped with an industrial camera, lens, light source and a manual motion axis machine vision module. This machine vision module is mainly composed of a micro-axis XY and a spectral confocal sensor probe, which is driven by a 3D motion module to collect a 2D image of the surface of the aluminum plate to be tested, that is, to obtain the image to be tested of the aluminum plate to be tested. The image to be tested is a 2D image, and the size of the image to be tested x is represented by H×W×C. The image to be tested is input into a weak scratch detection model that has been pre-trained based on multiple normal images of sample aluminum plates. The normal image of the sample aluminum plate is an aluminum plate with high reflective properties without weak scratches. Plate image, since the weak scratch detection model of the present application is pre-trained with multiple normal images of sample aluminum plates, after the test image with weak scratches is input into the weak scratch detection model for detection, the scratch detection result is abnormal based on the presence of scratches. It is established based on a deep convolutional neural network, wherein the specific training process is mature in the prior art and will not be elaborated here. The weak scratch detection model takes the ResNet50 model as an example, which includes L convolutional layers, and the network structure consists of a convolutional layer, batch normalization (BN) and rectified linear units (ReLU).

[0061] To address the interference caused by the high reflectivity of the aluminum sheet's metal surface, the experiment employed a coaxial light source for sampling, providing a clearer view of the surface. The aluminum sheet surface scratch detection device used an industrial camera (MER2-1220-32U3M-L) with a resolution of 4024 × 3036 pixels (12.2 megapixels) and a 30-millisecond exposure time to capture images. After calibration, the pixel equivalent coefficient was 6.2761 μm. The lens (Myutron-HF3514V-2) had a focal length of 35 mm, resulting in an actual working distance of 110 mm.

[0062] In step S102, after the image to be tested is processed by the faint scratch detection model, the L-layer convolution layer extracts the image features in the image to be tested based on the unsupervised learning UFFDM feature extraction method to obtain a feature map set, which is recorded as Among them, L is the feature map output by the Lth convolutional layer. The feature maps output by different convolutional layers have different sizes and spatial receptive fields, such as Figure 2As shown. Among them, the low-level shallow network, due to its relatively small receptive field, mainly captures the texture details of the image; while the high-level deep network, due to its larger receptive field, tends to capture the global or higher-level semantic information in the image, usually focusing on the overall structure and object recognition within the image, and selects multiple target feature maps from the feature map set according to the preset selection method; the target feature maps selected in this application are not output by the same convolutional layer, so the target feature maps output by different convolutional layers have different scales; target feature maps of different scales have different image features; the feature extraction method of UFFDM is significantly different from the traditional CNN pyramid feature extraction method. The unsupervised learning method uses the middle layer of the ResNet50 architecture, that is, the feature map of the middle layer is selected to ensure that its size matches the two shallow layers, and the feature map of the middle layer performs well in capturing the correlation between local and global features. This is especially important in scenes where the background and object are very similar and small differences are difficult to distinguish, because it can more accurately represent the key feature information of overlapping edges and differences. This application selects two sizes of feature layers, namely two adjacent shallow feature maps and two deep feature maps, as shown Figure 2 As shown in the figure, the two adjacent shallow feature maps are the left 2 feature maps, whose size is 56*56 and the number of channels is 64; the third is a deep feature map, whose size is 56*56 and the number of channels is 256; the fourth is also a deep feature map, whose size is 28*28 and the number of channels is 512, which is convenient for subsequent fusion processing.

[0063] In step S103, in order to make full use of the expression correlation between the shallow layer and the deep layer, especially for the subtle defects that are difficult to detect, the multi-scale fusion of the multiple target feature maps is used to obtain the collective feature map, that is, the collective feature map is not obtained by simple splicing, and multi-scale fusion is required, that is, the correlation of the spliced feature maps is learned based on the ResNet50 model. This method not only enhances the low-level texture information, but also constructs a feature space with 128 channels, which facilitates the subsequent feature fusion module. The collective feature map consists of N feature image blocks x of size H / 4×W / 4×C p The set feature map is then expanded to obtain multiple feature image blocks, where the feature image blocks of size H / 4×W / 4 match the size of the feature map in the ResNet50 model. During the training phase, each feature image block of the normal image is associated with the spatially corresponding activation vector in the pre-trained CNN activation feature map. The activation vectors from different layers in the fusion feature set Fout are connected to obtain an embedded feature vector containing information from different semantic levels and resolutions, and the dimensions of the multiple feature image blocks are reduced to obtain the corresponding embedded feature vectors. The embedded feature vector is one-dimensional, and each feature image block x pIt can be divided into grid cells (i, j), and the feature image block is expanded into one dimension and embedded into the feature vector x i，j Correspondingly, where (i,j) is the index i,j∈[H / 4,W / 4].

[0064] In the specific implementation process of step S103, there is an embodiment in which the multi-scale fusion of the multiple target feature maps to obtain the aggregate feature map includes:

[0065] S1031. Determine a first fusion method and a second fusion method for the target feature map according to different types of the target feature map;

[0066] S1032. Fuse the target feature map according to the first fusion method and the second fusion method to obtain a set feature map.

[0067] In steps S1031-S1032, the target feature maps selected in this application are not of the same feature layer, but are two adjacent shallow feature maps and two deep feature maps, so they belong to different types. Therefore, the first fusion method and the second fusion method of the target feature maps are determined. The first fusion method is: adjacent shallow feature maps are spliced on the channel scale to obtain a first spliced map, and the two deep feature maps are fused to obtain a first fused map. The second fusion method is to align and splice the first spliced map and the first fused map again, such as Figure 3-Figure 4 As shown, the target feature map is fused according to the first fusion method and the second fusion method. The unsupervised learning UFFDM selects the shallow and deep feature maps in the third residual block and the fourth residual block of the ResNet50 model for fusion and splicing. The reason for not selecting deeper feature maps is that deep features are often more abstract and often lose the fine-grained spatial information required for accurate positioning of small defects or scratches. Fusion and splicing easily lead to positioning diffusion rather than accurate scratch positioning. Subsequently, the present application adjusts the size of the fourth residual block feature map by interpolation method to make it consistent with the size of the third residual block feature map (H / 4, W / 4), while keeping the number of channels unchanged.

[0068]

[0069] Where x is a feature image patch of size H×W×C, the batch size is N, is the lth eigenimage of x, and The lth feature image is obtained by resizing, and the size is H / 4×W / 4×C L×N. After resizing the feature images using the above method, the intermediate and deep feature maps are element-by-element stacked and concatenated. A 1×1 convolution operation (conv1×1) is applied to each layer, using the pre-trained weights of ResNet50 to process the feature channels of each layer. The layers are then concatenated and aligned to form a representation of the entire input image. The number of channels is reduced through conv1x1 operations on each layer, maintaining the same feature layer size of H / 4 × W / 4. Interpolation is used to adjust the number of channels.

[0070]

[0071] Where F iout (i=1, 2, 3) is the output of the fusion of each feature image, For the activation feature layer of the pre-trained ResNet50 network, 4 is the feature map after channel adjustment, and Cat is the feature map spliced on the channel layer. Figure 3 CONCAT and Add in are feature map superpositions, and all processed feature maps are concatenated into a collective feature map with a size of H0×W0×C0.

[0072] Fout=Cat(F 1out ,F 2out ,F 3out )# (5)

[0073] Where Fout is the output set feature map.

[0074] In step S104, after obtaining multiple embedded feature vectors based on the unsupervised learning method, a target embedded feature vector is selected from the multiple embedded feature vectors according to a target selection method. The target embedded feature vector is greatly reduced in number compared to the number of embedded feature vectors, thereby reducing the redundant information contained therein. The feature vectors carried in the selected multiple target embedded feature vectors are not consistent. Some may have multiple feature vectors, but there may also be no feature vectors. Therefore, a feature vector distribution calculation is performed on the target embedded feature vector to obtain an abnormal distribution map, and positioning is performed based on the abnormal distribution map to obtain position information of weak scratches. Based on the position information and the abnormal distribution map, a scratch detection result is generated according to a preset result generation method, so as to determine the scratch test result for the aluminum plate to be tested based on the scratch detection result, and the scratch detection result is evaluated to determine the accuracy and effectiveness of the scratch detection, and new scratch detection data is collected to continuously train and optimize the weak scratch detection model to adapt to the scratch detection needs under different environments and conditions.

[0075] In a specific implementation process of step S104, there is an embodiment in which the calculating of the target embedded feature vector selected from the plurality of embedded feature vectors includes:

[0076] S10411. Randomly generate multiple sequence numbers and bind them one-to-one with the embedded feature vectors so that the sequence numbers correspond one-to-one with the embedded feature vectors;

[0077] S10412. Randomly select a preset number of sequence numbers, and determine the corresponding embedded feature vectors as the target embedded feature vectors.

[0078] In steps S10411-S10412, as Figure 5 As shown, for multiple embedded feature vectors, the number of the embedded feature vectors is counted, and multiple sequence numbers are randomly generated, and the sequence number is consistent with the number of the embedded feature vectors, and the sequence number is bound to the embedded feature vector one-to-one, that is, the sequence number corresponds to the embedded feature vector one-to-one, and a preset number of sequence numbers is randomly selected, the preset number is 4, and multiple groups of preset numbers of embedded feature vectors can also be selected, thereby reducing the size of the embedded feature vector and the existence of redundant information, significantly reducing the complexity of the weak scratch detection model during testing, while maintaining the most advanced performance, and the random sampling method is also applied to the process of training the weak scratch detection model. The normal image learned through training can reflect the accuracy of unsupervised learning UFFDM in subsequent abnormality positioning.

[0079] In the specific implementation process of step S104, there is another embodiment in which the calculation based on the target embedded feature vector selected from the multiple embedded feature vectors and the preset normal embedded feature vector to obtain the abnormal score map includes:

[0080] S10421, calling the normal embedding feature vector generated when training the faint scratch detection model, and inputting the normal embedding feature vector and the target embedding feature vector into a preset scratch calculation network;

[0081] S10422. Process the normal embedded feature vector and the target embedded feature vector based on the scratch calculation network to obtain an anomaly score map, and determine a scratch detection result based on values in the anomaly score map.

[0082] In steps S10421-S10422, the normal embedding feature vector of the normal image of the sample aluminum plate at each position is recorded when the weak scratch detection model is trained based on multiple normal images of the sample aluminum plate. The normal embedding feature vector is used to learn the Gaussian distribution. Indicates that the normal embedding feature vector Stored in the weak scratch detection model, to be called when used, and the normal embedding feature vector The target embedding feature vector is then put into a preset scratch calculation network, which is represented by formula (6):

[0083]

[0084] The scratch calculation network is the Mahalanobis distance M between the normal embedding feature vector and the target embedding feature vector. ij The computational network model is no longer limited to fixed positions and can adapt to unaligned data sets. Based on the Mahalanobis distance M ij Generate the anomaly score map, M ij The maximum value corresponds to the largest abnormal area, that is, different abnormal scores are generated for different target embedded feature vectors. That is, based on the abnormal scores corresponding to the target embedded feature vectors, the existence of weak scratches is detected, and on this basis, the weak scratches are located again to obtain position information, thereby generating scratch detection results according to the preset generation method.

[0085] The normal embedding feature vector is obtained based on the normal image of the sample aluminum plate. It is obtained based on the following steps: First, calculate the feature vector set X of the normal image of the N sample aluminum plates in training i，j ={x i，j , the information carried by k∈[1,N]}, such as Figure 6 As shown, assuming X i，j is a multivariate Gaussian distribution Generated, where μ ij For X i，j The sample mean and sample covariance ∑ ij The estimation is shown in formula (7):

[0086]

[0087] The purpose of adding the regularization term ∈I is to make the sample covariance matrix Σij fully rank-reversible, and each estimated normal embedding eigenvector is The semantic information and correlation between different levels in the feature block vector set can be associated, which has an enhanced effect on the expression of image feature information.

[0088] In a specific implementation process of step S10422, there is an embodiment in which determining the scratch detection result based on the values in the abnormal score map includes:

[0089] S104221. Determine whether the value corresponding to the abnormal score map meets the preset weak scratch existence condition;

[0090] S104222: If yes, locate the position information of the faint scratch based on the anomaly score map, so as to generate a scratch detection result based on the position information.

[0091] In steps S104221-S104222, each anomaly score in the anomaly score graph is judged to determine whether a preset weak scratch existence condition is met, wherein the weak scratch existence condition is whether the anomaly score in the anomaly score graph is greater than a preset scratch threshold. The preset scratch threshold is specifically set according to actual conditions and is not limited here. If the anomaly score does not meet the preset weak scratch existence condition, it is determined that the weak scratch does not exist. If the anomaly score meets the preset weak scratch existence condition, it is determined that the weak scratch exists. Based on the determination of the existence of the weak scratch, the weak scratch is located to obtain the position information of the weak scratch, and a scratch detection result is generated based on the position information. The scratch detection result includes the result that the weak scratch exists and the position information of the weak scratch, so as to clearly display the existence of the weak scratch.

[0092] In the specific implementation process of step S104222, there is an embodiment in which the positioning of the position information of the faint scratch based on the anomaly score map includes:

[0093] S1042221. Setting different first preprocessing thresholds to perform corresponding processing on the anomaly score map based on the different first preprocessing thresholds;

[0094] S1042222. Match the image to be tested of the aluminum plate to be tested based on the preprocessed anomaly score map, so as to locate the position information of the faint scratch on the image to be tested based on the position information of the faint scratch on the anomaly score map.

[0095] In steps S1042221-S1042222, different first preprocessing thresholds are set based on the presence of a weak scratch, so as to perform corresponding processing on the anomaly score map based on the different first preprocessing thresholds; the specific method of the processing is binarization processing, thereby reducing the difficulty of locating the weak scratch and the amount of data calculated, and the anomaly score map after binarization processing is consistent with M ijThe resolution is low and cannot completely match the size of the image to be tested of the aluminum plate to be tested. Therefore, the bilinear interpolation technology is used to scale the anomaly score map to match the size of the image to be tested of the aluminum plate to be tested. After the size is unified, the position information of the weak scratch in the image to be tested of the aluminum plate to be tested can be obtained based on the position information represented by i and j, thereby achieving accurate positioning of the weak scratch, that is, not only the detection of the weak scratch is achieved, but also the specific position is located, realizing effective detection of weak scratches based on unsupervised learning UFFDM.

[0096] In the specific implementation process of step S104, there is another embodiment in which the evaluating the scratch detection result includes:

[0097] S10431. Evaluate the scratch detection result based on multiple dimensions to obtain evaluation results of corresponding dimensions;

[0098] S10432. Combining the evaluation results of multiple dimensions to obtain a final evaluation result, and determining a result for scratch detection based on the final evaluation result.

[0099] In steps S10431-S10432, after obtaining the scratch detection result, the scratch detection result is evaluated based on multiple dimensions to obtain the evaluation results of the corresponding dimensions. This application selects (AUROC) as the evaluation standard. Image-AUROC, Pixel-AUROC, and F1-Score are selected as the dimensions for image-level evaluation. For the classification results of anomaly detection, the advanced methods perform similarly on multiple data sets. In order to further achieve accurate scratch positioning and segmentation, this application also selects the Pixel-AUROC pixel-level AUROC between the generated defect map and the true label as a dimension of evaluation. In addition, the pixel-level F1-Score is set as a dimension of evaluation. The higher the score, the higher the accuracy of scratch positioning.

[0100] The present application also sets up a test data set for the weak scratch detection model. The test data set includes multiple normal aluminum plate sample images and abnormal aluminum plate sample images. The test data set is made by selecting aluminum plates with high reflective properties and using an artificial scratch pattern. The main defects of the abnormal aluminum plate sample images are continuous scratches and discontinuous scratches, and also involve some block defects. Figure 7The dataset shows images of normal and abnormal aluminum sheet samples, which exhibit high-frequency linear textures. It also displays images of scratch defects with varying geometric features, created using a diamond stylus. To conduct small-sample learning experiments, this paper reduced the number of normal sample images to 100. The diversity and complexity of abnormal aluminum sheet sample images significantly complicates automated detection. Table 1 presents information about the aluminum sheet dataset.

[0101] Table 1 Aluminum plate surface dataset

[0102] Table 1 Division of Aluminum Plate surface dataset

[0103]

[0104] The tests of the faint scratch detection model in this application were conducted on a 14-core AMD EPYC 7453 CPU system configuration, paired with an NVIDIA RTX 4090 GPU with 25.2GB GDDR6X video memory, 64.4GB RAM and a 451.0GB hard drive. Based on the Pytorch deep learning framework, this application selected the ResNet50 network pre-trained on the ImageNet dataset as the feature extraction network, and integrated the multi-scale feature information of each feature layer. Patch embedding vectors were extracted from feature layers 6 and 7 (the second residual block), layer 37 (the second residual block), and layer 78 (the fourth residual block). At the same time, the experiment reproduced the Padim model

[23] and Patchcore model

[25] , and resized the images in all datasets to 224×224. For the final scratch detection localization map, bilinear interpolation was used, and Gaussian filtering was applied to the detection image with a parameter of δ=2.

[0105] This application is compared with several state-of-the-art algorithms based on patch feature embedding, including Padim, Patchcore, FCDD method based on single classification, and 2D flow model Fastflow. The experimental results of the comparative algorithms are obtained by debugging the original article code and third-party code. The scratch detection results are as follows: Figure 8 As shown, the scratch positioning is as Figure 9 As shown. Figure 8The comparative detection results for aluminum plate scratches demonstrate that the Padim and Patchcore models excel at image-level scratch detection by leveraging the high correlation of semantic information between feature layers. The FCDD model, on the other hand, accurately locates and detects prominent vertical and cross scratches, but performs poorly for horizontal scratches against similar backgrounds. The Fastflow model is susceptible to noise and often misinterprets background as anomalies. In contrast, the UFFDM algorithm proposed in this paper provides more reliable and stable results for scratch detection in real-world scenarios.

[0106] This application also compares the UFFDM method with the method in the Anomalib library on aluminum plate scratch data. The experimental results (using pixel-level Pixel-AUROC, image-level Image-AUROC and pixel F1 score as evaluation indicators) are shown in Table 2. From the analysis of Table 2, UFFDM outperforms other methods in fine-grained anomaly localization. Although the improvement in image AUROC and pixel AUROC compared with Fastflow is not significant, Figure 9 The actual positioning results show a significant difference, with a low false detection rate, especially for horizontal scratches that resemble the background. Subjective experimental results from other methods often miss weaker scratches and those that resemble the background. This algorithm enhances the deep feature representation of edges and lines in weak scratches, enabling better detection of discontinuous scratches. It not only effectively distinguishes scratches that resemble the background, but also more accurately locates hidden scratch anomalies.

[0107] In summary, the UFFDM of this application has advantages in positioning and detection performance. This application has improvements in average pixel AUROC and efficiency, and the pixel F1-Score indicator of UFFDM in surface anomaly detection / localization tasks is outstanding.

[0108] Table 2 Experimental comparison of scratch location methods on aluminum plate surface

[0109]

[0110] This application also conducted multiple ablation experiments to evaluate the performance of the UFFDM used in the faint scratch detection model. These experiments validated the effectiveness of small-sample training and explored the correlation between shallow and deep features. Furthermore, even with limited training data, optimal feature representation of subtle defects was achieved.

[0111] This application conducted comparative tests on different proportions of training data (from 5% to 100%) while keeping the test data unchanged to verify the feasibility of small sample learning. As shown in Table 3, even with a very small amount of training data, the embedded feature vector formed can still well characterize the image features, and the feature extraction process becomes more simplified. The ablation experiment results show that UFFDM exhibits strong robustness in real scene data. Future research will further explore the significant impact of scene changes on scratch detection, and strive to optimize learning strategies to meet cross-domain challenges and solve more complex defect localization problems.

[0112] Table 3 Comparison of small sample training experimental results

[0113] Table 3 Comparison of small sample training experimental results

[0114] Table3 Comparison of Few-shot Training Experiment Results

[0115]

[0116] Example 2

[0117] This application also provides a weak scratch detection device, such as Figure 10 The following is a block diagram of an electronic psychological sandbox intelligent display device. The functions implemented by this device correspond to the steps of executing a weak scratch detection method on a terminal device. This device can be understood as a component of a server including a processor. The device described in this application includes:

[0118] An acquisition module 1001 is configured to acquire an image of an aluminum plate to be tested, and input the image into a weak scratch detection model pre-trained based on multiple normal images of sample aluminum plates; the weak scratch detection model is established based on a deep convolutional neural network;

[0119] The extraction module 1002 is configured to extract image features from the image to be tested to obtain a feature map set, and select multiple target feature maps from the feature map set according to a preset selection method; different target feature maps have different scales; and target feature maps of different scales have different image features;

[0120] An expansion module 1003 is configured to fuse the multiple target feature maps at multiple scales to obtain a collective feature map, expand the collective feature map to obtain multiple feature image blocks, and reduce the dimensions of the multiple feature image blocks to obtain corresponding embedded feature vectors;

[0121] The calculation module 1004 is used to perform calculations based on the target embedded feature vector selected from the multiple embedded feature vectors and the normal embedded feature vector to obtain an abnormal score map, obtain a scratch detection result based on the abnormal score map, and evaluate the scratch detection result to complete the weak scratch detection of the aluminum plate to be tested.

[0122] In a feasible implementation, the calculation module includes:

[0123] A calling module, configured to call the learned Gaussian distribution generated when training the faint scratch detection model, and input the learned Gaussian distribution and the target embedded feature vector into a preset scratch calculation network;

[0124] A processing module is configured to process the learned Gaussian distribution and the target embedded feature vector based on the scratch calculation network to obtain an anomaly score map, and determine a scratch detection result based on a value in the anomaly score map.

[0125] In a feasible implementation manner, the calculation module further includes:

[0126] A judgment module, configured to judge whether the value corresponding to the abnormal score map meets a preset condition for the existence of a weak scratch;

[0127] A generating module is configured to, if yes, locate position information of the faint scratch based on the anomaly score map, so as to generate a scratch detection result based on the position information.

[0128] In a feasible implementation, the calculation module also includes:

[0129] a setting module, configured to set different first preprocessing thresholds, so as to perform corresponding processing on the anomaly score map based on the different first preprocessing thresholds;

[0130] A positioning module is used to match the image to be tested of the aluminum plate to be tested based on the preprocessed anomaly score map, so as to locate the position information of the faint scratch on the image to be tested based on the position information of the faint scratch on the anomaly score map.

[0131] In a feasible implementation, the calculation module further includes:

[0132] An evaluation module, configured to evaluate the scratch detection result based on multiple dimensions to obtain evaluation results of corresponding dimensions;

[0133] The combining module is used to combine the evaluation results of multiple dimensions to obtain a final evaluation result, so as to determine the result of scratch detection based on the final evaluation result.

[0134] In a feasible implementation, the expansion module includes:

[0135] A first fusion module is used to determine a first fusion mode and a second fusion mode for the target feature map according to different types of the target feature map;

[0136] The second fusion module is used to fuse the target feature map according to the first fusion method and the second fusion method to obtain a set feature map.

[0137] In a feasible implementation, the calculation module further includes:

[0138] Randomly generate a plurality of sequence numbers and bind them one-to-one with the embedded feature vectors so that the sequence numbers correspond one-to-one with the embedded feature vectors;

[0139] A preset number of sequence numbers are randomly selected, and the corresponding embedded feature vectors are determined as the target embedded feature vectors.

[0140] Example 3

[0141] The present application also provides an electronic device, such as Figure 11 As shown, it includes: a processor 1101, a memory 1102 and a bus 1103, the memory 1102 stores machine-readable instructions executable by the processor 1101, and when the electronic device is running, the processor 1101 communicates with the memory 1102 through the bus 1103, and when the machine-readable instructions are executed by the processor 1101, any step of the faint scratch detection method is performed.

[0142] Example 4

[0143] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods for detecting weak scratches are executed.

[0144] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0145] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0146] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0147] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, platform server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0148] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for detecting weak scratches, characterized in that: The method comprises: Obtaining a test image of the aluminum plate to be tested, and inputting the test image into a weak scratch detection model pre-trained based on multiple normal images of sample aluminum plates; the weak scratch detection model is established based on a deep convolutional neural network; Extracting image features from the image to be tested to obtain a feature map set, and selecting multiple target feature maps from the feature map set according to a preset selection method; different target feature maps have different scales; target feature maps of different scales have different image features; Multi-scale fusing the multiple target feature maps to obtain a set feature map, expanding the set feature map to obtain multiple feature image blocks, and reducing the dimensions of the multiple feature image blocks to obtain corresponding embedded feature vectors; Calculation is performed based on a target embedded feature vector selected from a plurality of embedded feature vectors and a normal embedded feature vector to obtain an anomaly score map, a scratch detection result is obtained based on the anomaly score map, and the scratch detection result is evaluated to complete the weak scratch detection of the aluminum plate to be tested.

2. The method according to claim 1, characterized in that The calculation based on the target embedded feature vector selected from the multiple embedded feature vectors and the preset normal embedded feature vector to obtain the abnormal score map includes: calling a normal embedding feature vector generated when training the faint scratch detection model, and inputting the normal embedding feature vector and the target embedding feature vector into a preset scratch calculation network; The normal embedded feature vector and the target embedded feature vector are processed based on the scratch calculation network to obtain an anomaly score map, and a scratch detection result is determined based on a value in the anomaly score map.

3. The method according to claim 2, characterized in that The determining of the scratch detection result based on the values in the abnormal score map includes: Determining whether the value corresponding to the abnormal score map meets the preset weak scratch existence condition; If so, position information of the faint scratch is located based on the anomaly score map to generate a scratch detection result based on the position information.

4. The method according to claim 3, characterized in that The locating the position information of the faint scratch based on the abnormal score map includes: Setting different first preprocessing thresholds to perform corresponding processing on the anomaly score map based on the different first preprocessing thresholds; Matching is performed based on the preprocessed anomaly score map with the image to be tested of the aluminum plate to locate the position information of the weak scratch on the image to be tested based on the position information of the weak scratch on the anomaly score map.

5. The method according to claim 1, wherein The evaluating the scratch detection result includes: Evaluate the scratch detection result based on multiple dimensions to obtain evaluation results of corresponding dimensions; The evaluation results of the multiple dimensions are combined to obtain a final evaluation result, so as to determine the result of the scratch detection based on the final evaluation result.

6. The method according to claim 1, characterized in that The multi-scale fusing of the multiple target feature maps to obtain a set feature map includes: Determining a first fusion method and a second fusion method for the target feature map according to different types of the target feature map; The target feature map is fused according to the first fusion method and the second fusion method to obtain a set feature map.

7. The method according to claim 1, characterized in that The calculating of a target embedded feature vector selected from a plurality of embedded feature vectors includes: Randomly generate a plurality of sequence numbers and bind them one-to-one with the embedded feature vectors so that the sequence numbers correspond one-to-one with the embedded feature vectors; A preset number of sequence numbers are randomly selected, and the corresponding embedded feature vectors are determined as the target embedded feature vectors.

8. A weak scratch detection device, characterized in that: The device comprises: An acquisition module is used to acquire a test image of the aluminum plate to be tested, and input the test image into a weak scratch detection model pre-trained based on multiple normal images of sample aluminum plates; the weak scratch detection model is established based on a deep convolutional neural network; An extraction module is used to extract image features from the image to be tested to obtain a feature map set, and select multiple target feature maps from the feature map set according to a preset selection method; different target feature maps have different scales; target feature maps of different scales have different image features; an expansion module for fusing the multiple target feature maps at multiple scales to obtain a set feature map, expanding the set feature map to obtain multiple feature image blocks, and reducing the dimensions of the multiple feature image blocks to obtain corresponding embedded feature vectors; A calculation module is used to perform calculations based on a target embedded feature vector selected from a plurality of embedded feature vectors and a normal embedded feature vector to obtain an anomaly score map, obtain a scratch detection result based on the anomaly score map, and evaluate the scratch detection result to complete the weak scratch detection of the aluminum plate to be tested.

9. An electronic device, characterized in that: include: A processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate via the bus. When the machine-readable instructions are executed by the processor, the steps of a faint scratch detection method as described in any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of a weak scratch detection method according to any one of claims 1 to 7.

Images