Object surface defect detection method and device, electronic equipment, medium and product

Abstract

The application relates to the technical field of defect detection, in particular to an object surface defect detection method and device, electronic equipment, medium and product, wherein the method comprises the following steps: obtaining an object surface image to be detected; inputting the object surface image to be detected into a defect detection model; and outputting a defect feature map of the object surface to be detected by the defect detection model, wherein the defect detection model comprises a feature pyramid network, an attention mechanism network and a feature decoder; the feature pyramid network extracts a target resolution feature map of the object surface image to be detected, and the attention mechanism network performs target feature processing on the target resolution feature map; and the feature decoder decodes the processed target resolution feature map to obtain the defect feature map; and defect data of the object surface to be detected is obtained by identifying the defect feature map. Thus, the problems of low efficiency and inconsistent evaluation standards in the related art for detecting object defects by using artificial detection are solved.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and in particular to a method, apparatus, electronic device, medium and product for detecting defects on the surface of an object. Background Technology

[0002] With the development of industrial production technology, modern manufacturing has increasingly stringent requirements for the quality of produced workpieces. Poor quality control not only affects product performance, leading to unusable products, but may even create safety hazards and cause accidents. In today's automated automotive manufacturing industry, welding defects, installation defects, and incorrect or missing parts are common occurrences.

[0003] Manual visual inspection is widely used in surface defect detection. Inspectors rely on their rich experience to accurately judge whether a workpiece is normal or abnormal. However, having manual inspectors inspect every product on the production line has the following problems: 1) Different manual inspectors have different inspection standards for workpieces, making it difficult to maintain consistent inspection results and stable inspection accuracy; 2) Skilled inspectors require high labor costs; 3) Manual inspection is labor-intensive, and inspectors find it difficult to maintain a high level of concentration for extended periods; 4) Traditional manual inspection methods are no longer suitable for large-scale manufacturing scenarios, resulting in low efficiency and potential delays in production line progress. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, medium, and product for detecting defects on the surface of objects, in order to solve the problems of low efficiency and inconsistent evaluation standards in the related art when using manual methods to detect defects in objects.

[0005] The first aspect of this application provides a method for detecting surface defects of an object, comprising the following steps: acquiring an image of the surface of an object to be detected; inputting the image of the surface of the object to be detected into a defect detection model, wherein the defect detection model outputs a defect feature map of the surface of the object to be detected, wherein the defect detection model includes a feature pyramid network, an attention mechanism network, and a feature decoder; the feature pyramid network extracts a target resolution feature map of the image of the surface of the object to be detected, and the attention mechanism network performs target feature processing on the target resolution feature map; the feature decoder decodes the processed target resolution feature map to obtain a defect feature map; and the defect feature map is identified to obtain defect data of the surface of the object to be detected.

[0006] Optionally, the attention mechanism network includes: a first residual network module, a global average pooling module, a global max pooling module, a feature map stacking and convolution module, a batch normalization module, and a second residual network module. The first residual network module directly inputs some features from the target resolution feature map into the second residual network module and extracts high-dimensional features from the target resolution feature map. The global average pooling module performs average pooling on the feature map output by the first residual network to output a first feature map. The global max pooling module performs max pooling on the feature map output by the first residual network to output a second feature map. The feature map stacking and convolution module fuses the features from the first and second feature maps to obtain a third feature map and reduces the dimensionality of the third feature map's channels through shared convolution. The batch normalization module accelerates the convergence speed of the attention mechanism network. The second residual network module performs unified weight processing on some features input from the first residual network module and features from the third feature map, readjusts the corresponding feature vectors, and outputs the processed target resolution feature map.

[0007] Optionally, the attention mechanism network also includes an activation function module, which is used to perform nonlinear mapping calculation on the third feature map to obtain a fourth feature map that is the same as the target resolution feature map.

[0008] Optionally, the attention mechanism network also includes an activation function module, which is used to perform nonlinear mapping calculations on the third feature map and change the resolution of the third feature map to be the same as the resolution of the target resolution feature map.

[0009] Optionally, the global average pooling module and the global max pooling module perform average pooling and max pooling operations on the coordinates of the feature values ​​in the feature map output by the first residual network, respectively.

[0010] Optionally, the feature pyramid network is divided into four layers from bottom to top. The first layer performs convolution operation on the surface image of the object to be detected to obtain a first resolution feature map; the second layer performs convolution operation on the first resolution feature map to obtain a second resolution feature map; the third layer performs convolution operation on the second resolution feature map to obtain a third resolution feature map; and the fourth layer performs convolution operation on the third resolution feature map to obtain a target resolution feature map.

[0011] Optionally, the feature decoder is divided into five to seven layers from top to bottom. The fifth layer decodes the processed target resolution feature map and fuses the features in the target resolution feature map to obtain the first decoded feature map. The sixth layer decodes the first decoded feature map and fuses the features in the third resolution feature map to obtain the second decoded feature map. The seventh layer decodes the second decoded feature map and fuses the features in the second resolution feature map to obtain the defect feature map.

[0012] A second aspect of this application provides a surface defect detection device, comprising: an acquisition module for acquiring an image of the surface of an object to be detected; an input module for inputting the image of the surface of the object to be detected into a defect detection model, wherein the defect detection model outputs a defect feature map of the surface of the object to be detected, the defect detection model including a feature pyramid network, an attention mechanism network, and a feature decoder, wherein the feature pyramid network extracts a target resolution feature map of the image of the surface of the object to be detected, and the attention mechanism network performs target feature processing on the target resolution feature map, and the feature decoder decodes the processed target resolution feature map to obtain a defect feature map; and an identification module for identifying the defect feature map to obtain defect data of the surface of the object to be detected.

[0013] Optionally, the attention mechanism network includes: a first residual network module, a global average pooling module, a global max pooling module, a feature map stacking and convolution module, a batch normalization module, and a second residual network module. The first residual network module directly inputs some features from the target resolution feature map into the second residual network module and extracts high-dimensional features from the target resolution feature map. The global average pooling module performs average pooling on the feature map output by the first residual network to output a first feature map. The global max pooling module performs max pooling on the feature map output by the first residual network to output a second feature map. The feature map stacking and convolution module fuses the features from the first and second feature maps to obtain a third feature map and reduces the dimensionality of the third feature map's channels through shared convolution. The batch normalization module accelerates the convergence speed of the attention mechanism network. The second residual network module performs unified weight processing on some features input from the first residual network module and features from the third feature map, readjusts the corresponding feature vectors, and outputs the processed target resolution feature map.

[0014] Optionally, the attention mechanism network also includes an activation function module, which is used to perform nonlinear mapping calculations on the third feature map and change the resolution of the third feature map to be the same as the resolution of the target resolution feature map.

[0015] Optionally, the global average pooling module and the global max pooling module perform average pooling and max pooling operations on the coordinates of the feature values ​​in the feature map output by the first residual network, respectively.

[0016] Optionally, the feature pyramid network is divided into four layers from bottom to top. The first layer performs convolution operation on the surface image of the object to be detected to obtain a first resolution feature map; the second layer performs convolution operation on the first resolution feature map to obtain a second resolution feature map; the third layer performs convolution operation on the second resolution feature map to obtain a third resolution feature map; and the fourth layer performs convolution operation on the third resolution feature map to obtain a target resolution feature map.

[0017] Optionally, the feature decoder is divided into five to seven layers from top to bottom. The fifth layer decodes the processed target resolution feature map and fuses the features in the target resolution feature map to obtain the first decoded feature map. The sixth layer decodes the first decoded feature map and fuses the features in the third resolution feature map to obtain the second decoded feature map. The seventh layer decodes the second decoded feature map and fuses the features in the second resolution feature map to obtain the defect feature map.

[0018] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the object surface defect detection method as described above.

[0019] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to perform the object surface defect detection method as described above.

[0020] The fifth aspect of this application provides a computer program product having a computer program or instructions stored thereon, which, when executed, implements the object surface defect detection method as described in the above embodiments.

[0021] Therefore, this application has at least the following beneficial effects:

[0022] This application embodiment can input an image of the surface of an object to be detected into a defect detection model. The defect detection model outputs a defect feature map, which is then further identified to obtain surface defect data, thus achieving automatic detection of surface defects. This improves the efficiency and accuracy of defect detection. Furthermore, the defect detection model combines an attention mechanism network with a feature pyramid network, effectively enhancing the network's ability to extract and detect defect features. Therefore, it solves the technical problems of low efficiency and inconsistent evaluation standards associated with manual defect detection in related technologies.

[0023] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0024] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0025] Figure 1 This is a flowchart of a method for detecting surface defects of an object according to an embodiment of this application;

[0026] Figure 2 This is a structural diagram of the attention mechanism network provided according to embodiments of this application;

[0027] Figure 3 This is a schematic diagram of a defect detection model provided according to a specific embodiment of this application;

[0028] Figure 4 This is a schematic diagram of an object surface defect detection device provided according to an embodiment of this application;

[0029] Figure 5 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0030] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0031] The following description, with reference to the accompanying drawings, outlines a method, apparatus, electronic device, medium, and product for detecting surface defects in objects according to embodiments of this application. Addressing the issues mentioned in the background section regarding the limitations of traditional manual inspection methods in large-scale manufacturing scenarios—namely, low efficiency, inconsistent evaluation standards, and potential delays in production line progress—this application provides a method for detecting surface defects in objects. In this method, an image of the object's surface to be inspected is input into a defect detection model, which outputs a defect feature map. Further identification of the defect feature map yields surface defect data, enabling automatic detection of surface defects. This solves the problems of low efficiency and inconsistent evaluation standards associated with manual inspection of object defects in related technologies.

[0032] Specifically, Figure 1 This is a schematic flowchart of a method for detecting surface defects of an object provided in an embodiment of this application.

[0033] like Figure 1As shown, the method for detecting surface defects of an object includes the following steps:

[0034] In step S101, an image of the surface of the object to be detected is acquired.

[0035] It is understood that the embodiments of this application can acquire the surface image of the object to be detected, for example, by acquiring it through an image acquisition device.

[0036] In step S102, the surface image of the object to be detected is input into the defect detection model, and the defect detection model outputs the defect feature map of the surface of the object to be detected. The defect detection model includes a feature pyramid network, an attention mechanism network, and a feature decoder. The feature pyramid network extracts the target resolution feature map of the surface image of the object to be detected, and the attention mechanism network performs target feature processing on the target resolution feature map. The feature decoder decodes the processed target resolution feature map to obtain the defect feature map.

[0037] The defect detection model consists of three parts: a feature pyramid network, an attention mechanism network, and a feature decoder. The feature pyramid network extracts features from the surface of the object to be detected to generate a target resolution feature map. The target resolution can be set according to specific circumstances, such as 40*40. The attention mechanism network performs target feature processing on the target resolution feature map. The feature decoder decodes the processed target resolution feature map to obtain the defect feature map.

[0038] It is understood that in the embodiments of this application, the surface image of the object to be detected is input into the defect detection model, and the defect detection model outputs the defect feature map of the surface of the object to be detected. The defect detection model integrates and feeds into a feature attention mechanism network. After the feature pyramid network is integrated, the feature extraction and detection capabilities of the network can be effectively improved.

[0039] In this embodiment, the attention mechanism network includes: a first residual network module, a global average pooling module, a global max pooling module, a feature map stacking and convolution module, a batch normalization module, and a second residual network module. The first residual network module directly inputs a portion of the features from the target resolution feature map into the second residual network module and extracts high-dimensional features from the target resolution feature map. The global average pooling module performs average pooling on the feature map output by the first residual network to output a first feature map. The global max pooling module performs max pooling on the feature map output by the first residual network to output a second feature map. The feature map stacking and convolution module fuses the features from the first and second feature maps to obtain a third feature map and reduces the dimensionality of the third feature map's channels through shared convolution. The batch normalization module accelerates the convergence speed of the attention mechanism network. The second residual network module performs unified weighting on the features input from the first residual network module and the features from the third feature map, readjusts the corresponding feature vectors, and outputs the processed target resolution feature map.

[0040] It is understood that the attention mechanism network in this application embodiment includes: a first residual network module, a global average pooling module, a global max pooling module, a feature map stacking and convolution module, a batch normalization module, and a second residual network module, wherein,

[0041] 1. The first residual network module is used to directly input some features from the target resolution feature map into the second residual network module and extract the high-dimensional features of the target resolution feature map.

[0042] This module is the residual module of the residual neural network, and its main function is to perform residual calculations. Residual networks are a type of convolutional neural network, and residual calculation is a unique computational method of residual networks, unlike traditional convolution calculations. The residual module can first output a portion of the input feature vector directly to the end of the module without performing residual calculations itself. This is done to prevent overfitting due to excessive depth in the neural network. Inputting the graph into this module is for performing relevant convolution calculations to obtain high-dimensional features of the graph's feature vectors; this is the most basic operation of all convolutional neural networks.

[0043] 2. The global average pooling module is used to perform average pooling on the feature map output by the first residual network to output the first feature map.

[0044] 3. The global max pooling module is used to perform max pooling on the feature map output by the first residual network to output the second feature map.

[0045] Global average pooling and global max pooling are similar in that they obtain the attention calculation results for the corresponding feature maps through average and max calculations. These two calculation methods result in attention to feature vectors of different dimensions of the feature map, preserving the key pixel features in the image. Their function is to preserve key feature vectors by using different calculation methods.

[0046] 4. Feature map overlay and convolution module, which is used to fuse the features in the first feature map and the second feature map to obtain the third feature map, and to reduce the dimensionality of the channel dimension of the third feature map by means of shared convolution.

[0047] 5. Batch normalization module, used to accelerate the convergence speed of attention mechanism networks.

[0048] The main function of this module is to accelerate the convergence speed of the network, prevent the gradient vanishing problem, and prevent overfitting. It is a basic module in neural networks.

[0049] 6. The second residual network module is used to perform unified weight processing on some features input from the first residual network module and features in the third feature map, readjust the corresponding feature vectors, and output the processed target resolution feature map.

[0050] This module receives part of the output feature vector from the first residual network module, and also receives the final feature output from two different attention mechanisms. It performs unified weight processing on the module, readjusts the corresponding feature vectors, and finally outputs the corresponding feature vectors.

[0051] In this embodiment, the attention mechanism network further includes an activation function module, which is used to perform nonlinear mapping calculation on the third feature map and change the resolution of the third feature map to be the same as the resolution of the target resolution feature map.

[0052] It is understood that the embodiments of this application also include an activation function module for the attention mechanism network, which is used to perform nonlinear mapping calculation on the third feature map and change the resolution of the third feature map to be the same as the resolution of the target resolution feature map.

[0053] In this embodiment, the global average pooling module and the global max pooling module perform average pooling and max pooling operations on the pixels of the feature values ​​in the feature map output by the first residual network, respectively.

[0054] It is understood that the global average pooling module and the global max pooling module in this application embodiment calculate the coordinates (including X and Y coordinates) of the feature values ​​in the feature map.

[0055] Specifically, the structure of the Parallel Attention Module (PAM) in this application embodiment is as follows: Figure 2 As shown, the input to this network is an H*W*C feature map, where H represents the length of the feature map, W represents the width of the feature map, and C represents the number of channels in the feature map. The Residual (first residual network module) is the residual network module. Its function is to directly input a portion of the feature data into the final Re-weight module (second residual network module) without further feature computation. The main purpose of this module is to prevent overfitting and improve the network's feature extraction capability. Next, the feature map H*W*C undergoes two global average pooling modules (Avg Pool() and Max Pool) and a global max pooling module to calculate the pixel (feature value) values ​​in the feature map. X and Y in these modules represent the coordinates of the feature values. After global pooling, the output feature maps are 1*W*C and H*1*C, respectively. These two feature maps are then input into the Concat+Conv2d module (feature map stacking and convolution module) for calculation. The output feature map size is 1*(W+H)*C / r. The main function of Concat (feature map stacking) is to fuse the two feature maps, while Conv2d (convolution calculation) reduces the channel dimension of the feature map by sharing convolutions. After this non-linear calculation, the non-linear expressive power of the feature map is enhanced. Next, to increase the dimensionality of the feature map, we use a 1*1 convolution to restore it to its original dimension. Then, we use the HardSigmoid activation function module to perform non-linear mapping calculations on the feature map. Finally, the image size becomes H*W*C.

[0056] The Residual module (first residual network module) is a type of convolutional neural network. Its main function is to perform residual calculations. Residual networks are unique to residual networks, and their calculation method differs from traditional convolution. The residual module can directly output a portion of the input feature vectors to the end of the module without performing residual calculations itself. This is done to prevent overfitting due to excessive depth. Inputting a graph into this module performs convolution calculations to obtain high-dimensional features from the graph's feature vectors—a fundamental operation in all convolutional neural networks.

[0057] Global average pooling and global max pooling modules: Average pooling and max pooling are similar in that they obtain the attention calculation results for the corresponding feature maps through average and max calculations. These two calculation methods obtain attention for feature vectors of different dimensions of the feature map, preserving the key pixel features in the image. They are two calculation methods of the attention mechanism, preserving the key feature vectors. The difference lies in the different calculation methods.

[0058] conv2d (convolution operation): has the same function, performing convolution calculations.

[0059] BatchNorm + Non-linear (Batch Normalization and Non-linear Mapping): BatchNorm: The main function of this module is to accelerate the convergence speed of the network; prevent the gradient vanishing problem; and prevent overfitting. It is a fundamental module in neural networks. Non-linear: This refers to non-linear regression. Introducing non-linear regression calculations is also mainly to prevent overfitting.

[0060] Re-weight module (second residual network module): This module receives part of the output feature vector from the Residual module, and also receives the final feature output from two different attention mechanisms. It performs unified weight processing, readjusts the corresponding feature vectors, and finally outputs the corresponding feature vectors.

[0061] In this embodiment, the feature pyramid network is divided into four layers from bottom to top. The first layer performs convolution operation on the surface image of the object to be detected to obtain a first resolution feature map; the second layer performs convolution operation on the first resolution feature map to obtain a second resolution feature map; the third layer performs convolution operation on the second resolution feature map to obtain a third resolution feature map; and the fourth layer performs convolution operation on the third resolution feature map to obtain a target resolution feature map.

[0062] It is understood that the feature pyramid network in this application embodiment is divided into four layers from bottom to top, namely the first layer, the second layer, the third layer and the fourth layer. Starting from the first layer at the bottom, convolution operation is performed on the surface image of the object to be detected to obtain the first resolution feature map. Starting from the second layer, convolution operation is performed on the feature maps obtained from the previous layer in sequence until the target resolution feature map is finally output. Among them, the first layer outputs the first resolution feature map, the second layer outputs the second resolution feature map, the third layer outputs the third resolution feature map, and the fourth layer outputs the fourth resolution feature map.

[0063] For example, the surface image of the object to be detected is input into the bottom layer structure. After some convolution operations, the size of the feature map is adjusted to 320*320. Then, the 320*320 feature map is input into the second layer. After another convolution operation, the size of the feature map becomes half of the input feature map, becoming 160*160. Then, the 160*160 feature map is input into the third layer. After corresponding calculations, the output feature map size becomes 80*80. This is then input into the fourth layer feature map for convolution calculation. The output feature map size of the fourth layer becomes 40*40. Finally, the 40*40 feature map is input into our proposed attention mechanism network. In this network, the main function is to perform weighted operations on important features in the feature map, which allows the features of small objects to be represented in a weighted manner.

[0064] In this embodiment of the application, the feature decoder is divided into five to seven layers from top to bottom. The fifth layer decodes the processed target resolution feature map and fuses the features in the target resolution feature map to obtain a first decoded feature map. The sixth layer decodes the first decoded feature map and fuses the features in the third resolution feature map to obtain a second decoded feature map. The seventh layer decodes the second decoded feature map and fuses the features in the second resolution feature map to obtain a defect feature map.

[0065] It is understandable that the feature decoder is divided into three layers, from top to bottom: the fifth, sixth and seventh layers. The fifth layer decodes the processed target resolution feature map and fuses the features in the target resolution feature map to obtain the first decoded feature map; the sixth layer decodes the first decoded feature map and fuses the features in the third resolution feature map to obtain the second decoded feature map; the seventh layer decodes the second decoded feature map and fuses the features in the second resolution feature map to obtain the defect feature map.

[0066] It should be noted that, in order to prevent feature loss during encoding and decoding, a portion of the features from the feature pyramid network is directly passed through to the decoder.

[0067] For example, the feature decoder transforms a 40*40 input feature map into an 80*80 output feature map. After the depooling operation (Opposite Pooling Operate, OPO) of the decoding layer, the final output feature map becomes 160*160.

[0068] In step S103, defect feature maps are identified to obtain defect data of the surface of the object to be inspected.

[0069] It is understood that the embodiments of this application can directly identify the defect feature map output by the defect detection model to obtain the defect data of the surface of the object to be tested, and automatically detect the object to be tested so that the defective items are not out of the market and the production progress of other production lines is not delayed.

[0070] In summary, the object surface defect recognition method of this application combines the attention mechanism module with a multi-scale feature pyramid network to enhance the network's ability to perceive contextual information, expand the range of network feature utilization, and enable the network to have stronger nonlinear feature expression capabilities; effectively improve the network's feature extraction and detection capabilities; and when used in conjunction with a multi-scale feature pyramid network, it can solve the problem of shallow feature loss in multi-scale networks, enhance the network's ability to perceive remote information and learn local features; it can effectively improve the detection of surface defects based on visual images in automobile manufacturing processes, improve the detection capability of small defects, and ensure zero outflow of defective products.

[0071] The following will illustrate the specific structure and application scenario of the defect detection model of this application through a specific embodiment, such as... Figure 3 As shown, the PAM module is integrated into the multi-scale feature pyramid network. Figure 3 The left half of the network route, from bottom to top, is the feature extraction stage of the network. As the downsampling calculation is performed, the size of each layer becomes half the size of the input feature map. After outputting a 40*40 feature image at the top of the feature pyramid, it is passed through the PAM module and then output to another feature pyramid for feature decoding.

[0072] The entire industrial vision scene detection network architecture is as follows: Figure 3As shown, the network can also perform image segmentation tasks. The entire architecture is divided into two parts: the left-hand image encoder (i.e., the feature pyramid network of this application embodiment) part and the left-hand image decoder (Decoder) part. For the Encoder section on the left, the input image is first preprocessed. The preprocessing operations mainly include cropping and interpolation. The preprocessed image is then input into the bottom layer of the Encoder module. After some convolution operations, the feature map size is adjusted to 320*320. This 320*320 feature map is then input into the second layer of the Encoder. After another convolution operation, the feature map size becomes half of the input feature map, 160*160. This 160*160 feature map is then input into the third layer of the Encoder. After corresponding calculations, the output feature map size becomes 80*80, which is then input into the fourth layer feature map for convolution. The output feature map size of the fourth layer becomes 40*40. This 40*40 feature map is then input into our proposed PAM (Pay Attention) module. This module mainly implements weighted operations on important features in the feature map, allowing for weighted representation of features of small targets.

[0073] The feature decoder section on the right is mainly divided into three decoding layers. First, the image processed by the PAM module is used as the input feature map and fed into the sixth layer of the decoder. Simultaneously, to prevent feature loss during encoding and decoding, a portion of the features from the encoder is directly passed through to the decoder. Figure 3 The black arrows between the encoding and decoding modules indicate that this operation is called the Concat Operate (CO). Afterward, the feature decoder transforms the 40*40 input feature map into an 80*80 output feature map. Then, after the Opposite Pooling Operate (OPO) operation in the third decoding layer, the final output feature map becomes 160*160.

[0074] According to the object surface defect recognition method proposed in the embodiments of this application, the surface image of the object to be detected can be input into the defect detection model, the defect detection model outputs a defect feature map, and the defect feature map is further identified to obtain the object surface defect data, thereby realizing the automatic detection of object surface defects, improving the defect detection efficiency and accuracy. Furthermore, the defect detection model combines the attention mechanism network and the feature pyramid network to effectively improve the network's ability to extract and detect defect features.

[0075] Next, the surface defect identification device proposed according to the embodiments of this application is described with reference to the accompanying drawings.

[0076] Figure 4 This is a block diagram of an object surface defect identification device according to an embodiment of this application.

[0077] like Figure 4 As shown, the object surface defect identification device 10 includes: an acquisition module 100, an input module 200, and an identification module 300.

[0078] The acquisition module 100 is used to acquire an image of the surface of the object to be detected; the input module 200 is used to input the image of the surface of the object to be detected into the defect detection model, and the defect detection model outputs a defect feature map of the surface of the object to be detected. The defect detection model includes a feature pyramid network, an attention mechanism network, and a feature decoder. The feature pyramid network extracts a target resolution feature map of the image of the surface of the object to be detected, and the attention mechanism network performs target feature processing on the target resolution feature map. The feature decoder decodes the processed target resolution feature map to obtain a defect feature map; the recognition module 300 is used to recognize the defect feature map to obtain defect data of the surface of the object to be detected.

[0079] In this embodiment, the attention mechanism network includes: a first residual network module, a global average pooling module, a global max pooling module, a feature map stacking and convolution module, a batch normalization module, and a second residual network module. The first residual network module directly inputs a portion of the features from the target resolution feature map into the second residual network module and extracts high-dimensional features from the target resolution feature map. The global average pooling module performs average pooling on the feature map output by the first residual network to output a first feature map. The global max pooling module performs max pooling on the feature map output by the first residual network to output a second feature map. The feature map stacking and convolution module fuses the features from the first and second feature maps to obtain a third feature map and reduces the dimensionality of the third feature map's channels through shared convolution. The batch normalization module accelerates the convergence speed of the attention mechanism network. The second residual network module performs unified weighting on the features input from the first residual network module and the features from the third feature map, readjusts the corresponding feature vectors, and outputs the processed target resolution feature map.

[0080] In this embodiment, the attention mechanism network further includes an activation function module, which is used to perform nonlinear mapping calculation on the third feature map and change the resolution of the third feature map to be the same as the resolution of the target resolution feature map.

[0081] In this embodiment, the global average pooling module and the global max pooling module perform average pooling and max pooling operations on the pixels of the feature values ​​in the feature map output by the first residual network, respectively.

[0082] In this embodiment, the feature pyramid network is divided into four layers from bottom to top. The first layer performs convolution operation on the surface image of the object to be detected to obtain a first resolution feature map; the second layer performs convolution operation on the first resolution feature map to obtain a second resolution feature map; the third layer performs convolution operation on the second resolution feature map to obtain a third resolution feature map; and the fourth layer performs convolution operation on the third resolution feature map to obtain a target resolution feature map.

[0083] In this embodiment of the application, the feature decoder is divided into five to seven layers from top to bottom. The fifth layer decodes the processed target resolution feature map and fuses the features in the target resolution feature map to obtain a first decoded feature map. The sixth layer decodes the first decoded feature map and fuses the features in the third resolution feature map to obtain a second decoded feature map. The seventh layer decodes the second decoded feature map and fuses the features in the second resolution feature map to obtain a defect feature map.

[0084] It should be noted that the foregoing explanation of the embodiment of the object surface defect detection method also applies to the object surface defect detection device of this embodiment, and will not be repeated here.

[0085] According to the object surface defect detection device proposed in the embodiments of this application, the surface image of the object to be detected can be input into the defect detection model, the defect detection model outputs a defect feature map, and the defect feature map is further identified to obtain the object surface defect data, thereby realizing the automatic detection of object surface defects, improving the defect detection efficiency and accuracy. Furthermore, the defect detection model combines the attention mechanism network and the feature pyramid network to effectively improve the network's ability to extract and detect defect features.

[0086] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0087] The memory 501, the processor 502, and the computer program stored on the memory 501 and capable of running on the processor 502.

[0088] When the processor 502 executes the program, it implements the object surface defect detection method provided in the above embodiments.

[0089] Furthermore, electronic devices also include:

[0090] Communication interface 503 is used for communication between memory 501 and processor 502.

[0091] The memory 501 is used to store computer programs that can run on the processor 502.

[0092] The memory 501 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0093] If the memory 501, processor 502, and communication interface 503 are implemented independently, then the communication interface 503, memory 501, and processor 502 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0094] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0095] Processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0096] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described method for detecting surface defects of an object.

[0097] This application also provides a computer program product that stores a computer program or instructions thereon. When the computer program or instructions are executed, the above-described method for detecting surface defects of an object is implemented.

[0098] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0099] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0100] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0101] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0102] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

Claims

1. A method for detecting defects on the surface of an object, characterized in that, Includes the following steps: Acquire an image of the surface of the object to be detected; The surface image of the object to be detected is input into a defect detection model, which outputs a defect feature map of the object's surface. The defect detection model includes a feature pyramid network, a forward attention mechanism network, and a feature decoder. The feature pyramid network extracts a target resolution feature map from the surface image of the object to be detected. The forward attention mechanism network performs target feature processing on the target resolution feature map. The feature decoder decodes the processed target resolution feature map to obtain the defect feature map. The forward attention mechanism network includes a first residual network module, a global average pooling module, a global max pooling module, a feature map stacking and convolution module, a batch normalization module, and a second residual network module. The first residual network module directly inputs a portion of the features from the target resolution feature map into the second residual network module. The high-dimensional features of the target resolution feature map are extracted; the global average pooling module performs average pooling on the feature map output by the first residual network to output a first feature map; the global max pooling module performs max pooling on the feature map output by the first residual network to output a second feature map; the feature map stacking and convolution module fuses the features in the first and second feature maps to obtain a third feature map, and reduces the dimensionality of the channel dimension of the third feature map by sharing convolution; the batch normalization module accelerates the convergence speed of the network towards the attention mechanism; the second residual network module performs unified weight processing on some features input from the first residual network module and features in the third feature map, readjusts the unified features, and outputs the processed target resolution feature map; The defect data of the surface of the object to be detected is obtained by identifying the defect feature map.

2. The method for detecting surface defects of an object according to claim 1, characterized in that, The attention mechanism network further includes an activation function module, which is used to perform nonlinear mapping calculation on the third feature map and change the resolution of the third feature map to be the same as the resolution of the target resolution feature map.

3. The method for detecting surface defects of an object according to claim 1, characterized in that, The global average pooling module and the global max pooling module perform average pooling and max pooling operations on the coordinates of the feature values ​​in the feature map output by the first residual network, respectively.

4. The method for detecting surface defects of an object according to claim 1, characterized in that, The feature pyramid network is divided into four layers from bottom to top, wherein... The first layer performs a convolution operation on the surface image of the object to be detected to obtain a first resolution feature map; The second layer performs a convolution operation on the first resolution feature map to obtain the second resolution feature map; The third layer performs a convolution operation on the second resolution feature map to obtain the third resolution feature map; The fourth layer performs a convolution operation on the third resolution feature map to obtain the target resolution feature map.

5. The method for detecting surface defects of an object according to claim 4, characterized in that, The feature decoder is divided into five to seven layers from top to bottom, wherein, The fifth layer decodes the processed target resolution feature map and fuses the features in the target resolution feature map to obtain the first decoded feature map; The sixth layer decodes the first decoding feature map and fuses the features in the third resolution feature map to obtain the second decoding feature map; The seventh layer decodes the second decoded feature map and fuses the features in the second resolution feature map to obtain a defect feature map.

6. A device for detecting surface defects of an object, characterized in that, include: The acquisition module is used to acquire images of the surface of the object to be detected. An input module is used to input the surface image of the object to be detected into a defect detection model. The defect detection model outputs a defect feature map of the surface of the object to be detected. The defect detection model includes a feature pyramid network, a forward attention mechanism network, and a feature decoder. The feature pyramid network extracts a target resolution feature map from the surface image of the object to be detected. The forward attention mechanism network performs target feature processing on the target resolution feature map. The feature decoder decodes the processed target resolution feature map to obtain the defect feature map. The forward attention mechanism network includes a first residual network module, a global average pooling module, a global max pooling module, a feature map stacking and convolution module, a batch normalization module, and a second residual network module. The first residual network module is used to directly input a portion of the features from the target resolution feature map into the second residual network. In this module, high-dimensional features of the target resolution feature map are extracted; the global average pooling module performs average pooling on the feature map output by the first residual network to output a first feature map; the global max pooling module performs max pooling on the feature map output by the first residual network to output a second feature map; the feature map stacking and convolution module fuses the features in the first and second feature maps to obtain a third feature map, and reduces the dimensionality of the third feature map by sharing convolution; the batch normalization module accelerates the convergence speed of the network towards the attention mechanism; the second residual network module performs unified weight processing on some features input from the first residual network module and features in the third feature map, readjusts the unified features, and outputs the processed target resolution feature map; The identification module is used to identify the defect feature map to obtain defect data on the surface of the object to be detected.

7. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the object surface defect detection method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the object surface defect detection method as described in any one of claims 1-5.

9. A computer program product having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed, they implement the object surface defect detection method as described in any one of claims 1-5.

Images