Industrial product defect detection method and device and computer readable storage medium
Through the convolutional block downsampling and upsampling processing of the encoder and decoder, the problem of inaccurate detection results of the image segmentation algorithm is solved, accurate detection of small defects and accurate measurement of large defect areas is achieved, and the detection speed and adaptability at low resolution are improved.
Patent Information
- Application Number
- CN202311753650.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-19
- Publication Date
- 2025-07-25
AI Technical Summary
In scenarios where detection accuracy is high, existing deep learning-based image segmentation algorithms are prone to inaccurate detection results or missed detection problems.
The encoder is used to downsample step by step through the convolution block, and the decoder is used to upsample step by step through the detailed information transmission module and the transposed convolution block to generate the final splicing information to generate a segmentation probability map.
The image segmentation model can process the detailed information accurately, detect small defects more accurately and measure the area of large defects, while improving detection speed and adaptability in low resolution conditions.
Smart Images
Figure CN120374482A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of image processing, and in particular, to an industrial product defect detection method, device, and computer-readable storage medium. Background Art
[0002] Due to its advantage of being able to handle various complex production environments, the image segmentation algorithm based on deep learning has been widely used in recent years. Most image segmentation algorithms based on deep learning usually detect image information through an image segmentation model.
[0003] In the related technical solutions, the image segmentation model first downsamples the image through the max pooling layer in the encoder, then extracts the image feature information, and then restores the resolution of the image through the decoder's upsampling. Since the max pooling layer filters out some detailed information as irrelevant information when extracting the image feature information, this will cause the related image segmentation model to easily have inaccurate detection results or missed detections in scenarios with high precision requirements for detection. This results in the defect that the accuracy of the detection results of the image segmentation algorithm is insufficient.
[0004] The above content is only used to assist in understanding the technical solution of the present application, and does not represent an admission that the above content is prior art. Summary of the Invention
[0005] The main purpose of the present application is to provide an industrial defect detection method, aiming to solve the technical problem of the defect that the accuracy of the detection results of the image segmentation algorithm is insufficient.
[0006] To achieve the above purpose, the present application provides an industrial defect detection method, and the industrial defect detection method includes the following steps:
[0007] When the encoder receives the image to be detected, it downsamples the image to be detected step by step through a convolutional block;
[0008] When the decoder receives the encoded result output by the encoder, it upsamples the encoded result step by step and determines the final splicing information according to the splicing information, where the decoder includes a detailed information transmission module and a transposed convolutional block, and the detailed information transmission module and the transposed convolutional block are used to generate the splicing information according to the upsampling result generated by itself and the sampling result generated by the corresponding-level convolutional block;
[0009] Generate a segmentation probability map according to the final splicing information.
[0010] Optionally, the encoder includes at least 3 convolutional blocks, and when the encoder receives the image to be detected, it downsamples the image to be detected step by step through the convolutional blocks;
[0011] When the convolutional block receives input data, it downsamples the input data and sends the generated downsampling result to the detail information transmission module or the transposed convolutional block at the corresponding level;
[0012] And, it sends the downsampling result to the next convolutional block.
[0013] Optionally, the encoder includes 3 first convolutional blocks and 1 second convolutional block arranged in sequence; the first convolutional block includes 1 convolutional layer with a stride of 2 and 1 convolutional layer with a stride of 1, and the second convolutional block includes 1 convolutional layer with a stride of 2 and 3 convolutional layers with a stride of 1.
[0014] Optionally, the decoder includes at least 1 transposed convolutional block and at least 2 detail information transmission modules. When the decoder receives the image to be detected, it upsamples the image to be detected step by step through the transposed convolutional block or the detail information transmission module;
[0015] When the transposed convolutional block or the detail information transmission module receives input data, it upsamples the input data, and after receiving the downsampling result sent by the convolutional block at the corresponding level, it splices the upsampling result and the downsampling result to generate the splicing information;
[0016] And, it sends the splicing information to the next transposed convolutional block or detail information transmission module, or outputs the splicing information as the final splicing information.
[0017] Optionally, the step of when the transposed convolutional block or the detail information transmission module receives input data, upsamples the input data, and after receiving the downsampling result sent by the convolutional block at the corresponding level, splicing the upsampling result and the downsampling result to generate the splicing information includes:
[0018] When the detail information transmission module receives the first splicing information and the downsampling result, it processes the first splicing information through the splicing information processing module to generate a first processing result, and processes the first splicing information through the downsampling result processing module to generate a second processing result;
[0019] Splice the first processing result and the second processing result through the first information splicing module, and splice the first processing result and the splicing result generated by the first information splicing module through the second information splicing module to generate the second splicing information.
[0020] Optionally, the detailed information transmission module includes a downsampling result processing module, a splicing information processing module, a first information splicing module, and a second information splicing module; the downsampling result processing module includes a convolutional layer with a stride of 1, the splicing information processing module includes a convolutional layer with a stride of 1 and a transposed convolutional layer with a stride of 2 arranged in sequence, and each of the first information splicing module and the second information splicing module includes a splicing layer and a convolutional layer with a stride of 1.
[0021] Optionally, the decoder includes a transposed convolutional block and two detailed information transmission modules arranged in sequence; the transposed convolutional block includes a transposed convolutional layer with a stride of 2, a splicing layer, and a convolutional layer with a stride of 1, and the detailed information transmission module includes four convolutional layers with a stride of 1, a transposed convolutional layer with a stride of 2, and two splicing layers.
[0022] Optionally, before the encoder performs successive downsampling on the image to be detected through a convolutional block when receiving the image to be detected, it further includes:
[0023] Obtain a product image and recognize the product image;
[0024] When the product image contains defect annotations, binarize the defect annotations, and based on the contour points of the defect annotations, determine the minimum bounding rectangle of the defect annotations, and determine the length of the short side of the minimum bounding rectangle; select the product image with the short side length less than a preset length threshold as the target product image;
[0025] Alternatively, when the product image does not contain defect annotations, select the product image as the target product image;
[0026] Crop the target product image according to a preset resolution to generate the image to be detected.
[0027] In addition, to achieve the above object, the present application further provides an industrial product defect detection device, which includes: a memory, a processor, and an industrial product defect detection program stored on the memory and executable on the processor, and the industrial product defect detection program is configured to implement the steps of the industrial product defect detection method as described above.
[0028] In addition, to achieve the above object, the present application further provides a computer-readable storage medium, on which an industrial product defect detection program is stored, and when the industrial product defect detection program is executed by a processor, it implements the steps of the industrial product defect detection method as described above.
[0029] This application improves the accuracy of the image segmentation model in processing detailed information, enabling the image segmentation model to more accurately detect small defects and more accurately identify the area of large defects. At the same time, this application performs downsampling through multiple convolutional blocks to reduce the image resolution and the amount of data to be processed during the image upsampling process, thereby improving the detection speed of the image. In addition, this application has better adaptability to the situation of low image resolution and can accurately identify small defects and the area of large defects at a lower resolution. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 is a schematic flowchart of the first embodiment of the industrial product defect detection method of this application;
[0031] Figure 2 is a schematic structural diagram of the image segmentation model involved in the first embodiment of the industrial product defect detection method of this application;
[0032] Figure 3 is a schematic flowchart of the image processing process of the image segmentation model involved in the first embodiment of the industrial product defect detection method of this application;
[0033] Figure 4 is a schematic structural diagram of the detailed information transmission module involved in the first embodiment of the industrial product defect detection method of this application;
[0034] Figure 5 is a schematic flowchart of the detailed information extraction process of the detailed information transmission module involved in the first embodiment of the industrial product defect detection method of this application;
[0035] Figure 6 is a schematic flowchart of the second embodiment of the industrial product defect detection method of this application;
[0036] Figure 7 is a schematic flowchart of the third embodiment of the industrial product defect detection method of this application;
[0037] Figure 8 is a schematic flowchart of the fourth embodiment of the industrial product defect detection method of this application;
[0038] Figure 9 is a schematic structural diagram of the industrial product defect detection device in the hardware operating environment involved in the embodiment solution of this application.
[0039] The realization, functional features and advantages of the purpose of this application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0040] It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.
[0041] In this application, downsampling is performed through a convolutional layer, and a detail information transmission module is used to extract detail information, which improves the accuracy of the image segmentation model in processing detail information, enables the image segmentation model to more accurately detect small defects, and measure the area of large defects.
[0042] To better understand the above technical solution, the exemplary embodiments of this application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.
[0043] To better understand the above technical solution, the above technical solution will be described in detail below in combination with the accompanying drawings of the specification and specific implementation manners.
[0044] Embodiment 1
[0045] In an actual production environment, the detection speed of the image segmentation model for images cannot be slower than the transmission speed of products on the assembly line. Therefore, the resolution of the images input into the image segmentation model is often reduced to improve the detection speed. To avoid the situation where the image segmentation model is not accurate enough in defect recognition or even misses detections when the image resolution is small, the embodiments of this application provide an industrial product defect detection method, referring to Figure 1 , Figure 1 which is a schematic flowchart of the first embodiment of the industrial product defect detection method of this application.
[0046] In this embodiment, the industrial product defect detection method includes:
[0047] Step S10: When the encoder receives the image to be detected, it performs successive downsampling on the image to be detected through a convolutional block;
[0048] In this embodiment, the image segmentation model consists of an encoder and a decoder. The encoder will perform downsampling processing on the image to be detected input into the image segmentation model to extract the feature information of the image to be detected. One or more convolutional blocks are included in this encoder, which can perform downsampling on the image to be detected.
[0049] As an alternative implementation, the encoder includes at least 3 convolutional blocks. When the encoder receives the image to be detected, it downsamples the image to be detected step by step through the convolutional blocks. When a convolutional block receives input data, it downsamples the input data through a convolutional layer and sends the generated downsampling result to the detail information transmission module or the transposed convolutional block at the corresponding level. At the same time, the convolutional block sends the downsampling result to the next convolutional block. Each convolutional block in the encoder corresponds to a detail information transmission module or a transposed convolutional block.
[0050] In specific implementation, the convolutional layer performs a convolution operation on the image to be detected according to the convolution kernel and the preset stride. Each convolutional layer is preset with a corresponding convolution kernel. After receiving the image to be detected, the convolutional layer starts from the upper left corner of the image and performs a weighted summation operation on the pixel points of the image. At the same time, after each operation is completed, the convolutional layer controls the convolution kernel to move once according to the preset stride and performs the weighted summation operation again. After the weighted summation operation is completed for all pixel points of the image, the convolutional layer outputs the convolution result according to the operation result of the weighted summation operation. By downsampling the image to be detected through the convolutional layer, the encoder can extract the feature information of the image to be detected and reduce the resolution of the image to be detected while retaining the detail information of the image to be detected.
[0051] Exemplarily, as Figure 2 shown, Figure 2 is a schematic diagram of the image segmentation model structure involved in the embodiments of the present application. Three first convolutional blocks and one second convolutional block are sequentially arranged in the encoder. Among them, the first convolutional block includes one convolutional layer with a stride of 2 and one convolutional layer with a stride of 1, and the second convolutional block includes one convolutional layer with a stride of 2 and three convolutional layers with a stride of 1.
[0052] As Figure 3 shown, Figure 3 is a schematic diagram of the image processing flow of the image segmentation model involved in the embodiments of the present application. Among them, Conv[stride = 1], BN, Relu are convolutional operations with a stride of 1, Conv[stride = 2], BN, Relu are convolutional operations with a stride of 2, DITM is a detail information processing module, TransConv[stride = 2], BN, Relu are convolutional operations with a stride of 2, and Concat is a concatenation operation performed by the concatenation layer. After the first convolutional block receives the input data, the encoder sequentially performs a convolutional operation with a stride of 2 and a convolutional operation with a stride of 1 on the image to be detected through the convolutional layer in the first convolutional block, and sends the downsampling result output by the convolutional block to the next convolutional block and the detail information transmission module or the transposed convolutional block at the corresponding level.
[0053] As Figure 3As shown, when the second convolutional block receives the downsampling result sent by the first convolutional block, the encoder will perform a convolutional operation with a stride of 2 once and three convolutional operations with a stride of 1 on the image to be detected in sequence according to the convolutional layers in the second convolutional block, so as to further extract the defect-related feature information in the image to be detected and suppress the irrelevant information. After completing the convolutional operation, the encoder will send the downsampling result output by the second convolutional block to the transposed convolutional block at the first level in the decoder.
[0054] Furthermore, before the decoder downsamples the image to be detected through the first convolutional block, it will perform a convolutional operation on the image to be detected through the convolutional layer to extract the detailed information in the image to be detected, so as to avoid losing the necessary detailed information in the subsequent operations of the image to be detected.
[0055] As another alternative embodiment, there is only one convolutional block in the encoder. The convolutional block includes at least one convolutional layer with a stride of 1 and at least one convolutional layer with a stride of 2. The encoder downsamples the image to be detected through this convolutional block and sends the downsampling result to each detail information transmission module and transposed convolutional block.
[0056] Step S20: When the decoder receives the encoded result output by the encoder, it performs upsampling on the encoded result level by level and determines the final splicing information according to the splicing information, where the decoder includes a detail information transmission module and a transposed convolutional block, and the detail information transmission module and the transposed convolutional block are used to generate the splicing information according to the upsampling result generated by themselves and the sampling result generated by the corresponding-level convolutional block;
[0057] In this embodiment, the decoder can perform transposed convolution on the upsampling result output by the encoder to restore the resolution of the image. At the same time, the decoder can extract more detailed information about the defect to be detected in the image to be detected through the detail information transmission module.
[0058] As an alternative embodiment, the encoder includes a detail information transmission module and a transposed convolutional block. Among them, each detail information transmission module and each transposed convolutional block have a corresponding-level convolutional block. After completing the downsampling, this convolutional block will send the downsampling result to the corresponding-level detail information transmission module or transposed convolutional block.
[0059] Specifically, as Figure 4 shown, Figure 4Schematic diagram of the structure of the detailed information transmission module involved in the embodiments of the present application. The detailed information transmission module includes a downsampling result processing module, a splicing information processing module, a first information splicing module, and a second information splicing module. The detailed information transmission module first processes the received downsampling result and splicing information through the downsampling result processing module and the splicing information transmission processing module, and then splices the processing results of the downsampling result processing module and the splicing information processing module through the first information splicing module and the second information splicing module.
[0060] During specific implementation, as Figure 5 shown, Figure 5 Schematic diagram of the detailed information extraction process of the detailed information transmission module involved in this embodiment. The downsampling result processing module includes a convolutional layer with a stride of 1, and can extract the detailed information of the obtained downsampling result through the convolutional operation with a stride of 1. The splicing information processing module includes a convolutional layer with a stride of 1 and a transposed convolutional layer with a stride of 2 arranged in sequence, and can perform upsampling on the received splicing information. Each of the first information splicing module and the second information splicing module includes a splicing layer and a convolutional layer with a stride of 1. The first information splicing module can splice the first processing result generated by the splicing information processing module and the second processing result generated by the downsampling result processing module, and perform a convolutional operation with a stride of 1. The second information splicing module will splice the splicing result of the first information splicing module and the first processing result, then perform a convolutional operation with a stride of 1, and output the splicing information generated by the detailed information transmission module.
[0061] Exemplarily, as Figure 2 shown, there is a transposed convolutional block and two detailed information transmission modules arranged in sequence in the decoder. When the decoder receives the downsampling result transmitted by the decoder, it first performs upsampling on the downsampling result through the transposed convolutional block, and then extracts detailed information through the detailed information transmission module. Among them, the transposed convolutional block includes a transposed convolutional layer with a stride of 2, a splicing layer, and a convolutional layer with a stride of 1. When the transposed convolutional layer receives the input data, it will perform a transposed convolution with a stride of 2 on the downsampling result through the transposed convolutional layer to restore the resolution of the image. The transposed convolutional block splices the transposed convolution result and the downsampling result sent by the corresponding level of convolutional block, and generates splicing information based on the convolutional operation with a stride of 1.
[0062] Step S30: Generate a segmentation probability map according to the final splicing information.
[0063] In this embodiment, the image segmentation model generates a corresponding segmentation probability map based on the final stitching information output by the decoder, and determines the probability of the existence of defects based on the segmentation probability map. The image segmentation model can perform a convolution operation on the final stitching information through a convolutional layer with a stride of 1 to generate a segmentation probability map, or can generate a segmentation probability map according to the feature information of the image to be detected.
[0064] It should be noted that the segmentation probability map is an image generated by the image segmentation model based on image processing technology to segment the input image. Each pixel point in the segmentation probability map is assigned a probability vector to represent the similarity between the pixel point and its adjacent pixel points.
[0065] As an alternative implementation of generating a segmentation probability map according to the final stitching information, the image segmentation model can assign a probability vector to each pixel point of the final stitching information based on the feature information of the image to be detected to generate a segmentation probability map.
[0066] As another alternative implementation of generating a segmentation probability map according to the final stitching information, the image segmentation model can also determine the probability of the existence of defects for each pixel point through a convolution operation and generate a segmentation probability map.
[0067] Exemplarily, the image segmentation model can map the features of each pixel point to the corresponding position in the segmentation probability map through a convolution with a stride of 1, obtain the probability of the existence of defects for each pixel point, and generate a segmentation probability map.
[0068] In the embodiment of the present application, a convolutional block is used to downsample the image to be detected, and transposed convolutional blocks and a detail information transmission module are used for upsampling, and the detail information in the image is extracted. The present application can more accurately detect small defects and the boundaries of large defects according to the extracted detail information. At the same time, in the embodiment of the present application, multiple convolutional blocks are used for downsampling to reduce the image resolution and the amount of data to be processed in the image upsampling process, thereby improving the detection speed of the image.
[0069] Embodiment Two
[0070] Based on the same inventive concept, the present application also provides a second embodiment. Refer to Figure 6 , Figure 6 which is a schematic flowchart of the second embodiment of the industrial product defect detection method of the present application.
[0071] In this embodiment, before the encoder as described in step S10 performs successive downsampling on the image to be detected through convolutional blocks after receiving the image to be detected, it further includes:
[0072] Step S11: Obtain a product image and identify the product image;
[0073] Step S12: Crop the product image according to a preset resolution to generate the image to be detected.
[0074] In this embodiment, a preset resolution is stored in the image segmentation model. The image segmentation model can crop the product image based on this preset resolution, remove the edge information irrelevant to defect detection, and generate the image to be detected with the preset resolution.
[0075] Exemplarily, after obtaining the product image, the image segmentation model will identify the product image, determine the position of the product image in the product image, and crop the product image based on the position of the product image and the preset resolution to generate the image to be detected, so as to ensure that the cropping process will not affect the accuracy of subsequent detection.
[0076] The embodiment of the present application can analyze and preprocess the product image, reduce the unnecessary operation process of the image segmentation model, and save resource consumption.
[0077] Since the system introduced in the second embodiment of the present application is the system adopted for implementing the method of the first embodiment of the present application, based on the method introduced in the first embodiment of the present application, those skilled in the art can understand the specific structure and deformation of the system, so it will not be elaborated here. Any system adopted by the method of the first embodiment of the present application falls within the scope of protection of the present application.
[0078] Embodiment Three
[0079] Based on the same inventive concept, the present application also provides a third embodiment. Refer to Figure 7 , Figure 7 which is a schematic flowchart of the third embodiment of the industrial product defect detection method of the present application.
[0080] In this embodiment, before the encoder as described in step S10 performs successive downsampling on the image to be detected through the convolutional block when receiving the image to be detected, it further includes:
[0081] Step S13: Obtain a defect annotation image and generate a data set based on the defect annotation image;
[0082] In this embodiment, the image processing tool will obtain and process the defect annotation image and generate a data set. The image processing tool can binarize the defect annotation and determine the circumscribed rectangle of the defect annotation according to the contour points of the binarized defect annotation. The image segmentation model can determine the minimum circumscribed rectangle of the defect annotation and the length of the short side of the minimum circumscribed rectangle according to the circumscribed rectangle of the defect annotation. When the length of the short side of the minimum circumscribed rectangle is less than a preset length threshold, the image segmentation model will select the defect annotation image as the target defect image. In addition, the image processing tool will crop the target defect image based on a preset resolution to generate a data set.
[0083] Exemplarily, after determining the contour points of the defect annotation in the product image, the image segmentation model can determine the first circumscribed rectangle of the defect annotation according to the contour points, as well as the corner coordinates and area of the circumscribed rectangle. By means of rotation, the image segmentation model can determine other circumscribed rectangles of the defect annotation, as well as the corresponding corner coordinates and areas. At the same time, for each determined circumscribed rectangle, the currently smallest circumscribed rectangle can be determined through the bubble sort algorithm, and after all circumscribed rectangles are determined, the smallest circumscribed rectangle is determined. According to the corner coordinates of the smallest circumscribed rectangle, the image segmentation model can determine the short side length of the smallest circumscribed rectangle and determine whether the product image is the target product image.
[0084] Step S14: Construct an image segmentation network based on the Unet neural network;
[0085] In this embodiment, an image segmentation network can be generated based on the Unet neural network by replacing the max pooling layer in the decoder with a convolutional layer and embedding a detail information transmission module into the decoder. When the image segmentation network downsamples the image to be detected through the convolutional layer, the detail information of the image to be detected is retained. Through the detail information transmission module, the detail information of the image to be detected can be extracted to accurately identify the boundaries of small and large defects in the image.
[0086] It should be noted that the Unet neural network is a neural network model commonly used for image segmentation. The Unet neural network usually consists of two parts: the left and the right. The left part is a feature extraction network for extracting the abstract features of the image, while the right part is a feature fusion operation. The Unet neural network realizes the fusion of shallow low-resolution and deep high-resolution information through the feature fusion operation, making full use of the context information of the image.
[0087] Step S15: Train and test the image segmentation network based on the dataset, and iterate the image segmentation network based on the test results.
[0088] In this embodiment, the dataset is divided into a training set and a test set. The image segmentation network can be trained through the training set, and the training results can be tested through the test set. Based on the test results, the parameters of the image segmentation network are adjusted to generate a high-precision image segmentation model. The adjustable parameters of the image segmentation network include but are not limited to convolutional kernel data, the number of convolutional layers, the number of steps of convolutional operations, the number of steps of transposed convolution, etc.
[0089] Exemplarily, after the image segmentation network is trained, if there is a situation of missing detail information, a detail information extraction module in the decoder can be added to increase the accuracy of the image segmentation network in extracting detail information.
[0090] In the embodiments of the present application, an image segmentation network constructed based on the Unet neural network can be trained and tested with a dataset, and the parameters of the image segmentation network can be adjusted based on the test results to generate a high-precision image segmentation model.
[0091] Since the system introduced in the third embodiment of the present application is the system adopted for implementing the method of the first embodiment of the present application, based on the method introduced in the first embodiment of the present application, those skilled in the art can understand the specific structure and variations of the system, so it will not be elaborated here. Any system adopted for the method of the first embodiment of the present application falls within the scope of protection of the present application.
[0092] Embodiment 4
[0093] Based on the same inventive concept, the present application also provides a fourth embodiment. Refer to Figure 8 , Figure 8 , which is a schematic flowchart of the fourth embodiment of the industrial product defect detection method of the present application.
[0094] In this embodiment, after generating the segmentation probability map according to the final stitching information as described in step S30, the following steps are further included:
[0095] Step S31: Read the segmentation probability map, and confirm and mark the target pixel points whose defect existence probability is higher than the probability threshold;
[0096] Step S32: When the number of the target pixel points is higher than the number threshold, output a prompt message indicating the existence of a defect.
[0097] In this embodiment, the image segmentation model can identify the defect existence probability corresponding to each pixel point in the segmentation probability map. When the defect existence probability is higher than the preset probability threshold, the image segmentation model will mark the pixel point. After traversing all the pixel points, the image segmentation model will calculate the number of target pixel points based on the marks, and when the target pixel points are higher than the preset number threshold, output a prompt message indicating the existence of a defect.
[0098] Optionally, the image segmentation model can also calculate the probability of the existence of a defect in the image to be detected based on the number of target pixel points and the position of the target pixel points, and output a prompt message indicating the existence of a defect when the probability of the existence of a defect is higher than the defect existence probability threshold.
[0099] Exemplarily, the image segmentation model will traverse each pixel point in the segmentation probability map, read the label of the pixel point, and determine the defect existence probability of the pixel point through the label information.
[0100] In the embodiments of the present application, the image segmentation model can judge the probability of the existence of a defect based on the number of target pixel points higher than the probability threshold in the segmentation probability map.
[0101] Since the system introduced in the fourth embodiment of this application is the system adopted for implementing the method in the first embodiment of this application, based on the method introduced in the first embodiment of this application, those skilled in the art can understand the specific structure and variations of the system, so it will not be elaborated here. Any system adopted for the method in the first embodiment of this application falls within the scope of protection of this application.
[0102] Embodiment Five
[0103] Refer to Figure 9 , Figure 9 which is a schematic structural diagram of an industrial product defect detection device for the hardware operating environment involved in the solution of the embodiment of this application.
[0104] As Figure 9 shown, the industrial product defect detection device may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a network interface 1003, and a memory 1004. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The network interface 1003 may optionally include a standard wired interface and a wireless interface (such as a Wireless-Fidelity (WI-FI) interface). The memory 1004 may be a high-speed Random Access Memory (RAM) memory or a stable Non-Volatile Memory (NVM), such as a disk memory. The memory 1004 may also optionally be a storage device independent of the aforementioned processor 1001.
[0105] Those skilled in the art can understand that Figure 9 the structure shown in
[0106] As Figure 9 shown, the memory 1004, as a storage medium, may include an operating system, a data storage module, a network communication module, and an industrial product defect detection program.
[0107] In Figure 9In the industrial product defect detection device shown, the network interface 1003 is mainly used for data communication with other devices; the processor 1001 and the memory 1004 in the industrial product defect detection device of the present application can be arranged in the industrial product defect detection device. The industrial product defect detection device calls the industrial product defect detection program stored in the memory 1004 through the processor 1001 and executes the industrial product defect detection method provided by the embodiments of the present application.
[0108] In addition, the present application also provides a computer-readable storage medium storing the industrial product defect detection program, and the industrial product defect detection program can also be executed by a processor to implement the steps of the above-mentioned industrial product defect detection method embodiments.
[0109] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.
[0110] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified functions in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.
[0111] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the specified functions in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.
[0112] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are executed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions for implementing the functions specified in one process or a plurality of processes and / or boxes Figure 1 one process or a plurality of processes and / or boxes Figure 1 steps for implementing the functions specified in one box or a plurality of boxes.
[0113] It should be noted that, in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of other elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims listing several means, several of these means can be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words can be interpreted as names.
[0114] Although the preferred embodiments of the present application have been described, additional changes and modifications can be made by those skilled in the art once they learn of the basic inventive concept. Therefore, the appended claims are intended to be construed to cover the preferred embodiments as well as all changes and modifications falling within the scope of the present application.
[0115] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.
Claims
1. An industrial product defect detection method, characterized in that, The industrial product defect detection method includes the following steps: When the encoder receives the image to be detected, it performs progressive downsampling on the image to be detected through a convolutional block; When the decoder receives the encoded result output by the encoder, it performs progressive upsampling on the encoded result and determines the final splicing information according to the splicing information. Wherein, the decoder includes a detail information transmission module and a transposed convolutional block, and the detail information transmission module and the transposed convolutional block are used to generate the splicing information according to the upsampling result generated by itself and the sampling result generated by the convolutional block at the corresponding level; Generate a segmentation probability map according to the final splicing information.
2. The industrial product defect detection method according to claim 1, wherein The encoder includes at least 3 of the convolutional blocks. When the encoder receives the image to be detected, it performs progressive downsampling on the image to be detected through the convolutional blocks; When the convolutional block receives the input data, it performs downsampling on the input data and sends the generated downsampling result to the detail information transmission module or the transposed convolutional block at the corresponding level; And send the downsampling result to the next convolutional block.
3. The industrial product defect detection method according to any one of claims 1 to 2, characterized in that, The encoder includes 3 first convolutional blocks and 1 second convolutional block arranged in sequence; the first convolutional block includes 1 convolutional layer with a stride of 2 and 1 convolutional layer with a stride of 1, and the second convolutional block includes 1 convolutional layer with a stride of 2 and 3 convolutional layers with a stride of 1.
4. The industrial product defect detection method according to claim 1, wherein The decoder includes at least 1 of the transposed convolutional blocks and at least 2 of the detail information transmission modules. When the decoder receives the image to be detected, it performs progressive upsampling on the image to be detected through the transposed convolutional block or the detail information transmission module; When the transposed convolutional block or the detail information transmission module receives the input data, it performs upsampling on the input data, and after receiving the downsampling result sent by the convolutional block at the corresponding level, it splices the upsampling result and the downsampling result to generate the splicing information; And send the splicing information to the next transposed convolutional block or the detail information transmission module, or output the splicing information as the final splicing information.
5. The industrial product defect detection method according to claim 4, wherein, The step of when the transposed convolutional block or the detail information transmission module receives the input data, performs upsampling on the input data, and after receiving the downsampling result sent by the convolutional block at the corresponding level, splices the upsampling result and the downsampling result to generate the splicing information includes: When the detail information transmission module receives the first splicing information and the downsampling result, it processes the first splicing information through the splicing information processing module to generate a first processing result, and processes the first splicing information through the downsampling result processing module to generate a second processing result; Splice the first processing result and the second processing result through the first information splicing module, and splice the first processing result and the splicing result generated by the first information splicing module through the second information splicing module to generate a second splicing information.
6. The industrial product defect detection method according to claim 5, characterized in that, The detailed information transmission module includes a downsampling result processing module, a splicing information processing module, a first information splicing module, and a second information splicing module; the downsampling result processing module includes a convolutional layer with a stride of 1, the splicing information processing module includes a convolutional layer with a stride of 1 and a transposed convolutional layer with a stride of 2 arranged in sequence, and each of the first information splicing module and the second information splicing module includes a splicing layer and a convolutional layer with a stride of 1.
7. The industrial product defect detection method according to claim 1 or 4, characterized in that, The decoder includes a transposed convolutional block and two detailed information transmission modules arranged in sequence; the transposed convolutional block includes a transposed convolutional layer with a stride of 2, a splicing layer, and a convolutional layer with a stride of 1, and the detailed information transmission module includes four convolutional layers with a stride of 1, a transposed convolutional layer with a stride of 2, and two splicing layers.
8. The industrial product defect detection method according to claim 1, characterized in that, Before the encoder performs the step of gradually downsampling the image to be detected through the convolutional block when receiving the image to be detected, it further includes: Obtaining a product image and identifying the product image; When the product image contains a defect annotation, binarizing the defect annotation, and based on the contour points of the defect annotation, determining the minimum bounding rectangle of the defect annotation, and determining the length of the short side of the minimum bounding rectangle; selecting the product image with the short side length less than a preset length threshold as the target product image; Alternatively, when the product image does not contain a defect annotation, selecting the product image as the target product image; Cropping the target product image according to a preset resolution to generate the image to be detected.
9. An industrial product defect detection device, characterized in that, The industrial product defect detection device includes: a memory, a processor, and an industrial product defect detection program stored on the memory and executable on the processor, and the industrial product defect detection program is configured to implement the steps of the industrial product defect detection method according to any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that, An industrial product defect detection program is stored on the computer-readable storage medium, and when the industrial product defect detection program is executed by a processor, it implements the steps of the industrial product defect detection method according to any one of claims 1 to 8.