Product defect detection method and device, computer equipment and storage medium

Abstract

The invention relates to a product defect detection method and device, computer equipment and a storage medium. The method comprises the steps of obtaining an original image and a size adjustment image of a target product, wherein the size adjustment image is obtained by performing size reduction adjustment on the original image; multiple rounds of down-sampling processing and feature fusion processing are carried out based on the size adjustment image, and feature fusion images under different target sizes are obtained; determining a defect connected domain in the size adjustment image based on the feature fusion images under different target sizes; based on the defect connected domain, segmenting a defect category detection image containing a background from the original image; and performing defect category detection based on the defect connected domain and the defect category detection image to obtain a defect detection result corresponding to the target product. According to the invention, the accuracy of product defect detection can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a product defect detection method, device, computer equipment and storage medium. Background Art

[0002] With the rapid development of industrial technology, products are widely used in production. For example, square-shell power batteries can be produced and then used in new energy vehicles. Products may have defects during the production process. For example, square-shell power batteries may have defects such as aluminum wire during the production process. If product defects are not discovered and handled in a timely manner, the safety of the product may be reduced.

[0003] At present, the main method is to collect images of products and convert the collected images into grayscale images, and detect defects in products by the grayscale difference between defects and background. However, this method is difficult to adapt to complex environments. Changes in lighting, changes in the optical properties of the product surface, etc. will cause changes in image grayscale, and thus cannot accurately detect defects. Summary of the invention

[0004] Based on this, it is necessary to provide a product defect detection method, apparatus, computer equipment, computer readable storage medium and computer program product to address the above technical problems, which can improve the accuracy of product defect detection.

[0005] In a first aspect, the present application provides a product defect detection method, comprising:

[0006] Obtaining an original image and a resized image of a target product, wherein the resized image is obtained by reducing the size of the original image;

[0007] Perform multiple rounds of downsampling and feature fusion processing based on the resized image to obtain feature fusion images at different target sizes;

[0008] Based on the feature fusion images at different target sizes, the defect connected domain in the resized image is determined;

[0009] Based on the defect connected domain, the defect category detection image including the background is segmented from the original image;

[0010] Defect category detection is performed based on the defect connected domain and defect category detection image to obtain the defect detection result corresponding to the target product.

[0011] In a second aspect, the present application also provides a product defect detection device, comprising:

[0012] An image acquisition module is used to acquire an original image and a resized image of a target product, wherein the resized image is obtained by reducing the size of the original image;

[0013] A feature fusion module is used to perform multiple rounds of downsampling and feature fusion processing based on the resized image to obtain feature fusion images at different target sizes;

[0014] A connected domain determination module, for determining a defect connected domain in a resized image based on feature fusion images at different target sizes;

[0015] An image segmentation module is used to segment a defect category detection image including a background from an original image based on a defect connected domain;

[0016] The category detection module is used to perform defect category detection based on the defect connected domain and the defect category detection image to obtain the defect detection result corresponding to the target product.

[0017] In a third aspect, the present application provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method when executing the computer program.

[0018] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above method when executed by a processor.

[0019] In a fifth aspect, the present application also provides a computer program product, which includes a computer program, and the computer program implements the steps in the above method when executed by a processor.

[0020] The above-mentioned product defect detection method, apparatus, computer device, computer-readable storage medium and computer program product can obtain feature fusion images at different target sizes, and can determine feature information of the resized image at different sizes based on the feature fusion images at different target sizes, thereby improving the accuracy of determining the defect connected domain based on the feature information at different sizes. Since defect category detection can be performed based on the defect connected domain in the resized image and the defect category detection image segmented from the original image, the connected domain information at different sizes can be fully utilized in the process of defect category detection, thereby further improving the accuracy of product defect detection based on the fully utilized information.

[0021] In addition, since the resized image is subjected to multiple rounds of downsampling and feature fusion processing to obtain feature fusion images at different target sizes, compared to directly performing multiple rounds of downsampling and feature fusion processing on the original image, when the resized image is obtained by reducing the original image, the amount of data for calculation can also be reduced, thereby improving the detection efficiency of product defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 An application environment diagram of a product defect detection method provided in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of a process for detecting product defects provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of downsampling and feature fusion provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of defect connectivity domain mapping provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of the overall process of product defect detection provided in an embodiment of the present application;

[0027] Figure 6 A schematic diagram of a product area provided in an embodiment of the present application;

[0028] Figure 7 A schematic diagram of another overall process of product defect detection provided in an embodiment of the present application;

[0029] Figure 8 A structural block diagram of a product defect detection device provided in an embodiment of the present application;

[0030] Fig. 9 An internal structure diagram of a computer device provided in an embodiment of the present application;

[0031] Fig.10 An internal structure diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0033] The product defect detection method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0034] like Figure 2 As shown, the embodiment of the present application provides a product defect detection method, which is applied to Figure 1 The terminal 102 or the server 104 in the example is used for explanation. It is understandable that the computer device may include at least one of the terminal and the server. The method includes the following steps:

[0035] Step 202: obtaining an original image and a resized image of the target product. The resized image is obtained by reducing the size of the original image.

[0036] The original image refers to the image acquired by capturing the target product. For example, the original image may be obtained by photographing the top cover of a square shell power battery.

[0037] For example, when defect detection of a target product is required, the computer device may directly obtain the original image and a resized image corresponding to the original image. The computer device may also obtain the original image and resize the original image to obtain a corresponding resized image.

[0038] Step 204 , performing multiple rounds of downsampling processing and feature fusion processing based on the resized image to obtain feature fusion images at different target sizes.

[0039] Among them, downsampling refers to the operation of changing the spatial size of the feature map, and downsampling is usually used to reduce the spatial size of the feature map. Feature fusion refers to the process of fusing feature images of different sizes. Different target sizes can be pre-set sizes. For example, different target sizes can be 64*48*32, 32*24*64, 16*12*128, and 8*6*256, so that feature fusion images of different target sizes include feature fusion images of 64*48*32 size, feature fusion images of 32*24*64 size, feature fusion images of 16*12*128 size, and feature fusion images of 8*6*256 size.

[0040] Exemplarily, after obtaining the resized image, the computer device may perform multiple rounds of downsampling processing and feature fusion processing on the resized image to obtain feature fusion images at different target sizes. For example, each round is provided with a corresponding downsampling multiple set. In each round, the computer device may obtain the corresponding downsampling multiple set, and downsample the resized image based on the downsampling multiples in the corresponding downsampling multiple set to obtain multiple downsampled images, and perform feature fusion on the multiple downsampled images to obtain feature fusion images. The computer device uses the feature fusion images generated in each round as feature fusion images at different target sizes.

[0041] In some embodiments, during the multiple rounds of downsampling and feature fusion processing, except for the first round, the output of the previous round is the input of the next round. For example, the computer device determines the feature fusion images of different sizes of the first round based on the resized image, and in the current round starting from the second round, the downsampling process is performed based on at least part of the feature fusion image of the previous round to obtain the downsampled image of the current round, and the feature fusion process is performed based on the downsampled image of the current round and at least part of the feature fusion image of the previous round to obtain the feature fusion image of different sizes of the current round.

[0042] Step 206 , based on the feature fusion images at different target sizes, determine the defect connected domain in the resized image.

[0043] Step 208 : segmenting a defect category detection image including the background from the original image based on the defect connected domain.

[0044] The defect connected domain refers to the area where defects may exist. Defects include but are not limited to aluminum wire, dirt, foreign matter, breakage, deformation, scratches, abrasions, crushing, bubbles, different colors, wrinkles, cracks and bumps.

[0045] The defect category detection image refers to an image used for defect category detection, and the defect category detection image may include not only defects but also background.

[0046] Exemplarily, a segmentation model is deployed in a computer device, wherein the segmentation model includes at least a feature extraction module and a target detection module. The feature extraction module is used to determine a feature fusion image at different target sizes based on the resized image; the target detection module is used to perform defect detection based on the feature fusion image at different target sizes to obtain a defect connected domain in the resized image. The computer device determines the scaling ratio between the resized image and the original image, determines a mapping area in the original image based on the scaling ratio and the position information of the defect connected domain in the resized image, and segments a defect category detection image from the original image based on the mapping area in the original image.

[0047] In some embodiments, after the feature extraction module in the segmentation model outputs feature fusion images at different target sizes, the target detection module can determine the defect connected domain in the resized image based on the feature fusion images at different target sizes. For example, when the target detection module obtains the feature fusion images at different target sizes, it can determine whether each pixel in the resized image is a pixel where a defect is located based on the feature fusion images at different target sizes, and if it is determined to be a pixel where a defect is located, the pixel is marked. The target detection module determines the defect connected domain based on the pixels marked as "defect" and clustered together.

[0048] In some embodiments, the segmentation model further includes an image segmentation module, and the image segmentation module is used to segment the defect category detection image from the original image based on the defect connected domain in the resized image.

[0049] In some embodiments, when the mapping area in the original image is determined, the computer device can directly segment the mapping area in the original image to obtain a defect category detection image. Alternatively, the computer device can expand the mapping area in the original image outward by a preset number of pixels to obtain an outward expansion connected domain, and segment the outward expansion connected domain from the original image to obtain a defect category detection image.

[0050] Step 210 , performing defect category detection based on the defect connected domain and the defect category detection image to obtain a defect detection result of the target product.

[0051] Exemplarily, a defect classification model is deployed in a computer device, and the computer device outputs the defect connected domain in the resized image and the defect category detection image to the defect classification model, and performs defect category detection through the defect classification model to obtain a defect detection result corresponding to the target product.

[0052] In some embodiments, the defect detection results may include a first type defect, a second type defect, and a third type defect. The first type defect represents that the defect is a scratch; the second type defect represents that the defect is an aluminum wire; the third type defect represents that there is no defect, or a defect other than a scratch or an aluminum wire.

[0053] In this embodiment, by obtaining the original image and the resized image corresponding to the original image, multiple rounds of downsampling processing and feature fusion processing can be performed based on the resized image to obtain feature fusion images at different target sizes. By obtaining feature fusion images at different target sizes, feature information of the resized image at different sizes can be determined based on the feature fusion images at different target sizes, and then the accuracy of determining the defect connected domain can be improved based on the feature information at different sizes. By determining the defect connected domain in the resized image, a defect category detection image can be segmented from the original image based on the defect connected domain in the resized image, and then defect category detection can be performed based on the defect connected domain in the resized image and the defect category detection image to obtain a defect detection result. Since defect category detection can be performed based on the defect connected domain in the resized image and the defect category detection image segmented from the original image, the connected domain information at different sizes can be fully utilized in the process of defect category detection, and then the accuracy of defect detection can be further improved based on the fully utilized information.

[0054] In addition, since the resized image is subjected to multiple rounds of downsampling and feature fusion processing to obtain feature fusion images at different target sizes, compared to directly performing multiple rounds of downsampling and feature fusion processing on the original image, when the resized image is obtained by reducing the original image, the amount of data for calculation can also be reduced, thereby improving the detection efficiency of defect detection.

[0055] In some embodiments, the above steps 204 to 208 may be performed by a segmentation model in a computer device, and step 210 may be performed by a defect classification model in a computer device. Before performing steps 204 to 208 based on the segmentation model, the segmentation model may be trained first, and before performing step 210 based on the defect classification model, the defect classification model may also be trained first. Training the segmentation model includes: annotating the acquired training sample images with defect connected domains, and training the segmentation model based on the training samples annotated with defect connected domains to obtain a trained segmentation model. Training the defect classification model includes: annotating the acquired training samples with defect connected domains and defect categories, respectively, and training the defect classification model based on the training samples annotated with defect connected domains and defect categories to obtain a trained defect classification model. It is easy to understand that the segmentation model and the defect classification model may be trained based on the same training samples, or may be trained based on different training samples.

[0056] In some embodiments, multiple rounds of downsampling processing and feature fusion processing are performed based on the resized image to obtain feature fusion images at different target sizes, including:

[0057] Downsampling the resized image according to the multiple downsampling multiples corresponding to the first round to obtain multiple downsampling images of the first round; performing feature fusion processing based on the multiple downsampling images of the first round to obtain feature fusion images of different sizes of the first round;

[0058] In the current round starting from the second round, downsampling processing is performed on at least part of the feature fusion image of the previous round according to at least one downsampling multiple corresponding to the current round to obtain a downsampled image of the current round; feature fusion processing is performed based on the downsampled image of the current round and at least part of the feature fusion image of the previous round to obtain feature fusion images of different sizes of the current round;

[0059] Based on the feature fusion images at different sizes in the last round, feature fusion images at different target sizes are determined.

[0060] For example, in the first round, the segmentation model may obtain multiple downsampling multiples corresponding to the first round, and perform downsampling processing on the resized image based on the obtained downsampling multiples to obtain multiple downsampled images of the first round. Figure 3 , when the size of the resized image is 255*192*3, the resized image can be downsampled multiple times to obtain a downsampled image of 64*48*256 with a size of 4 times and a downsampled image of 32*24*64 with a size of 8 times. Furthermore, the segmentation model can perform feature fusion processing on multiple downsampled images of the first round to obtain feature fused images of different sizes of the first round. For example, refer to Figure 3 , the downsampled image of 64*48*256 can be downsampled by 2 times, and then feature fused with the downsampled image of 32*24*64 to obtain a feature fused image of 32*24*64 in the first round; the downsampled image of 32*24*64 can be upsampled by 2 times, and then feature fused with the downsampled image of 64*48*256 to obtain a feature fused image of 64*48*256 in the first round.

[0061] In the current round starting from the second round, the segmentation model downsamples at least part of the feature fusion image of the previous round according to at least one downsampling multiple corresponding to the current round to obtain the downsampled image of the current round. For example, the segmentation model may determine the feature fusion image of the minimum size of the previous round and downsample the feature fusion image of the minimum size of the previous round. For example, referring to Figure 3, the segmentation model can downsample the feature fusion image of the first round with a size of 32*24*64 to obtain a downsampled image of 16*12*128 with a size that is 16 times the size of the resized image. The segmentation model performs feature fusion processing on the downsampled image of the current round and at least part of the feature fusion image of the previous round to obtain feature fusion images of different sizes of the current round. For example, the segmentation model can perform feature fusion processing on the feature fusion image of the previous round that has not been downsampled with the downsampled image of the current round. For example, refer to Figure 3 , the segmentation model can perform feature fusion processing on the first round of 64*48*256 feature fusion images, the first round of 32*24*64 feature fusion images, and the second round of 16*12*128 downsampled images to obtain the second round of 64*48*256 feature fusion images, 32*24*64 feature fusion images, and 16*12*128 feature fusion images. Iterate in this way until the last round of feature fusion images of different sizes are obtained.

[0062] The segmentation model may determine the feature fusion images at different target sizes based on the feature fusion images at different sizes of the last round. For example, the segmentation model may use the feature fusion images at different sizes of the last round as the feature fusion images at different target sizes. Alternatively, the segmentation model may perform some preset processing on the feature fusion images at different sizes of the last round to obtain the feature fusion images at different target sizes. Figure 3 A schematic diagram of downsampling and feature fusion in one embodiment is shown. Figure 3 The Bottle Block in the figure is the Bottle Block module in ResNet (Residual Network). BasicBlock is the residual block. The downsampled image can be passed through the BasicBlock module before feature fusion. The BN layer is the normalization layer.

[0063] In some embodiments, after obtaining the feature fusion images at different target sizes, the segmentation model can perform feature fusion processing on the feature fusion images at different target sizes to obtain a target fusion image, and determine the defect connected domain in the resized image based on the target fusion image. Figure 3 The last round of 64*48*256 feature fusion images, 32*24*64 feature fusion images, 16*12*128 feature fusion images, and 8*6*256 feature fusion images can be fused to obtain a target fusion image with a size of 64*48*256, and then the defect connected domain in the resized image can be determined based on the target fusion image.

[0064] In product defect detection, the images to be detected are usually larger than 5000x3672. The size of scratches and aluminum wires is sometimes only two or three pixels wide, with an area of ​​tens of pixels. When detecting defects in traditional solutions, the features are downsampled to less than 800x800 inside the machine learning model. Excessive downsampling causes the features at the defect to be reduced or even invisible, thereby reducing the accuracy of defect detection. In this embodiment, feature maps with multiple resolutions can be maintained, effectively capturing information at different sizes, and thus improving the accuracy of defect detection based on information at different sizes.

[0065] In some embodiments, based on the defect connected domain, segmenting the defect category detection image containing the background from the original image includes:

[0066] Get the scaling ratio between the original image and the resized image;

[0067] Based on the scaling ratio and the position information of the defect connected domain, the defect connected domain is mapped to the original image to obtain the mapping area in the original image;

[0068] Based on the mapped area, the defect category detection image including the background is segmented from the original image.

[0069] For example, since the resized image is obtained by scaling the original image, the segmentation model can determine the scaling ratio between the original image and the resized image. For example, when the original image is reduced by two times to obtain the resized image, the scaling ratio between the original image and the resized image is 2:1. The segmentation model can determine the coordinates of the defective connected domain in the resized image, and map the defective connected domain in the resized image to the original image according to the scaling ratio and the coordinates to obtain the mapping area in the original image. For example, referring to Figure 4 , when the scaling ratio of the original image to the resized image is 2:1, the defect connected domain in the resized image is a long strip, and the coordinates of the four vertices are (0pix, 0 pix), (0pix, 10pix), (5pix, 0pix), (5pix, 10pix), after mapping the defect connected domain in the resized image to the original image, the coordinates of the mapping area in the original image are (0pix, 0 pix), (0pix, 20pix), (10pix, 0pix), (10pix, 20pix). Figure 4 A schematic diagram of defect connected domain mapping in an embodiment is shown.

[0070] Once the mapping area in the original image is determined, the segmentation model can segment the defect category detection image from the original image based on the mapping area in the original image.

[0071] In this embodiment, by determining the scaling ratio and the location information of the defect connected domain, the defect connected domain can be accurately mapped to the original image, thereby accurately obtaining the mapping area in the original image, and then based on the accurately determined mapping area, the accuracy of the defect category detection image segmented from the original image is improved.

[0072] In some embodiments, segmenting a defect category detection image including a background from an original image based on the mapping area includes:

[0073] Expand the mapping area by a preset number of pixels to obtain an expanded connected domain;

[0074] When the size of the outward-expanded connected domain does not meet the connected domain size condition, the outward-expanded connected domain is expanded again to obtain an outward-expanded connected domain that meets the connected domain size condition;

[0075] Based on the externally expanded connected domain that meets the connected domain size condition, the defect category detection image including the background is segmented from the original image.

[0076] For example, the segmentation model expands the mapping area by a preset number of pixels, for example, 10 pixels, to obtain an expanded connected domain, and determines whether the size of the expanded connected domain meets the preset connected domain size condition, for example, determines whether the length and width of the expanded connected domain are both multiples of 32. If the connected domain size condition is not met, the segmentation model expands the expanded connected domain again to obtain an expanded connected domain that meets the connected domain size condition, for example, the expanded connected domain is expanded again so that the length and width of the connected domain after the expansion are both multiples of 32. The segmentation model segments a defect category detection image from the original image based on the expanded connected domain that meets the connected domain size condition. For example, the segmentation model segments the expanded connected domain that meets the connected domain size condition from the original image to obtain a defect category detection image.

[0077] In some embodiments, with reference to 5, after the computer device acquires the original image, the computer device may reduce the original image to obtain a resized image, and input the resized image into the segmentation model, determine the defect connected domain in the resized image through the segmentation model, and the segmentation model maps the defect connected domain in the resized image to the original image to obtain the mapping area in the original image, and expands the mapping area in the original image to obtain an expanded connected domain that contains not only defects but also background. The segmentation model determines whether the expanded connected domain meets the connected domain size condition. If it does, the expanded connected domain is segmented from the original image to obtain a defect category detection image, and the defect category detection image is input into the defect classification model. If it does not meet the condition, the expanded connected domain is expanded again to obtain an expanded connected domain that meets the connected domain size condition, and the expanded connected domain that meets the connected domain size condition is segmented from the original image to obtain a defect category detection image, and the defect category detection image is input into the defect classification model, and the defect classification model outputs the defect detection result. Figure 5 A schematic diagram of the overall process of defect detection in one embodiment is shown.

[0078] In this embodiment, by performing an expansion operation, the defect category detection image obtained based on the expansion operation includes not only defects but also background, and then the defect classification model can obtain some global views based on the background, and improve the accuracy of defect classification based on the obtained global views. In addition, when the expanded connected domain does not meet the connected domain size condition, the expanded connected domain will cause the loss of part of the connected domain data due to the size not meeting the requirements. By setting the connected domain size condition, the expanded connected domain that does not meet the connected domain size condition can be expanded again based on the connected domain size condition, so that the expanded connected domain after the second expansion will not cause the loss of part of the connected domain data due to the size problem.

[0079] In some embodiments, defect category detection is performed based on the defect connected domain and the defect category detection image to obtain a defect detection result corresponding to the target product, including:

[0080] Input the defect connected domain and defect category detection image into the trained defect classification model for processing, and output the target defect category;

[0081] Determine the target product area corresponding to the defect category detection image based on the position information of the defect category detection image in the original image;

[0082] Determine the target defect detection conditions based on the target defect category and target product area;

[0083] Determine defect attribute information;

[0084] Based on the defect attribute information and the target defect detection conditions, the defect detection results corresponding to the target product are determined.

[0085] For example, the defect connected domain and defect category detection image in the resized image can be input into a trained defect classification model for processing, and the target defect category can be output. The defect classification model can also be a machine learning model with HRNeT as the backbone network. The target defect category can be a first defect, a second defect, or a third defect. The first type of defect represents a defect that is a scratch; the second type of defect represents a defect that is an aluminum wire; the third type of defect represents no defect, or a defect other than a scratch or an aluminum wire.

[0086] Furthermore, the computer device can determine the position information of each product area in the original image. For example, when the original image is a cover image of a square shell power battery, the computer device can determine the position information of each product area in the original image. Figure 6 When the cover of the square shell power battery includes multiple preset product areas such as the edge, the large aluminum surface, and the coding area, the computer device can determine the position information of the edge, the position information of the large aluminum surface, and the position information of the coding area. The position information can specifically be the area range, for example, the position information of the edge can specifically be the area range of the edge. The computer device can determine the target product area corresponding to the defect category detection image based on the position information of the defect category detection image segmented from the original image and the area range of each preset product area. For example, the computer device can compare the position coordinates of the defect category detection image in the original image with the area range of each preset product area, and determine the target product area corresponding to the defect category detection image based on the comparison result, and then the computer device can determine the corresponding target defect detection condition according to the target defect category and the target product area. For example, multiple defect detection conditions are pre-set in the computer device, and different defect detection conditions correspond to different defect categories and product areas. For example, defect detection condition 1 corresponds to defect category A and product area A, and defect detection condition 2 corresponds to defect category B and product area C. Therefore, when the target product area and target defect detection conditions are determined, the target defect detection conditions corresponding to the target product area and target defect category can be determined. Figure 6 A schematic diagram of a product area in one embodiment is shown.

[0087] The computer device may determine the defect attribute information, and determine whether the target product meets the corresponding target defect detection condition based on the defect attribute information. The defect attribute information may include the confidence of the target defect category and the attribute information of the defect connected domain, and the attribute information of the defect connected domain includes area size, length, width, gray value, etc. Furthermore, the defect attribute information may include sub-information of multiple dimensions, for example, the multiple dimensions may include area dimension, length dimension, width dimension, gray value dimension, confidence dimension, etc., and the sub-information of multiple dimensions may include the area size of the defect connected domain, the length of the defect connected domain, the width of the defect connected domain, the gray value of the defect connected domain, etc. The confidence may be the confidence of the target defect category determined by the defect classification model.

[0088] The target defect detection condition may include sub-conditions in multiple dimensions. For example, the target defect detection condition may include sub-conditions in the area dimension, sub-conditions in the length dimension, sub-conditions in the width dimension, sub-conditions in the gray value dimension, and sub-conditions in the confidence dimension. The computer device may compare the sub-information and sub-conditions in the same dimension to determine whether the sub-information satisfies the sub-conditions in the corresponding dimension. For example, the sub-condition of the area size may be "whether the area is greater than 100pix", and then the computer device determines whether the area of ​​the defect connected domain is greater than 100pix based on the sub-information in the area dimension. If it is greater than 100pix, it is determined that the sub-information in the area dimension satisfies the sub-condition in the area dimension.

[0089] When a preset number of sub-conditions are met, it can be determined that the target product meets the target defect detection condition. At this time, the target defect category is used as the defect detection result corresponding to the target product. If the target product does not meet the target defect detection condition, it is determined that the target product has no product defects.

[0090] In some embodiments, when the size of the defect connected domain is obtained, the size of the defect connected domain can also be converted into the size of the mapping area. For example, the computer device can determine the scaling ratio between the original image and the resized image. After determining the size information of the defect connected domain, the size information of the mapping area can be determined based on the scaling ratio and the size information of the defect connected domain. Exemplarily, when the resized image is obtained by reducing the original image by half, if the area of ​​the defect connected domain is 50pix, the area of ​​the mapping area is 100pix. Furthermore, the computer device can compare the size of the mapping area with the corresponding sub-condition in the target defect detection condition to obtain a comparison result, and determine whether the target product meets the target defect detection condition based on the comparison result.

[0091] In some embodiments, the safety risks caused by defects appearing in different positions are different. For example, aluminum wires appearing at the edge of the square shell power battery cover plate will bring the risk of road disorder. Therefore, different detection conditions can be pre-set for each preset product area. For example, since defects appearing in the edge area will bring greater safety risks, more stringent defect detection conditions can be set for the edge area. For example, it can be set that when the area of ​​the defect connected domain corresponding to the edge area reaches 100pix, it is determined that the sub-condition of the area dimension in the defect detection condition is met; since the safety risks caused by defects appearing on large aluminum surfaces are relatively small, looser defect detection conditions can be set for large aluminum surfaces. For example, it can be set that when the area of ​​the defect connected domain corresponding to the large aluminum surface reaches 200pix, it is determined that the sub-condition of the area dimension in the defect detection condition is met.

[0092] In some embodiments, before determining the target defect category through the defect classification model, it is also possible to determine whether defect category detection is required based on the attribute information of the defect connected domain. For example, it can be set to perform defect category detection when the area of ​​the defect connected domain corresponding to the large aluminum surface reaches 200pix. In this way, some unnecessary defects can be detected without defect category detection, thereby saving computer resources consumed during defect detection.

[0093] In this embodiment, by setting different defect detection conditions for different defect categories and product areas, defective products can be selectively determined based on the defect detection conditions. For some products that only have minor defects and do not affect their use, there is no need to define them as defective products. This improves the first-time quality rate of product production and reduces product production costs.

[0094] In some embodiments, before performing defect category detection based on the defect connected domain and the defect category detection image to obtain a defect detection result corresponding to the target product, the method further includes:

[0095] Acquire a training sample image; the training sample image is annotated with a defect category label of a defect object; the defect object includes at least a first defect object and a second defect object; the defect detection importance of the first defect object is higher than the defect detection importance of the second defect object;

[0096] Acquire a concentrated occurrence area of ​​the first defect object;

[0097] For the training sample image, the defect category label of the second defect object in the concentrated appearance area is modified to the defect category label of the first defect object;

[0098] Based on the training sample images after the defect category labels are modified, an untrained defect classification model is trained to obtain a trained defect classification model; the trained defect classification model is used for product defect category detection.

[0099] For example, a computer device may obtain a training sample image, wherein the training sample image is annotated with a defect category label of a defect object, for example, the aluminum wire in the training sample is labeled and the scratch is also labeled. The defect objects in the training sample include at least a first defect object and a second defect object, and the importance of defect detection of the first defect object is higher than the importance of defect detection of the second defect object. For example, the first defect object may be an aluminum wire, and the second defect object may be a scratch. For a square shell power battery cover, the safety risk brought by the aluminum wire is greater than the safety risk brought by the scratch. Therefore, the importance of defect detection of the aluminum wire is higher than the importance of defect detection of the scratch.

[0100] The computer device can obtain the concentrated appearance area of ​​the first defect detection object, and modify the defect category label of the second defect object in the concentrated appearance area in the training sample to the defect category label of the first defect object. For example, when aluminum wire mostly appears at the edge of the cover plate and around the explosion-proof valve, the defect classification label of the scratches around the cover plate and the explosion-proof valve can be modified to the defect classification label of aluminum wire. Then, the computer device trains the untrained defect classification model based on the training sample with the modified defect classification label to obtain the trained defect classification model.

[0101] In this embodiment, since the importance of defect detection for the first defect object is higher than that for the second defect object, the defect category label of the second defect object in the concentrated appearance area is modified to the defect category label of the first defect object, so that the defect classification model trained based on the training samples with the modified defect category label can detect all the first defect objects when performing defect category detection, thereby reducing the probability of missing important defects.

[0102] In some embodiments, before obtaining the concentrated occurrence area of ​​the first defect object, the method further includes:

[0103] Acquire multiple annotated images; annotate the positions of the first defective objects in the annotated images;

[0104] According to the position of the first defect object, each of the annotated images is converted into a mask image; the pixel value of the pixel point at the position of the first defect object in the mask image is different from the pixel values ​​of the pixel points at other positions;

[0105] Accumulate the pixel values ​​at the same position in each mask image to obtain an accumulation map;

[0106] Based on the cumulative graph, a concentrated occurrence area of ​​the first defect object is determined.

[0107] For example, a computer device may obtain multiple annotated images, each of which is annotated with the location of the first defect object. For example, when the first defect object is aluminum wire, the aluminum wire in each annotated image may be selected. The computer device sets the pixel value of the pixel point at the location of the first defect object in the annotated image to a first preset pixel value, and sets the pixel value of the pixel point at other locations to a second preset pixel value, so as to obtain a mask image of the annotated image. The first preset pixel value is different from the second preset pixel value.

[0108] The computer device accumulates the pixel values ​​at the same position in each mask image to obtain an accumulated image. For example, the computer device superimposes the pixel values ​​of the first pixel in each mask image to obtain the pixel value of the first pixel in the accumulated image, superimposes the pixel values ​​of the second pixel in each mask image to obtain the pixel value of the second pixel in the accumulated image, and so on. The computer device normalizes the pixel values ​​of each pixel in the accumulated image to obtain a defect thermal distribution map, and then determines the concentrated appearance area of ​​the first defect object based on the defect thermal distribution map. For example, the darkest area in the defect thermal distribution map is the concentrated appearance area of ​​the first defect object.

[0109] In some embodiments, before converting each annotated image into a mask image, each annotated image may be aligned to obtain an aligned annotated image. For example, the sizes of each annotated image may be unified, or the sizes of products in the annotated images may be unified.

[0110] In this embodiment, by generating a defect distribution heat map, the concentrated occurrence area of ​​the first defect object can be accurately determined based on the defect distribution heat map, thereby improving the accuracy of the concentrated occurrence area.

[0111] In some embodiments, reference Figure 7 , Figure 7The overall process diagram of defect detection in another embodiment is shown. The computer device obtains the original image, resizes the original image, obtains the resized image, and determines the defect connected domain in the resized image and the defect category detection image in the original image based on the segmentation model. The computer device inputs the defect connected domain in the resized image and the defect category detection image in the original image into the defect classification model, determines the defect detection condition corresponding to the defect category detection image, and determines whether the defect category detection image meets the defect detection condition. If it meets the condition, the defect category detection is performed. Before performing image processing based on the segmentation model and the defect classification model, the segmentation model and the defect classification model can also be trained first, the segmentation model training data can be obtained, and the segmentation model can be trained based on the segmentation model training data. The segmentation model training data can be analyzed, and the defect category label in the segmentation model training data can be modified based on the analysis result to obtain the defect classification model training data, and the defect classification model is trained based on the defect classification model training data.

[0112] In this embodiment, the segmentation model is used for the preliminary positioning of defects, which can better accommodate changes in image brightness and does not require interference from unstable motion mechanisms during shooting, thus achieving the purpose of better adaptability to complex environments. Strip defects are subdivided into scratches and aluminum wires, which are closer to industrial production needs, and different standards can be used for detection after subdivision, which improves the first-time quality rate of product production and reduces product production costs. The connected domain in the resized image and the defect category detection image in the original image are input into the classification model together. Compared with directly using the segmentation model to classify scratched aluminum wires or using the connected domain features for classification, it effectively reduces misjudgments and missed detections and improves the overall detection capability of the equipment.

[0113] This embodiment can be applied in the industrial field and customized for the special needs of industrial production. For example, configurations such as defect detection conditions are added, and different detection standards are adopted for different defects in different areas to better meet the requirements of practical applications. In addition, this embodiment realizes high-precision defect detection in complex scenarios, and is versatile and robust, and can achieve better results in actual production. It can be widely used in the production and manufacturing of new energy batteries, quality control and other fields that require high-precision defect detection.

[0114] It should be understood that, although the steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0115] Based on the same inventive concept, the embodiment of the present application also provides a product defect detection device. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more product defect detection device embodiments provided below can refer to the limitations of the product defect detection method above, and will not be repeated here.

[0116] like Figure 8 As shown, the embodiment of the present application provides a product defect detection device 800, including:

[0117] An image acquisition module 802 is used to acquire an original image and a resized image of a target product, where the resized image is obtained by reducing the size of the original image;

[0118] A feature fusion module 804 is used to perform multiple rounds of downsampling processing and feature fusion processing based on the resized image to obtain feature fusion images at different target sizes;

[0119] A connected domain determination module 806 is used to determine a defect connected domain in the resized image based on the feature fusion images at different target sizes;

[0120] An image segmentation module 808 is used to segment a defect category detection image including a background from an original image based on a defect connected domain;

[0121] The category detection module 810 is used to perform defect category detection based on the defect connected domain and the defect category detection image to obtain a defect detection result corresponding to the target product.

[0122] In some embodiments, in performing multiple rounds of downsampling processing and feature fusion processing based on the resized image to obtain feature fusion images at different target sizes, the feature fusion module 804 is specifically used to:

[0123] Downsampling the resized image according to the multiple downsampling multiples corresponding to the first round to obtain multiple downsampling images of the first round; performing feature fusion processing based on the multiple downsampling images of the first round to obtain feature fusion images of different sizes of the first round;

[0124] In the current round starting from the second round, downsampling processing is performed on at least part of the feature fusion image of the previous round according to at least one downsampling multiple corresponding to the current round to obtain a downsampled image of the current round; feature fusion processing is performed based on the downsampled image of the current round and at least part of the feature fusion image of the previous round to obtain feature fusion images of different sizes of the current round;

[0125] Based on the feature fusion images at different sizes in the last round, feature fusion images at different target sizes are determined.

[0126] In some embodiments, in terms of segmenting a defect category detection image including a background from an original image based on a defect connected domain, the image segmentation module 808 is specifically configured to:

[0127] Get the scaling ratio between the original image and the resized image;

[0128] Based on the scaling ratio and the position information of the defect connected domain, the defect connected domain is mapped to the original image to obtain the mapping area in the original image;

[0129] Based on the mapped area, the defect category detection image including the background is segmented from the original image.

[0130] In some embodiments, in terms of segmenting the defect category detection image including the background from the original image based on the mapping area, the image segmentation module 808 is specifically used to:

[0131] Expand the mapping area by a preset number of pixels to obtain an expanded connected domain;

[0132] When the size of the outward-expanded connected domain does not meet the connected domain size condition, the outward-expanded connected domain is expanded again to obtain an outward-expanded connected domain that meets the connected domain size condition;

[0133] Based on the externally expanded connected domain that meets the connected domain size condition, the defect category detection image including the background is segmented from the original image.

[0134] In some embodiments, in performing defect category detection based on the defect connected domain and the defect category detection image to obtain a defect detection result corresponding to the target product, the category detection module 810 is specifically used to:

[0135] Input the defect connected domain and defect category detection image into the trained defect classification model for processing, and output the target defect category;

[0136] Determine the target product area corresponding to the defect category detection image based on the position information of the defect category detection image in the original image;

[0137] Determine the target defect detection conditions based on the target defect category and target product area;

[0138] Determine defect attribute information;

[0139] Based on the defect attribute information and the target defect detection conditions, the defect detection results corresponding to the target product are determined.

[0140] In some embodiments, the product defect detection device 800 further includes a model training module 812, which is used to:

[0141] Acquire a training sample image; the training sample image is annotated with a defect category label of a defect object; the defect object includes at least a first defect object and a second defect object; the defect detection importance of the first defect object is higher than the defect detection importance of the second defect object;

[0142] Acquire a concentrated occurrence area of ​​the first defect object;

[0143] For the training sample image, the defect category label of the second defect object in the concentrated appearance area is modified to the defect category label of the first defect object;

[0144] Based on the training sample images after the defect category labels are modified, an untrained defect classification model is trained to obtain a trained defect classification model.

[0145] In some embodiments, the model training module 812 is further configured to:

[0146] Acquire multiple annotated images; annotate the positions of the first defective objects in the annotated images;

[0147] According to the position of the first defect object, each of the annotated images is converted into a mask image; the pixel value of the pixel point at the position of the first defect object in the mask image is different from the pixel values ​​of the pixel points at other positions;

[0148] Accumulate the pixel values ​​at the same position in each mask image to obtain an accumulation map;

[0149] Based on the cumulative graph, a concentrated occurrence area of ​​the first defect object is determined.

[0150] Each module in the above-mentioned product defect detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0151] In some embodiments, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, the steps in the above-mentioned product defect detection method are implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0152] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0153] In some embodiments, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0154] In some embodiments, a computer-readable storage medium 1000 is provided on which a computer program 1002 is stored. When the computer program 1002 is executed by a processor, the steps in the above-mentioned method embodiments are implemented. The internal structure diagram thereof can be as follows: Fig.10 shown.

[0155] In some embodiments, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0156] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0157] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0158] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0159] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A product defect detection method, characterized in that: include: Acquire an original image and a resized image of a target product, wherein the resized image is obtained by reducing the size of the original image; Performing multiple rounds of downsampling processing and feature fusion processing based on the resized image to obtain feature fusion images at different target sizes; Determining a defect connected domain in the resized image based on the feature fused images at different target sizes; Based on the defect connected domain, segmenting a defect category detection image including a background from the original image; Defect category detection is performed based on the defect connected domain and the defect category detection image to obtain a defect detection result corresponding to the target product.

2. The method according to claim 1, characterized in that The step of performing multiple rounds of downsampling processing and feature fusion processing based on the resized image to obtain feature fusion images at different target sizes includes: Downsampling the resized image according to multiple downsampling multiples corresponding to the first round to obtain multiple downsampled images of the first round; performing feature fusion processing based on the multiple downsampled images of the first round to obtain feature fusion images of different sizes of the first round; In the current round starting from the second round, downsampling processing is performed on at least part of the feature fusion image of the previous round according to at least one downsampling multiple corresponding to the current round to obtain a downsampled image of the current round; feature fusion processing is performed based on the downsampled image of the current round and at least part of the feature fusion image of the previous round to obtain feature fusion images of different sizes of the current round; Based on the feature fusion images at different sizes in the last round, feature fusion images at different target sizes are determined.

3. The method according to claim 1, characterized in that The step of segmenting a defect category detection image including a background from the original image based on the defect connected domain comprises: Obtaining a scaling ratio between the original image and the resized image; Based on the scaling ratio and the position information of the defect connected domain, mapping the defect connected domain to the original image to obtain a mapping area in the original image; Based on the mapping area, a defect category detection image including a background is segmented from the original image.

4. The method according to claim 3, characterized in that The step of segmenting a defect category detection image including a background from the original image based on the mapping area includes: Expanding the mapping area outward by a preset number of pixels to obtain an outward-expanded connected domain; When the size of the outward-expanded connected domain does not meet the connected domain size condition, the outward-expanded connected domain is expanded again to obtain an outward-expanded connected domain that meets the connected domain size condition; Based on the outward-expanded connected domain that satisfies the connected domain size condition, a defect category detection image including a background is segmented from the original image.

5. The method according to claim 1, characterized in that: The defect category detection is performed based on the defect connected domain and the defect category detection image to obtain a defect detection result corresponding to the target product, including: Input the defect connected domain and the defect category detection image into a trained defect classification model for processing, and output a target defect category; Determine a target product area corresponding to the defect category detection image based on position information of the defect category detection image in the original image; Determining a target defect detection condition based on the target defect category and the target product area; Determine defect attribute information; Based on the defect attribute information and the target defect detection condition, a defect detection result corresponding to the target product is determined.

6. The method according to claim 1, characterized in that Before performing defect category detection based on the defect connected domain and the defect category detection image to obtain a defect detection result corresponding to the target product, the method further includes: Acquire a training sample image; the training sample image is annotated with a defect category label of a defect object; the defect object includes at least a first defect object and a second defect object; the defect detection importance of the first defect object is higher than the defect detection importance of the second defect object; Acquire a concentrated occurrence area of ​​the first defect object; For the training sample image, modify the defect category label of the second defect object in the concentrated appearance area to the defect category label of the first defect object; Based on the training sample images after the defect category labels are modified, an untrained defect classification model is trained to obtain a trained defect classification model.

7. The method according to claim 6, characterized in that Before obtaining the concentrated occurrence area of ​​the first defect object, the method further includes: Acquire a plurality of annotated images; the annotated images are annotated with the positions of the first defect objects; According to the position of the first defect object, each of the annotated images is converted into a mask image; the pixel value of the pixel point at the position of the first defect object in the mask image is different from the pixel values ​​of the pixel points at other positions; Accumulating pixel values ​​at the same position in each of the mask images to obtain an accumulation image; Based on the accumulated graph, a concentrated occurrence area of ​​the first defect object is determined.

8. A product defect detection device, characterized in that: include: An image acquisition module, used to acquire an original image and a resized image of a target product, wherein the resized image is obtained by reducing the size of the original image; A feature fusion module, used for performing multiple rounds of downsampling processing and feature fusion processing based on the resized image to obtain feature fusion images at different target sizes; A connected domain determining module, used for determining a defect connected domain in the resized image based on the feature fusion images at different target sizes; An image segmentation module, used for segmenting a defect category detection image including a background from the original image based on the defect connected domain; The category detection module is used to perform defect category detection based on the defect connected domain and the defect category detection image to obtain a defect detection result corresponding to the target product.

9. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images