Defect detection method, device and equipment and storage medium

Abstract

The invention provides a defect detection method and device, equipment and a storage medium, a to-be-detected product image is obtained, the to-be-detected product image comprises an image of an auxiliary material of the to-be-detected product, a gray threshold corresponding to each auxiliary material is determined according to color information of the auxiliary material of the to-be-detected product, and on the basis of the gray threshold, a defect detection result is obtained. Determining a reference image area corresponding to each auxiliary material in the to-be-detected product image and a to-be-detected image area where each auxiliary material is currently located, determining a first geometric feature of the reference image area, determining a second geometric feature of the to-be-detected image area, and determining a feature difference between the first geometric feature and the second geometric feature, and determining whether the to-be-detected product has the target defect according to the characteristic difference. By adopting the method, whether the deviation defect exists in the auxiliary material or not can be accurately judged, and the product quality is ensured.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a defect detection method, device, equipment and storage medium. Background Art

[0002] With the continuous development of the manufacturing industry, product quality assurance has become one of the key factors in measuring product advantages. Whether the installation position of auxiliary materials such as shrapnel and film on the product board is accurate is crucial to product quality. If the auxiliary materials are misaligned, it may affect the performance, appearance and function of the product, leading to quality problems. Therefore, how to accurately detect misalignment defects in products has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present disclosure provides a defect detection method, device, equipment and storage medium to at least solve the above technical problems existing in the prior art.

[0004] According to a first aspect of the present disclosure, a defect detection method is provided, the method comprising:

[0005] Acquire an image of a product to be inspected, wherein the image of the product to be inspected includes an image of an auxiliary material of the product to be inspected;

[0006] Determine the grayscale threshold corresponding to each auxiliary material according to the color information of the auxiliary materials of the product to be detected;

[0007] Based on the grayscale threshold, determining the reference image area corresponding to each auxiliary material in the image of the product to be detected and the image area to be detected where each auxiliary material is currently located;

[0008] Determining a first geometric feature of the reference image area, and determining a second geometric feature of the image area to be measured;

[0009] A feature difference between the first geometric feature and the second geometric feature is determined, and whether the product to be inspected has a target defect is determined based on the feature difference.

[0010] In one possible implementation manner, the step of acquiring an image of a product to be inspected includes:

[0011] Collect color images of the products to be inspected;

[0012] Extracting a first color space of the color image, and converting the first color space of the color image into a second color space based on a preset conversion relationship, the first color space and the second color space being color spaces formed by different image channels of the color image;

[0013] For each image channel in the second color space, based on a pixel lower threshold and a pixel upper threshold corresponding to the image channel, a target mask corresponding to the image channel is determined, wherein the target mask is used to extract an image area in the color image that is consistent with the color of the auxiliary material;

[0014] Based on the target mask, the pixel value of the image area where the auxiliary material is located in the color image is set to the first pixel value, and the pixel value of the remaining area is set to the second pixel value to obtain the image of the product to be detected.

[0015] In one possible implementation manner, the step of acquiring an image of a product to be inspected includes:

[0016] Collect color images of the products to be inspected;

[0017] extracting a first color space of the color image;

[0018] Based on the second conversion relationship, the first color space of the color image is converted into a grayscale space to obtain an image of the product to be detected.

[0019] In one possible implementation manner, determining the grayscale threshold corresponding to each auxiliary material according to the color information of the auxiliary material of the product to be detected includes:

[0020] Dividing the image of the product to be inspected into regions based on the position of the auxiliary materials to obtain a plurality of sub-image regions;

[0021] Calculating the average grayscale value of each sub-image area;

[0022] Based on a first correspondence between grayscale values ​​and colors, determining color information of the sub-image area corresponding to the average grayscale value;

[0023] A grayscale threshold corresponding to the sub-image area is determined according to the second corresponding relationship between color and grayscale and the color information.

[0024] In one possible implementation, determining, based on the grayscale threshold, the reference image region corresponding to each auxiliary material in the image of the product to be detected and the image region to be detected where each auxiliary material is currently located, includes:

[0025] Determine, according to the grayscale threshold, a plurality of connected regions corresponding to the auxiliary materials in the image of the product to be detected;

[0026] Merging multiple connected regions to obtain multiple merged image regions;

[0027] Based on the attribute information of each merged image area, the merged image area currently corresponding to the auxiliary material is determined as the image area to be tested, and the merged image area to which the auxiliary material needs to be configured is determined as the reference image area.

[0028] In one possible implementation manner, determining the multiple connected regions corresponding to the auxiliary materials in the image of the product to be detected according to the grayscale threshold comprises:

[0029] For each pixel in the image of the product to be inspected, an area formed by the pixel and a target adjacent pixel of the pixel is determined as a pixel connected area, where the target adjacent pixel refers to an adjacent pixel whose pixel difference with the pixel is less than a grayscale threshold among the adjacent pixels of the pixel;

[0030] The connected regions of each pixel point that can be connected are merged to obtain multiple connected regions.

[0031] In one possible implementation manner, the merging of multiple connected regions includes:

[0032] Determine the average gray value corresponding to each connected area;

[0033] Calculate the difference in average gray values ​​between each connected area;

[0034] The connected areas whose difference is less than the preset grayscale difference are merged.

[0035] In one possible implementation, determining, based on the grayscale threshold, the reference image region corresponding to each auxiliary material in the image of the product to be detected and the image region to be detected where each auxiliary material is currently located, includes:

[0036] Input the grayscale threshold and the image of the product to be detected into a pre-trained image segmentation model to obtain a reference image area corresponding to each auxiliary material in the image of the product to be detected and an image area to be detected where each auxiliary material is currently located;

[0037] The image segmentation model is obtained by training based on the sample image and the identification area corresponding to the sample image.

[0038] In one embodiment, the first geometric feature includes at least one of the following features:

[0039] The coordinates of a first center point of the minimum circumscribed rectangle of the reference image area;

[0040] a first offset angle of the minimum circumscribed rectangle of the reference image area;

[0041] a first area of ​​the reference image region;

[0042] The second geometric feature includes at least one of the following features:

[0043] The coordinates of the second center point of the minimum circumscribed rectangle of the image area to be measured;

[0044] A second offset angle of the minimum circumscribed rectangle of the image area to be measured;

[0045] The second area of ​​the image region to be measured.

[0046] In one embodiment, the method further comprises:

[0047] Extracting a first edge line segment of the reference image area, and extracting a second edge line segment of the image area to be measured;

[0048] The determining a feature difference between the first geometric feature and the second geometric feature comprises:

[0049] determining an absolute value of a difference between an inclination angle of the first edge segment and an inclination angle of the second edge segment as a feature difference; or,

[0050] A difference between a maximum distance and a minimum distance between the first edge line segment and the second edge line segment is calculated as a feature difference.

[0051] In one possible implementation manner, determining whether the product to be inspected has a target defect according to the feature difference includes:

[0052] If the characteristic difference is greater than a preset difference threshold, it is determined that the product to be tested has an auxiliary material misalignment defect.

[0053] According to a second aspect of the present disclosure, a defect detection device is provided, the device comprising:

[0054] An image acquisition module, used to acquire an image of the product to be detected, wherein the image of the product to be detected includes an image of an auxiliary material of the product to be detected;

[0055] A grayscale threshold determination module, used to determine the grayscale threshold corresponding to each auxiliary material according to the color information of the auxiliary materials of the product to be detected;

[0056] An image region determination module, used to determine, based on the grayscale threshold, a reference image region corresponding to each auxiliary material in the image of the product to be detected and an image region to be detected where each auxiliary material is currently located;

[0057] A feature extraction module, used to determine a first geometric feature of the reference image area, and to determine a second geometric feature of the image area to be measured;

[0058] The defect detection module is used to determine the feature difference between the first geometric feature and the second geometric feature, and determine whether the product to be inspected has a target defect based on the feature difference.

[0059] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0060] at least one processor; and

[0061] a memory communicatively connected to the at least one processor; wherein,

[0062] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.

[0063] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure.

[0064] The defect detection method, device, equipment and storage medium disclosed in the present invention obtain an image of a product to be detected, the image of the product to be detected includes an image of an auxiliary material of the product to be detected, the grayscale threshold corresponding to each auxiliary material is determined according to the color information of the auxiliary material of the product to be detected, based on the grayscale threshold, the reference image area corresponding to each auxiliary material in the image of the product to be detected and the image area to be detected where each auxiliary material is currently located are determined, the first geometric feature of the reference image area is determined, and the second geometric feature of the image area to be detected is determined, the feature difference between the first geometric feature and the second geometric feature is determined, and whether the product to be detected has a target defect is determined according to the feature difference. That is, the grayscale threshold is determined by the color information of the auxiliary material, the reference image area and the image area to be detected where the auxiliary material is currently located are accurately determined based on the grayscale threshold, and then by comparing the difference in geometric features between the image area to be detected where the auxiliary material is currently located and the reference image area, it can be accurately determined whether the auxiliary material has a misalignment defect, thereby ensuring product quality.

[0065] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0067] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0068] Figure 1 A schematic diagram of a process of a defect detection method provided by an embodiment of the present disclosure is shown;

[0069] Figure 2A schematic diagram of an image region flow provided by an embodiment of the present disclosure is shown;

[0070] Figure 3 A schematic diagram of a minimum circumscribed rectangle of an image area provided by an embodiment of the present disclosure is shown;

[0071] Figure 4 A schematic diagram of a reference image area and an image area to be tested provided by an embodiment of the present disclosure is shown;

[0072] Figure 5 A schematic diagram of an edge segment provided by an embodiment of the present disclosure is shown;

[0073] Figure 6 A schematic diagram of comparing images of products to be inspected provided by an embodiment of the present disclosure is shown;

[0074] Figure 7 A schematic diagram of the structure of a defect detection device provided by an embodiment of the present disclosure is shown;

[0075] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0076] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0077] Since the installation position of auxiliary materials such as shrapnel and film on the product mid-board is critical to product quality, if the auxiliary materials are misaligned, it may affect the performance, appearance and function of the product, resulting in product quality problems. Therefore, in order to accurately detect misalignment defects of products, the present disclosure provides a defect detection method, device, equipment and storage medium. The defect detection method provided by the present disclosure can be applied to any electronic device that can perform image processing, including but not limited to computers, mobile phones and tablet computers.

[0078] The technical solution of the embodiment of the present disclosure will be described below in conjunction with the drawings in the embodiment of the present disclosure.

[0079] Figure 1 A schematic diagram of a defect detection method provided by an embodiment of the present disclosure is shown. Figure 1 As shown, the method includes:

[0080] S101, obtaining an image of a product to be inspected, wherein the image of the product to be inspected includes an image of an auxiliary material of the product to be inspected.

[0081] In the present disclosure, the product to be tested may be an electronic product such as a mobile phone and a tablet computer. The auxiliary materials of the product to be tested include but are not limited to adhesives, protective coatings and insulating materials, among which adhesives can be used to fix product components, protective coatings can protect products and components from corrosion and oxidation, and insulating materials can protect product circuits.

[0082] In the present disclosure, an image acquisition device may be used to pre-acquire images of auxiliary materials of the product to be inspected, and the acquired images may be stored. When defect detection is required, the stored images may be loaded offline. Alternatively, in the present disclosure, an image acquisition device may be used to acquire images of auxiliary materials of the product to be inspected in real time during the assembly process of the product to be inspected. In the present disclosure, the acquired images may be pre-processed, and the pre-processed images may be used as images of the product to be inspected; or, the acquired images may be converted into grayscale images, and the converted grayscale images may be used as images of the product to be inspected. The image acquisition device may be a device such as a video camera or a mobile phone camera.

[0083] S102: Determine the grayscale threshold corresponding to each auxiliary material according to the color information of the auxiliary materials of the product to be detected.

[0084] In the present disclosure, the colors of different auxiliary materials of the product to be detected may be different, and there are differences in the grayscale value ranges of auxiliary materials of different colors. For example, under the same light source parameters, the grayscale value of dark auxiliary materials is relatively low, while the grayscale value of light auxiliary materials is relatively high. Therefore, in order to accurately extract the area related to the auxiliary materials, the color information of the auxiliary materials can be combined to set corresponding grayscale thresholds for different auxiliary materials. In the present disclosure, the auxiliary material color can be determined by selecting the auxiliary material area and calculating the average grayscale of the auxiliary material area, and then determining the appropriate grayscale threshold according to the auxiliary material color. Specifically, determining the grayscale threshold corresponding to each auxiliary material according to the color information of the auxiliary material of the product to be detected can include steps A1-A4:

[0085] Step A1, dividing the image of the product to be inspected into regions based on the position of the auxiliary materials to obtain a plurality of sub-image regions.

[0086] In the present disclosure, the standard area where the auxiliary materials are located can be determined according to the design drawing of the product to be inspected, and then the image of the product to be inspected can be divided into multiple sub-image areas. Alternatively, in the present disclosure, the image of the product to be inspected can be identified, and the area range where the auxiliary materials are located can be identified, so as to divide the image of the product to be inspected into multiple sub-image areas.

[0087] Step A2, calculating the average grayscale value of each sub-image area.

[0088] For example, the average grayscale value of each sub-image area may be calculated with reference to the following formula:

[0089] Gray i =sum(I) / n

[0090] Among them, Gray i is the average grayscale value of the ith sub-image area, I is the pixel value of the pixel in the ith sub-image area, and n is the number of pixels in the ith sub-image area.

[0091] Step A3: determining the color information of the sub-image area corresponding to the average gray value based on the first corresponding relationship between gray value and color.

[0092] In the present disclosure, auxiliary materials of different colors correspond to different grayscale ranges. For example, yellow auxiliary materials generally correspond to a grayscale range of 100-200, and green auxiliary materials generally correspond to a grayscale range of 50-150. The correspondence between grayscale values ​​and colors is the first correspondence, and the color of the auxiliary material in the sub-image area can be determined based on the first correspondence between grayscale values ​​and colors and the average grayscale value of the sub-image area.

[0093] Step A4: determining a grayscale threshold corresponding to the sub-image area according to the first corresponding relationship between color and grayscale and the color information.

[0094] In the present disclosure, different grayscale thresholds can be set for different colors. For example, the grayscale threshold corresponding to the yellow auxiliary material can be set to 150-200, and the grayscale threshold corresponding to the green auxiliary material can be set to 50-100, etc. After determining the color corresponding to the sub-image area, the grayscale threshold corresponding to the color can be determined as the grayscale threshold corresponding to the sub-image area.

[0095] S103: Based on the grayscale threshold, determine the reference image area corresponding to each auxiliary material in the image of the product to be detected and the image area to be detected where each auxiliary material is currently located.

[0096] In the present disclosure, the reference image area corresponding to each auxiliary material in the image of the product to be inspected refers to a reference area used for comparison and calibration. The reference image area may include a standard area where auxiliary materials should be configured or a reference area used to define what is qualified and unqualified.

[0097] In the present disclosure, the image region to be tested where the auxiliary material is currently located refers to the region where the auxiliary material is actually configured in the product to be tested.

[0098] S104, determining a first geometric feature of the reference image region, and determining a second geometric feature of the image region to be measured.

[0099] In the present disclosure, the first geometric feature may include at least one of the following features:

[0100] The coordinates of a first center point of the minimum circumscribed rectangle of the reference image area;

[0101] a first offset angle of the minimum circumscribed rectangle of the reference image area;

[0102] A first area of ​​the reference image region.

[0103] The second geometric feature may include at least one of the following features:

[0104] The coordinates of the second center point of the minimum circumscribed rectangle of the image area to be measured;

[0105] A second offset angle of the minimum circumscribed rectangle of the image area to be measured;

[0106] The second area of ​​the image region to be measured.

[0107] The offset angle refers to the angle between the side of the rectangle and the horizontal line.

[0108] S105, determining a feature difference between the first geometric feature and the second geometric feature, and determining whether the product to be inspected has a target defect based on the feature difference.

[0109] In the present disclosure, the target defect may be a misaligned configuration defect of an auxiliary material or a missing defect of an auxiliary material, etc.

[0110] In the present disclosure, if the feature difference is greater than the preset difference threshold, it is determined that the product to be inspected has an auxiliary material deviation defect. The preset difference threshold can be set according to the type of geometric feature and actual application requirements. For example, if the geometric feature is an offset angle, the preset difference threshold is a preset angle threshold, that is, if the offset angle between the reference image area and the image area to be tested is greater than the preset angle threshold, it can be determined that the product to be inspected has a target defect. Specifically, the preset difference threshold can be set to 30° or 45°, etc.

[0111] By adopting this method, an image of a product to be inspected is obtained, the image of the product to be inspected includes an image of the auxiliary materials of the product to be inspected, the grayscale threshold corresponding to each auxiliary material is determined according to the color information of the auxiliary materials of the product to be inspected, the reference image area corresponding to each auxiliary material in the image of the product to be inspected and the image area to be inspected where each auxiliary material is currently located are determined based on the grayscale threshold, the first geometric feature of the reference image area is determined, and the second geometric feature of the image area to be inspected is determined, the feature difference between the first geometric feature and the second geometric feature is determined, and whether the product to be inspected has a target defect is determined based on the feature difference. That is, the grayscale threshold is determined by the color information of the auxiliary material, the reference image area and the image area to be inspected where the auxiliary material is currently located are accurately determined based on the grayscale threshold, and then by comparing the difference in geometric features between the image area to be inspected where the auxiliary material is currently located and the reference image area, it can be accurately judged whether the auxiliary material has a displacement defect, thereby ensuring product quality.

[0112] In a possible implementation manner, the step of acquiring the image of the product to be inspected may include steps B1-B4:

[0113] Step B1, collecting a color image of the product to be inspected.

[0114] In the present disclosure, an image acquisition device such as a mobile phone camera or a digital camera may be used to acquire a color image of the product to be inspected.

[0115] Step B2, extracting the first color space of the color image, and converting the first color space of the color image into a second color space based on a preset conversion relationship, the first color space and the second color space being color spaces formed by different image channels of the color image.

[0116] In the present disclosure, the first color space may be an RGB (red, green, and blue) color space of an image, and the second color space may be an HSV (hue, saturation, and value) color space of an image.

[0117] In the present disclosure, the pixel values ​​in the RGB color space of the color image of the product to be detected may be normalized. Specifically, the pixel values ​​in the RGB color space of the color image may be mapped to the range of [0, 1] using the following formula:

[0118] Rn=R / 255;

[0119] Gn = G / 255;

[0120] Bn=B / 255

[0121] Among them, Rn is the normalized pixel value corresponding to the R channel of the color image, Gn is the normalized pixel value corresponding to the G channel of the color image, Bn is the normalized pixel value corresponding to the B channel of the color image, and R, G and B are the pixel values ​​corresponding to the R channel, G channel and B channel of the RGB color space of the color image respectively.

[0122] The difference between the maximum and minimum pixel values ​​obtained after mapping is calculated Δ=Cmax-Cmin, where Cmax=max(Rn, Gn, Bn) and Cmin=min(Rn, Gn, Bn). Then, the pixel values ​​in the RGB color space of the image are converted to the pixel values ​​in the HSV color space based on the preset conversion relationship. The preset conversion relationship may include the following conversion formula:

[0123]

[0124] V=Cmax

[0125] Step B3, for each image channel in the second color space, based on the pixel lower threshold and pixel upper threshold corresponding to the image channel, determine the target mask corresponding to the image channel, and the target mask is used to extract the image area consistent with the color of the auxiliary material in the color image.

[0126] In the present disclosure, the pixel lower threshold and pixel upper threshold corresponding to each image channel in the second color space can be determined according to the color of the auxiliary material. Specifically, the second color space can be the HSV color space of the image, wherein the pixel value ranges of the H channel corresponding to different colors are different. For example, if the auxiliary material color is orange, the pixel lower threshold and pixel upper threshold of the H channel in the HSV color space can be set to 10 and 25, that is, the pixel points with pixel values ​​in the range of [10,25] on the H channel appear orange. If the auxiliary material color is yellow, the pixel lower threshold and pixel upper threshold of the H channel in the HSV color space can be set to 25 and 35, that is, the pixel points with pixel values ​​in the range of [25,35] on the H channel appear yellow. The S channel of the HSV color space represents the saturation of the image. Usually, the lower pixel threshold of the S channel can be set to 100 or 110, and the upper pixel threshold of the S channel can be set to 255; the V channel of the HSV color space represents the brightness of the image. Usually, the lower pixel threshold of the V channel can be set to 100 or 120, and the upper pixel threshold of the V channel can be set to 255.

[0127] The pixel lower limit threshold and pixel upper limit threshold corresponding to each channel in the HSV color space are used as the minimum pixel value and maximum pixel value of the target mask corresponding to the image channel to obtain the target mask corresponding to each image channel. Since the target mask is determined based on the pixel lower limit threshold and pixel upper limit threshold corresponding to each channel in the HSV color space, and the pixel lower limit threshold and pixel upper limit threshold corresponding to each channel in the HSV color space reflect the color of the auxiliary materials, the area extracted by the target mask is the image area in the image of the product to be detected that is consistent with the color of the auxiliary materials.

[0128] Step B4, based on the target mask, the pixel value of the image area where the auxiliary material is located in the color image is set to the first pixel value, and the pixel value of the remaining area is set to the second pixel value to obtain the image of the product to be detected.

[0129] In the present disclosure, a target mask can be used to traverse a color image, and the pixel values ​​of pixels whose pixel values ​​are within the target mask range are reset to a first pixel value, and the pixel values ​​of pixels whose pixel values ​​are not within the target mask range are reset to a second pixel value. The first pixel value can be set to 255, and the second pixel value can be set to a relatively small pixel value such as 0 or 1.

[0130] The target mask can be used to extract image areas in the color image that are consistent with the color of the auxiliary materials. By resetting the pixel values ​​of the extracted area and the remaining area, the binarization processing of the auxiliary material-related area and the background area can be achieved, so that the obtained image of the product to be tested can more accurately determine the reference image area corresponding to each auxiliary material and the image area to be tested where each auxiliary material is currently located.

[0131] In a possible implementation manner, the acquiring of the image of the product to be inspected may include steps C1-C3:

[0132] Step C1, collecting a color image of the product to be inspected.

[0133] This step can refer to step A1 and will not be described in detail here.

[0134] Step C2: extracting the first color space of the color image.

[0135] In the present disclosure, an image acquisition device such as a mobile phone camera or a digital camera may be used to acquire a color image of the product to be inspected.

[0136] In the present disclosure, the first color space may be an RGB color space of an image, and the pixel values ​​in the RGB color space of the color image of the product to be detected may be normalized. The specific normalization process may refer to step A2 and will not be described in detail here.

[0137] Step C3: based on a second conversion relationship, convert the first color space of the color image into a grayscale space to obtain an image of the product to be detected.

[0138] In the present disclosure, the second conversion relationship may include the following formula, and the first color space of the color image may be converted into a grayscale space with reference to the following formula to obtain an image of the product to be detected:

[0139] Gray=n r ×R+n g ×G+n b ×B

[0140] Among them, Gray represents the image of the product to be detected, n r 、n g and n b are the weight parameters corresponding to the R channel, G channel and B channel, n r +n g +n b =1,n r 、n g and n b It can be set based on experimental data and standardization requirements of human eye sensitivity to color. For example, n r =0.299, n g =0.587, n b =0.114.

[0141] In one possible implementation, Figure 2 A schematic diagram of an image region flow chart provided by an embodiment of the present disclosure is shown. Figure 2 As shown, the determination of the reference image area corresponding to each auxiliary material in the image of the product to be detected and the image area to be detected where each auxiliary material is currently located based on the grayscale threshold comprises:

[0142] S201, determining a plurality of connected regions corresponding to auxiliary materials in the image of the product to be detected according to the grayscale threshold.

[0143] In the present disclosure, the pixel values ​​of the pixels in the image of the product to be detected within the grayscale threshold range can be reset to 255, and the pixel values ​​of the pixels in the image of the product to be detected that are not within the grayscale threshold range can be reset to 0, so as to realize the binarization of the image of the product to be detected. Then, the connected domains of the binarized image can be marked to obtain multiple connected regions.

[0144] Alternatively, in a possible implementation manner, determining the multiple connected regions corresponding to the auxiliary materials in the image of the product to be detected according to the grayscale threshold may further include steps D1-D2:

[0145] Step D1: for each pixel in the image of the product to be detected, determine the area formed by the pixel and the target adjacent pixel of the pixel as a pixel connected area.

[0146] The target adjacent pixel point refers to the adjacent pixel point whose pixel difference with the adjacent pixel point is less than the grayscale threshold. The grayscale threshold refers to the grayscale threshold corresponding to the pixel point, that is, the grayscale threshold corresponding to the area where the pixel point is located. For example, the pixel point is in the auxiliary material area A, and the grayscale threshold corresponding to the auxiliary material area A is 100-150, then the grayscale threshold corresponding to all the pixels in the auxiliary material area A is 100-150. When the pixel difference with the adjacent pixel point of the pixel point is less than the minimum value in the grayscale threshold, the area formed by the adjacent pixel point and the pixel point can be determined as the pixel point connected area.

[0147] In the present disclosure, traversal can be started from any pixel point in the product image to be inspected, and the area formed by the pixel point and the target adjacent pixel point of the pixel point is determined as the pixel point connected area, that is, adjacent pixels with similar grayscale are merged into the same area, thereby obtaining multiple pixel point connected areas.

[0148] Step D2, merging the connected regions of each pixel point that can be connected to obtain multiple connected regions.

[0149] In the present disclosure, after obtaining a plurality of pixel point connected regions, the plurality of pixel point connected regions may be merged. Specifically, the regions may be merged according to the grayscale difference between the pixel point connected regions.

[0150] S202, merging the multiple connected regions to obtain multiple merged image regions.

[0151] In a possible implementation manner, the merging of multiple connected regions may include steps E1-E3:

[0152] Step E1, determining the average grayscale value corresponding to each connected area.

[0153] Step E2, calculating the difference in average grayscale values ​​between each connected region.

[0154] Step E3, merging the connected areas whose difference is less than the preset grayscale difference.

[0155] In the present disclosure, the preset gray-scale difference value can be a specific value. For example, the preset gray-scale difference value can be set to 5, 10, or the like. The preset gray-scale difference value can also be a value range. When the difference value of the average gray-scale values between connected regions satisfies any value within this value range, the connected regions can be merged. For example, if the gray-scale value G1 is the average gray-scale value of connected region 1, the gray-scale value G2 is the average gray-scale value of connected region 2, and T is the preset gray-scale difference value, if |G1 - G2| < T, then connected region 1 and connected region 2 can be merged into one connected region. However, in order to reduce the problems of possible misclassification of the segmented connected regions and the inaccurate threshold setting of T affecting the accuracy of connected region merging, the preset gray-scale difference value T can be set to belong to the value range (T1, T2), that is, T1 < T < T2. Furthermore, the connected regions can be accurately merged by traversing the preset gray-scale difference value T. Specifically, the initial value of T can be set to be equal to T1, and use T1 to traverse whether the difference value between the average gray-scale values of each connected region is less than T1. If it is less than T1, the connected regions can be merged. If not, the value of T can be reset based on the value range (T1, T2), and then traverse again whether the difference value between the average gray-scale values of each connected region is less than T. If until all the values of T in the value range (T1, T2) cannot make the difference value between the average gray-scale values of each connected region less than T, it can be determined that the connected regions cannot be merged.

[0156] In the present disclosure, the connected regions can be merged pixel by pixel. Specifically, the difference value between the gray-scale value of a pixel point and the average gray-scale value of the connected region can be calculated. If the difference value is less than the preset gray-scale difference value, then this pixel point can be merged into this connected region, and iterate this merging process until no new pixel points can be added to the connected region.

[0157] S203, based on the attribute information of each merged image region, determine the merged image region corresponding to the auxiliary material currently as the image region to be measured, and determine the merged image region that needs to be configured with the auxiliary material as the reference image region.

[0158] In the present disclosure, the attribute information of the merged image region can include information such as the area and aspect ratio of the merged image region. For the merged image region of the auxiliary material configuration region whose area and aspect ratio are close to the standard, this merged image region can be determined as the reference image region. For the merged image region whose area and aspect ratio are quite different from the standard auxiliary material configuration region, this merged image region can be determined as the image region to be measured.

[0159] In another possible implementation, determining the reference image area corresponding to each auxiliary material in the image of the product to be inspected and the image area to be inspected where each auxiliary material is currently located based on the grayscale threshold comprises: inputting the grayscale threshold and the image of the product to be inspected into a pre-trained image segmentation model to obtain the reference image area corresponding to each auxiliary material in the image of the product to be inspected and the image area to be inspected where each auxiliary material is currently located; wherein the image segmentation model is trained based on a sample image and an identification area corresponding to the sample image.

[0160] In the present disclosure, the image segmentation model can be obtained by training a model to be trained based on a sample image and an identification area corresponding to the sample image, wherein the model to be trained can adopt a neural network structure such as Unet (convolutional neural network for image segmentation) or resnet (deep convolutional neural network), the sample image can include multiple product images, and the identification area corresponding to the sample image includes a reference area corresponding to the product accessories and an area where the product accessories are actually configured.

[0161] In the present disclosure, a sample image can be input into a model to be trained, and the model to be trained identifies the predicted reference area and the predicted area to be tested corresponding to the auxiliary material in the sample image; then, the current loss function value of the model to be trained is determined based on the identification area, the predicted reference area, and the predicted area to be tested corresponding to the sample image. If the loss function value is less than a preset loss function threshold, or the current number of iterations reaches a preset number of iterations, the current model to be trained can be determined as an image segmentation model, otherwise, the training iteration is returned to continue. Among them, the loss function can adopt a cross entropy loss function or a logarithmic loss function, etc., the preset loss function threshold can be set to 0.95 or 0.98 according to actual needs, and the preset number of iterations can be set to 100 times or 200 times according to actual needs.

[0162] The image segmentation model can be used to accurately segment the product area to be inspected with complex background.

[0163] In a possible implementation manner, the minimum bounding rectangle of the reference image area and the minimum bounding rectangle of the image area to be measured may be determined. Figure 3 FIG. 4 shows a schematic diagram of a minimum bounding rectangle of an image area provided by an embodiment of the present disclosure, such as Figure 3 As shown, the image to be detected 310 includes a reference image region 301 and a detected image region 302 . The minimum bounding rectangle 303 of the detected image region 302 can be determined, and the minimum bounding rectangle 304 of the reference image region 301 can be determined.

[0164] After determining the minimum circumscribed rectangle of the reference image area and the minimum circumscribed rectangle of the image area to be measured, the first center point coordinates of the minimum circumscribed rectangle of the reference image area and the second center point coordinates of the minimum circumscribed rectangle of the image area to be measured can be calculated, and / or the first offset angle of the minimum circumscribed rectangle of the reference image area and the second offset angle of the minimum circumscribed rectangle of the image area to be measured, and / or the first area of ​​the reference image area and the second area of ​​the image area to be measured can be calculated.

[0165] In the present disclosure, after calculating the first center point coordinates of the minimum circumscribed rectangle of the reference image area and the second center point coordinates of the minimum circumscribed rectangle of the image area to be tested, the offset between the reference image area and the image area to be tested can be determined according to the first center point coordinates and the second center point coordinates. Specifically, if the first center point coordinates are (x1, y1) and the second center point coordinates are (x2, y2), the coordinate difference △x in the x direction can be calculated as |x1-x2|, and the coordinate difference △y in the y direction can be calculated as |y1-y2|. If △x is greater than the preset x direction threshold, it indicates that there is an x ​​direction offset defect between the reference image area and the image area to be tested. If △y is greater than the preset y direction threshold, it indicates that there is a y direction offset defect between the reference image area and the image area to be tested, that is, there is an auxiliary material offset defect between the reference image area and the image area to be tested, and there is an offset problem in the auxiliary material configuration of the product to be tested, and the user needs to be reminded to reconfigure the auxiliary material position. Among them, the preset x direction threshold and the preset y direction threshold can be the same or different, and the preset x direction threshold and the preset y direction threshold can be set according to actual application requirements.

[0166] In the present disclosure, the first offset angle of the minimum circumscribed rectangle of the reference image area may refer to the angle between the side of the minimum circumscribed rectangle of the reference image area and the horizontal line, and the second offset angle of the minimum circumscribed rectangle of the image area to be measured may refer to the angle between the side of the minimum circumscribed rectangle of the image area to be measured and the horizontal line. In the present disclosure, the angle between the reference image area and the image area to be measured can be calculated based on the first offset angle and the second offset angle. If the angle is greater than a preset angle threshold, it indicates that there is an auxiliary material misalignment defect in the reference image area and the image area to be measured. Specifically, the absolute value of the difference between the first offset angle and the second offset angle can be calculated as the angle between the reference image area and the image area to be measured.

[0167] In the present disclosure, after calculating the first area of ​​the reference image area and the second area of ​​the image area to be tested, the area of ​​the image area to be tested excluding the part intersecting with the reference image area can be determined. If the area is greater than the preset area threshold, it indicates that the area where the image area to be tested and the reference image area are misaligned is large, that is, the offset between the image area to be tested where the auxiliary material is currently located and the reference image area is large. Therefore, it can be determined that there is an auxiliary material misalignment defect in the reference image area and the image area to be tested. The preset area threshold can be set according to actual application requirements. For example, the preset area threshold can be set to 50 or 60 pixels. Figure 4 A schematic diagram of a reference image area and an image area to be tested provided by an embodiment of the present disclosure is shown. Figure 4 As shown, the product image 410 to be inspected includes a reference image area 401 and an image area to be inspected 402. Figure 4 In the figure, the area of ​​the image area 402 to be tested excluding the part intersecting with the reference image area 401, the remaining area is the image area 403. If the area of ​​the image area 403 is greater than the preset area threshold, it indicates that the offset between the image area 402 to be tested where the auxiliary material is currently located and the reference image area 401 is large, that is, there is an auxiliary material misalignment defect between the reference image area 401 and the image area 401 to be tested.

[0168] In the present disclosure, the minimum distance and / or maximum distance between the reference image area and the image area to be tested can also be calculated. Specifically, the distance between each pixel pair between the reference image area and the image area to be tested can be calculated, and the minimum value is taken as the minimum distance between the two areas, and the maximum value is taken as the maximum distance between the two areas. If the minimum distance is less than the first preset distance threshold, it indicates that the reference image area and the image area to be tested overlap or are close, that is, there is no offset. Otherwise, it is determined that there is an auxiliary material offset defect between the reference image area and the image area to be tested; if the maximum distance is greater than the second preset distance threshold, it indicates that the reference image area and the image area to be tested have a large offset, and it can be determined that there is an auxiliary material offset defect between the reference image area and the image area to be tested. Among them, the second preset distance threshold is greater than the first preset distance threshold, and the second preset distance threshold and the first preset distance threshold can be set according to the actual application scenario.

[0169] In another possible implementation, the present disclosure may also extract the first edge segment of the reference image area, and extract the second edge segment of the image area to be measured. In the present disclosure, multiple pixel points at the edge of the reference image area may be extracted, and a straight line may be fitted based on the multiple edge pixel points as the first edge segment of the reference image area. Similarly, multiple pixel points at the edge of the image area to be measured may be extracted, and a straight line may be fitted based on the multiple edge pixel points as the second edge segment of the image area to be measured. Specifically, the least squares method may be used to fit multiple pixel points at the edge of the reference image area to obtain the first edge segment, and multiple pixel points at the edge of the image area to be measured may be fitted to obtain the second edge segment. Among them, a weight function can be introduced when fitting a line segment using the least squares method. The weight function represents the distance between a pixel point and a fitted line segment. According to the weight function, the pixel point with a larger distance has a smaller weight, and the pixel point with a smaller distance has a larger weight. At the same time, the weighted least squares method combined with weights can be used in combination with the RANSAC (random sampling consensus algorithm) algorithm. By randomly increasing control parameters such as the number of edge pixels in the image area and the number of fitting iterations, the mean square error of the fitted line segment can be minimized or the number of inner points can be maximized, thereby improving the accuracy of the fitted line segment.

[0170] After fitting to obtain the first edge line segment of the reference image region and the second edge line segment of the image region to be measured, in the present disclosure, the determining of the feature difference between the first geometric feature and the second geometric feature may further include step F1:

[0171] Step F1, determining the absolute value of the difference between the inclination angle of the first edge segment and the inclination angle of the second edge segment as the feature difference; or calculating the difference between the maximum distance and the minimum distance between the first edge segment and the second edge segment as the feature difference.

[0172] In the present disclosure, the angle between the first edge segment and the horizontal line can be determined as the inclination angle of the first edge segment, and the angle between the second edge segment and the horizontal line can be determined as the inclination angle of the second edge segment. Then, the angle difference obtained by subtracting the small inclination angle from the large inclination angle is used as the characteristic difference. Alternatively, in the present disclosure, the first edge segment can be used as the reference segment to determine the inclination angle of the second edge segment relative to the first segment, and the inclination angle can be used as the characteristic difference. Specifically, the inclination angle formula can be used to calculate the inclination angle of the second edge segment relative to the first segment. The inclination angle formula can be: inclination angle = arctan((y2-y1) / (x2-x1)), where x1, y1 are the coordinates of the two endpoints of the first edge segment, and x2, y2 are the coordinates of the two endpoints of the second edge segment. For example, Figure 5 A schematic diagram of an edge segment provided by an embodiment of the present disclosure is shown in FIG. Figure 5 As shown, taking the first edge line segment 501 of the reference image area in the image to be detected as the reference line segment, it can be determined that the inclination angle between the second edge line segment 502 of the image area to be detected and the first line segment 501 is α, and the angle α can be used as the characteristic difference. If the angle α is greater than the preset angle threshold, it indicates that the rotation offset between the image area to be detected where the auxiliary material is currently located and the reference image area is large, that is, there is an auxiliary material deviation defect between the reference image area and the image area to be detected.

[0173] In the present disclosure, the distance between each pixel point on the first edge segment and each pixel point on the second edge segment can be calculated, and the maximum value is taken as the maximum distance between the first edge segment and the second edge segment, and the minimum value is taken as the minimum distance between the first edge segment and the second edge segment, and then the difference between the maximum distance and the minimum distance is taken as the characteristic difference. Since the difference between the maximum distance and the minimum distance between the line segments reflects the parallel relationship between the two line segments, when the two line segments are infinitely close to parallel, the difference between the maximum distance and the minimum distance is close to 0, and the larger the difference, the less parallel the two line segments are. The parallelism of the two line segments indicates that the reference image area and the image area to be measured are relatively parallel, and there is no rotational offset. Otherwise, it indicates that there is a rotational offset between the reference image area and the image area to be measured. Therefore, it is possible to determine whether there is a rotational offset between the reference image area and the image area to be tested based on the difference between the maximum distance and the minimum distance between the first edge line segment and the second edge line segment. Specifically, if the difference between the maximum distance and the minimum distance between the first edge line segment and the second edge line segment is greater than a preset difference threshold, it indicates that there is a rotational offset between the reference image area and the image area to be tested, that is, there is an auxiliary material misalignment defect between the reference image area and the image area to be tested. If the difference between the maximum distance and the minimum distance between the first edge line segment and the second edge line segment is not greater than a preset difference threshold, it indicates that there is no rotational offset between the reference image area and the image area to be tested, that is, there is no auxiliary material misalignment defect between the reference image area and the image area to be tested.

[0174] In one possible implementation, Figure 6 FIG. 4 shows a schematic diagram of comparing images of products to be inspected provided by an embodiment of the present disclosure, such as Figure 6 As shown, Figure 6 (a) shows a color image 610 of a product to be inspected captured by an image acquisition device such as a mobile phone camera or a digital camera. The color image 610 includes multiple auxiliary material regions such as auxiliary material 601, auxiliary material 602, and auxiliary material 603. The auxiliary material 601, auxiliary material 602, and auxiliary material 603 are different types of auxiliary materials. After normalization and binarization preprocessing of the color image 610, the following can be obtained: Figure 6(b) shows the preprocessed image 620. In the preprocessed image 620, the pixel values ​​of the pixels in other areas except the auxiliary material area are set to 0, so as to distinguish the auxiliary material related area from the background area, and facilitate the extraction of accurate auxiliary material reference image area and image area to be tested, so as to perform accurate defect detection.

[0175] Based on the same inventive concept, according to the defect detection method provided by the above embodiment of the present disclosure, correspondingly, another embodiment of the present disclosure further provides a defect detection device, whose structural schematic diagram is shown in FIG. Figure 7 As shown, specifically including:

[0176] An image acquisition module 701 is used to acquire an image of a product to be detected, wherein the image of the product to be detected includes an image of an auxiliary material of the product to be detected;

[0177] A grayscale threshold determination module 702 is used to determine the grayscale threshold corresponding to each auxiliary material according to the color information of the auxiliary materials of the product to be detected;

[0178] An image region determination module 703 is used to determine, based on the grayscale threshold, a reference image region corresponding to each auxiliary material in the image of the product to be detected and an image region to be detected where each auxiliary material is currently located;

[0179] A feature extraction module 704 is used to determine a first geometric feature of the reference image region and a second geometric feature of the image region to be measured;

[0180] The defect detection module 705 is used to determine the feature difference between the first geometric feature and the second geometric feature, and determine whether the product to be inspected has a target defect based on the feature difference.

[0181] The device is used to obtain an image of a product to be inspected, the image of the product to be inspected includes an image of the auxiliary materials of the product to be inspected, the grayscale threshold corresponding to each auxiliary material is determined according to the color information of the auxiliary materials of the product to be inspected, and based on the grayscale threshold, the reference image area corresponding to each auxiliary material in the image of the product to be inspected and the image area to be inspected where each auxiliary material is currently located are determined, the first geometric feature of the reference image area is determined, and the second geometric feature of the image area to be inspected is determined, the feature difference between the first geometric feature and the second geometric feature is determined, and whether the product to be inspected has a target defect is determined based on the feature difference. That is, the grayscale threshold is determined by the color information of the auxiliary material, the reference image area and the image area to be inspected where the auxiliary material is currently located are accurately determined based on the grayscale threshold, and then by comparing the difference in geometric features between the image area to be inspected where the auxiliary material is currently located and the reference image area, it can be accurately determined whether the auxiliary material has a displacement defect, thereby ensuring product quality.

[0182] In one possible implementation, the image acquisition module 701 is specifically used to collect a color image of a product to be inspected; extract a first color space of the color image, and convert the first color space of the color image into a second color space based on a preset conversion relationship, the first color space and the second color space being color spaces formed by different image channels of the color image; for each image channel in the second color space, based on a pixel lower threshold and a pixel upper threshold corresponding to the image channel, determine a target mask corresponding to the image channel, the target mask being used to extract an image area in the color image that is consistent with the color of the auxiliary material; based on the target mask, set the pixel value of the image area where the auxiliary material is located in the color image to the first pixel value, and set the pixel value of the remaining area to the second pixel value, to obtain an image of the product to be inspected.

[0183] In one possible implementation, the image acquisition module 701 is specifically used to collect a color image of the product to be inspected; extract a first color space of the color image; and based on a second conversion relationship, convert the first color space of the color image into a grayscale space to obtain an image of the product to be inspected.

[0184] In one possible implementation, the grayscale threshold determination module 702 is specifically used to divide the image of the product to be inspected into regions based on the position of auxiliary materials to obtain multiple sub-image regions; calculate the average grayscale value of each of the sub-image regions; based on a first correspondence between grayscale value and color, determine the color information of the sub-image region corresponding to the average grayscale value; and determine the grayscale threshold corresponding to the sub-image region according to a second correspondence between color and grayscale and the color information.

[0185] In one possible implementation, the image area determination module 703 is specifically used to determine, based on the grayscale threshold, multiple connected areas corresponding to the auxiliary materials in the image of the product to be detected; merge the multiple connected areas to obtain multiple merged image areas; based on the attribute information of each merged image area, determine that the merged image area currently corresponding to the auxiliary materials is the image area to be tested, and determine that the merged image area where the auxiliary materials need to be configured is the reference image area.

[0186] In one possible implementation, the image area determination module 703 is specifically used to determine, for each pixel point in the image of the product to be inspected, an area formed by the pixel point and the target adjacent pixel points of the pixel point as a pixel point connected area, where the target adjacent pixel point refers to an adjacent pixel point among the adjacent pixel points of the pixel point whose pixel difference with the pixel point is less than a grayscale threshold; and merge the connected pixel areas of each pixel point that can be connected to obtain multiple connected areas.

[0187] In one possible implementation, the image region determination module 703 is specifically used to determine the average grayscale value corresponding to each connected region; calculate the difference in average grayscale values ​​between each connected region; and merge connected regions whose difference is less than a preset grayscale difference.

[0188] In one possible implementation, the image area determination module 703 is specifically used to input the grayscale threshold and the image of the product to be inspected into a pre-trained image segmentation model to obtain a reference image area corresponding to each auxiliary material in the image of the product to be inspected and an image area to be inspected where each auxiliary material is currently located; wherein the image segmentation model is trained based on a sample image and an identification area corresponding to the sample image.

[0189] In one embodiment, the first geometric feature includes at least one of the following features:

[0190] The coordinates of a first center point of the minimum circumscribed rectangle of the reference image area;

[0191] a first offset angle of the minimum circumscribed rectangle of the reference image area;

[0192] a first area of ​​the reference image region;

[0193] The second geometric feature includes at least one of the following features:

[0194] The coordinates of the second center point of the minimum circumscribed rectangle of the image area to be measured;

[0195] A second offset angle of the minimum circumscribed rectangle of the image area to be measured;

[0196] The second area of ​​the image region to be measured.

[0197] In one possible implementation manner, the feature extraction module 704 is further used to extract a first edge segment of the reference image region, and to extract a second edge segment of the image region to be measured;

[0198] The defect detection module 705 is specifically used to determine the absolute value of the difference between the inclination angle of the first edge segment and the inclination angle of the second edge segment as a feature difference; or, calculate the difference between the maximum distance and the minimum distance between the first edge segment and the second edge segment as a feature difference.

[0199] In one possible implementation, the defect detection module 705 is specifically configured to determine that the product to be detected has an auxiliary material misalignment defect if the feature difference is greater than a preset difference threshold.

[0200] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0201] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0202] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0203] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0204] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the defect detection method. For example, in some embodiments, the defect detection method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the defect detection method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the defect detection method in any other appropriate manner (e.g., by means of firmware).

[0205] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0206] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0207] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0208] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0209] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0210] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0211] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0212] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0213] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A defect detection method, characterized in that: The method comprises: Acquire an image of a product to be inspected, wherein the image of the product to be inspected includes an image of an auxiliary material of the product to be inspected; Determine the grayscale threshold corresponding to each auxiliary material according to the color information of the auxiliary materials of the product to be detected; Based on the grayscale threshold, determining the reference image area corresponding to each auxiliary material in the image of the product to be detected and the image area to be detected where each auxiliary material is currently located; Determining a first geometric feature of the reference image area, and determining a second geometric feature of the image area to be measured; A feature difference between the first geometric feature and the second geometric feature is determined, and whether the product to be inspected has a target defect is determined based on the feature difference.

2. The method according to claim 1, characterized in that The step of obtaining the image of the product to be detected includes: Collect color images of the products to be inspected; Extracting a first color space of the color image, and converting the first color space of the color image into a second color space based on a preset conversion relationship, the first color space and the second color space being color spaces formed by different image channels of the color image; For each image channel in the second color space, based on a pixel lower threshold and a pixel upper threshold corresponding to the image channel, a target mask corresponding to the image channel is determined, wherein the target mask is used to extract an image area in the color image that is consistent with the color of the auxiliary material; Based on the target mask, the pixel value of the image area where the auxiliary material is located in the color image is set to the first pixel value, and the pixel value of the remaining area is set to the second pixel value to obtain the image of the product to be detected.

3. The method according to claim 1, characterized in that The step of obtaining the image of the product to be detected includes: Collect color images of the products to be inspected; extracting a first color space of the color image; Based on the second conversion relationship, the first color space of the color image is converted into a grayscale space to obtain an image of the product to be detected.

4. The method according to claim 1, characterized in that: Determining the grayscale threshold corresponding to each auxiliary material according to the color information of the auxiliary material of the product to be detected includes: Dividing the image of the product to be inspected into regions based on the position of the auxiliary materials to obtain a plurality of sub-image regions; Calculating the average grayscale value of each sub-image area; Based on a first correspondence between grayscale values ​​and colors, determining color information of the sub-image area corresponding to the average grayscale value; A grayscale threshold corresponding to the sub-image area is determined according to the first corresponding relationship between color and grayscale and the color information.

5. The method according to claim 1, characterized in that The step of determining, based on the grayscale threshold, a reference image region corresponding to each auxiliary material in the image of the product to be detected and an image region to be detected where each auxiliary material is currently located, comprises: Determine, according to the grayscale threshold, a plurality of connected regions corresponding to the auxiliary materials in the image of the product to be detected; Merging multiple connected regions to obtain multiple merged image regions; Based on the attribute information of each merged image area, the merged image area currently corresponding to the auxiliary material is determined as the image area to be tested, and the merged image area to which the auxiliary material needs to be configured is determined as the reference image area.

6. The method according to claim 5, characterized in that The step of determining, according to the grayscale threshold, a plurality of connected regions corresponding to the auxiliary materials in the image of the product to be detected comprises: For each pixel in the image of the product to be inspected, an area formed by the pixel and a target adjacent pixel of the pixel is determined as a pixel connected area, where the target adjacent pixel refers to an adjacent pixel whose pixel difference with the pixel is less than a grayscale threshold among the adjacent pixels of the pixel; The connected regions of each pixel point that can be connected are merged to obtain multiple connected regions.

7. The method according to claim 5, characterized in that The merging of the multiple connected regions includes: Determine the average gray value corresponding to each connected area; Calculate the difference in average gray values ​​between each connected area; The connected areas whose difference is less than the preset grayscale difference are merged.

8. A defect detection device, characterized in that: The device comprises: An image acquisition module, used to acquire an image of the product to be detected, wherein the image of the product to be detected includes an image of an auxiliary material of the product to be detected; A grayscale threshold determination module, used to determine the grayscale threshold corresponding to each auxiliary material according to the color information of the auxiliary materials of the product to be detected; An image region determination module, used to determine, based on the grayscale threshold, a reference image region corresponding to each auxiliary material in the image of the product to be detected and an image region to be detected where each auxiliary material is currently located; A feature extraction module, used to determine a first geometric feature of the reference image area, and to determine a second geometric feature of the image area to be measured; The defect detection module is used to determine the feature difference between the first geometric feature and the second geometric feature, and determine whether the product to be inspected has a target defect based on the feature difference.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to any one of claims 1-7.

Images