Workpiece defect detection method and device, electronic equipment and storage medium

Abstract

The application discloses a workpiece defect detection method and device, electronic equipment and storage medium, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a first workpiece image of a workpiece to be detected; inputting the first workpiece image into a defect detection model to obtain first defect information output by the defect detection model; wherein inputting the first workpiece image into the defect detection model to obtain the first defect information output by the defect detection model further comprises the following steps: performing mask processing on a mask area of the first workpiece image to obtain a second workpiece image, the mask area of the first workpiece image being determined based on a target defect distribution density map; and performing defect detection on the second workpiece image to obtain the first defect information; wherein the defect detection model is obtained by training based on a sample image set and corresponding sample labels, and the target defect distribution density map is used to represent the relative density of distribution of each defect category in the sample image set. The method can accurately locate the mask area in the image.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to a method, apparatus, electronic device and storage medium for detecting defects in workpieces. Background Technology

[0002] With the development of image processing technology and the upgrading of vision hardware, machine vision defect detection methods have gradually replaced manual inspection and have been verified and widely used in industrial production inspection. Machine vision inspection technology is a non-contact automatic inspection technology with advantages such as safety and reliability, high detection accuracy, and long-term operation in complex production environments. It is an important technology for realizing industrial intelligence and automation.

[0003] Machine vision inspection first uses machine vision equipment to convert the captured target into an image signal and transmit it to a dedicated image processing system. Based on information such as pixel distribution, brightness, and color, the signal is converted into a digital signal. The image processing system performs various calculations on these signals to obtain the target's features, performs detection, and then controls the actions of other equipment on the production line based on the detection results.

[0004] Machine vision defect detection technology is becoming increasingly popular. Deep learning segmentation models are commonly used to achieve visual detection. When the acquired image is large, both image detection and model training are very time-consuming. Usually, the regions in the original image that do not need to be detected are masked to reduce the image analysis area.

[0005] Therefore, determining the mask area and improving detection efficiency are urgent technical problems that need to be solved. Summary of the Invention

[0006] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a workpiece defect detection method, apparatus, electronic device, and storage medium that accurately determines the image mask area, effectively improves detection efficiency, and avoids interference from the mask area with the detection effect.

[0007] In a first aspect, this application provides a method for detecting defects in a workpiece, the method comprising:

[0008] Acquire the first workpiece image of the workpiece to be tested;

[0009] The first workpiece image is input into the defect detection model to obtain the first defect information output by the defect detection model.

[0010] The step of inputting the first workpiece image into the defect detection model to obtain the first defect information output by the defect detection model further includes:

[0011] The masked area of ​​the first workpiece image is masked to obtain the second workpiece image. The masked area of ​​the first workpiece image is determined based on the target defect distribution density map.

[0012] Defect detection is performed on the second workpiece image to obtain the first defect information;

[0013] The defect detection model is trained based on a set of sample images and their corresponding sample labels, and the target defect distribution density map is used to characterize the relative density of each defect category in the set of sample images.

[0014] According to the workpiece defect detection method of this application, the relative density law of the distribution of image defect categories is analyzed by the target defect distribution density map, the mask area in the image is accurately located, the defect detection efficiency is effectively improved, and the mask area is avoided from interfering with the detection results.

[0015] According to one embodiment of this application, after obtaining the first defect information output by the defect detection model, the method further includes:

[0016] Based on the first defect information, a first defect distribution density map is obtained. The first defect distribution density map is used to characterize the relative density of each defect category in the first workpiece image.

[0017] If the defect distribution density of the first defect distribution density map is greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map, the defect detection result of the workpiece to be tested is obtained based on the first defect information.

[0018] If the defect detection result of the workpiece under test is determined to be incorrect, the defect detection model is retrained.

[0019] According to one embodiment of this application, after obtaining the first defect distribution density map based on the first defect information, the method further includes:

[0020] If the defect distribution density of the first defect distribution density map is less than or equal to the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map, the defect rate of the production line to which the workpiece to be tested belongs is obtained.

[0021] If the defect rate is determined to be greater than the defect rate threshold, the defect distribution density of the first defect distribution density map is re-determined.

[0022] According to one embodiment of this application, after obtaining the defect detection result of the workpiece to be tested, the method further includes:

[0023] If the defect detection result of the workpiece under test is confirmed to be correct, the production process of the workpiece under test shall be inspected.

[0024] If the manufacturing process of the workpiece under test is confirmed to be correct, the target defect distribution density map is regenerated.

[0025] According to one embodiment of this application, determining that the defect distribution density of the first defect distribution density map is greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map includes:

[0026] Obtain the first defect coordinate information of the first defect distribution density map, and obtain the target defect coordinate information of the corresponding type of defect in the target defect distribution density map;

[0027] Based on the first defect coordinate information and the target defect coordinate information, the defect coordinate difference is determined;

[0028] If the defect coordinate difference is determined to be greater than the target threshold, the defect distribution density of the first defect distribution density map is determined to be greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map.

[0029] According to one embodiment of this application, the target defect distribution density map is generated through the following steps:

[0030] Multiple sample images from the sample image set are input into the defect detection model for training, and multiple sample defect information output by the defect detection model is obtained. The multiple sample defect information corresponds one-to-one with the multiple sample images.

[0031] Based on the multiple sample images and the multiple sample defect information, a target defect distribution density map is generated. The sample defect information includes the category information and location information of various defects in the sample images.

[0032] Secondly, this application provides a workpiece defect detection device, the device comprising:

[0033] The acquisition module is used to acquire the first workpiece image of the workpiece to be tested;

[0034] The processing module is used to input the first workpiece image into the defect detection model and obtain the first defect information output by the defect detection model;

[0035] The step of inputting the first workpiece image into the defect detection model to obtain the first defect information output by the defect detection model further includes:

[0036] The masked area of ​​the first workpiece image is masked to obtain the second workpiece image. The masked area of ​​the first workpiece image is determined based on the target defect distribution density map.

[0037] Defect detection is performed on the second workpiece image to obtain the first defect information;

[0038] The defect detection model is trained based on a set of sample images and their corresponding sample labels, and the target defect distribution density map is used to characterize the relative density of each defect category in the set of sample images.

[0039] According to the workpiece defect detection device of this application, the relative density law of the distribution of image defect categories is analyzed by the target defect distribution density map, the mask area in the image is accurately located, the defect detection efficiency is effectively improved, and the mask area is avoided from interfering with the detection results.

[0040] Thirdly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the workpiece defect detection method as described in the first aspect above.

[0041] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the workpiece defect detection method as described in the first aspect above.

[0042] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the workpiece defect detection method as described in the first aspect above.

[0043] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0044] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0045] Figure 1 This is one of the flowcharts illustrating the workpiece defect detection method provided in the embodiments of this application;

[0046] Figure 2 This is a second schematic flowchart of the workpiece defect detection method provided in the embodiments of this application;

[0047] Figure 3 This is a schematic diagram of the workpiece defect detection device provided in the embodiments of this application;

[0048] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0049] Figure 5 This is a hardware schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0051] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0052] The workpiece defect detection method, workpiece defect detection device, electronic device, and readable storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0053] Among them, the workpiece defect detection method can be applied to the terminal, and can be executed by the hardware or software in the terminal.

[0054] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0055] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0056] The workpiece defect detection method provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the workpiece defect detection method. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices. The workpiece defect detection method provided in this application embodiment will be described below using an electronic device as the execution subject as an example.

[0057] like Figure 1 As shown, the workpiece defect detection method includes steps 110 and 120.

[0058] Step 110: Obtain the first workpiece image of the workpiece to be tested.

[0059] The workpiece to be tested refers to the workpiece that needs to be inspected for defects.

[0060] In this step, an industrial vision system can be set up on the industrial production line. The industrial vision system can include equipment such as cameras, light sources and acquisition cards. The industrial vision system acquires image data of the workpiece to be tested and obtains the first workpiece image.

[0061] In the actual generation process, the workpiece to be tested can be a lithium battery, screen, rod, or other workpiece that has been processed by manufacturing technology. The first workpiece image of the workpiece to be tested acquired by the industrial vision system can be a color image or a grayscale image.

[0062] Step 120: Input the first workpiece image into the defect detection model to obtain the first defect information output by the defect detection model.

[0063] The defect detection model is trained based on a set of sample images and their corresponding sample labels. The defect detection model is used to detect the location and type of defects in the images.

[0064] In this step, the first workpiece image is input into the defect detection model, which performs defect detection. The resulting first defect information includes information characterizing the location and type of defects in the first workpiece image.

[0065] It is understandable that the sample image set includes multiple sample images, each of which contains certain data defects. Each defect can be manually labeled with a corresponding sample label, allowing the defect detection model to learn the relationship between the features of the defects in the sample images and the labels.

[0066] In this embodiment, step 120, inputting the first workpiece image into the defect detection model and obtaining the first defect information output by the defect detection model, further includes:

[0067] The masked area of ​​the first workpiece image is masked to obtain the second workpiece image. The masked area of ​​the first workpiece image is determined based on the target defect distribution density map.

[0068] Defect detection is performed on the second workpiece image to obtain the first defect information.

[0069] Among them, the target defect distribution density map is used to characterize the relative density of the distribution of each defect category in the sample image set.

[0070] In actual implementation, the first workpiece image is input into the defect detection model. The defect detection model first performs masking processing on the masked area of ​​the first workpiece image, and then performs defect detection on the masked image to obtain the first defect information of the workpiece to be tested.

[0071] The target defect distribution density map is used to characterize the relative density of each defect category in the sample image set. The target defect distribution density map can accurately reflect the relative density pattern of defect category distribution in the image.

[0072] For example, image A includes a first region, a second region, and a third region. Image A has four types of defects: a first defect, a second defect, a third defect, and a fourth defect. The first and second defects are densely distributed in the first region, and their relative density is high. The third and fourth defects are densely distributed in the third region, and their relative density is also high.

[0073] Based on the relative density pattern of defect categories in the image reflected by the target defect distribution density map, it can be determined that the second region in image A is a region without defect distribution.

[0074] The defect detection model determines the mask area of ​​the first workpiece image based on the target defect distribution density map. Based on the relative density law of the image defect category distribution represented by the target defect distribution density map, it can accurately determine the defect-free area in the image, and then automatically generate a mask area for masking processing of the defect-free area.

[0075] In this embodiment, the defect detection model accurately determines the defect-free region (i.e., the mask region) in the image based on the target defect distribution density map, performs masking processing on the mask region to obtain the second workpiece image, performs defect detection on the second workpiece image to obtain the first defect information, effectively improving the defect detection efficiency and avoiding interference from the mask region with the detection results.

[0076] According to the workpiece defect detection method provided in the embodiments of this application, the relative density law of the distribution of image defect categories is analyzed by the target defect distribution density map, the mask area in the image is accurately located, the defect detection efficiency is effectively improved, and the mask area is avoided from interfering with the detection results.

[0077] In some embodiments, after obtaining the first defect information output by the defect detection model in step 120, the workpiece defect detection method may further include:

[0078] Based on the first defect information, the first defect distribution density map is obtained;

[0079] If the defect distribution density of the first defect distribution density map is greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map, the defect detection result of the workpiece to be tested is obtained based on the first defect information.

[0080] If the defect detection results of the workpiece under test are found to be incorrect, the defect detection model is retrained.

[0081] The first defect distribution density map is used to characterize the relative density of each defect category in the first workpiece image.

[0082] The first defect information includes information characterizing the location and type of defects in the first workpiece image. Based on the first defect information, the distribution location of each type of defect in the first workpiece image can be obtained, and then the relative density of each defect type can be determined to obtain the first defect distribution density map.

[0083] In this embodiment, the relative density pattern of the defect category distribution represented by the first defect distribution density map and the target defect distribution density map are compared to determine whether the relative density pattern of the defect category distribution represented by the first defect distribution density map conforms to the relative density pattern of the defect category distribution represented by the target defect distribution density map.

[0084] In actual implementation, the relative density of each type of defect distribution in the first defect distribution density map is compared with the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map.

[0085] For example, the first defect distribution density map includes four types of defects: first defect, second defect, third defect, and fourth defect, totaling five defects. The relative density of the first defect distribution in the first defect distribution density map is compared with the relative density of the fourth defect in the target defect distribution density map.

[0086] The defect distribution density threshold of a defect is compared; the relative density of the second defect distribution in the first defect distribution density map is compared with the defect distribution density threshold of the second defect in the target defect distribution density map.

[0087] It should be noted that the defect distribution density threshold for each type of defect in the target defect distribution density map can be preset.

[0088] The settings can be configured based on the defect distribution density range of each defect in each sample image of the sample image set. In this embodiment, the defect distribution density of the first defect distribution density map is determined to be greater than that of the target defect distribution density map.

[0089] The defect distribution density threshold of the corresponding type of defect in the first defect distribution density map is greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map. This indicates that the relative density law of the defect category distribution represented by the first defect distribution density map does not conform to the relative density law of the defect category distribution represented by the target defect distribution density map.

[0090] 5. In some embodiments, the defect distribution density of the first defect distribution density map is determined to be greater than the target defect distribution density.

[0091] The defect distribution density thresholds for corresponding defect types in the figure can provide automatic warnings to indicate to the user that the relative density pattern of the defect category distribution of the current workpiece under test does not conform to the relative density pattern of the defect category distribution represented by the target defect distribution density map.

[0092] If the defect distribution density of the first defect distribution density map is greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map, the defect detection result of the workpiece under test is obtained based on the first defect information.

[0093] The defect detection results of the workpiece under test are judged manually or by other models to determine the detection status of the defect detection model.

[0094] In this embodiment, if the defect detection result of the workpiece under test is determined to be incorrect, it indicates that the defect detection model is performing an error and the defect detection model needs to be retrained and iteratively updated.

[0095] 5. In some embodiments, after obtaining a first defect distribution density map based on the first defect information, the workpiece defects...

[0096] Detection methods may also include:

[0097] If the defect distribution density of the first defect distribution density map is less than or equal to the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map, the defect rate of the production line to which the workpiece to be tested belongs is obtained.

[0098] If the defect rate is determined to be greater than the defect rate threshold, the defect 0 distribution density of the first defect distribution density map is redefined.

[0099] In this embodiment, the defect distribution density of the first defect distribution density map is determined to be less than or equal to the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map. That is, the defect distribution density of all types of defects in the first defect distribution density map is less than or equal to the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map. This indicates that the relative density law of the defect category distribution represented by the first defect distribution density map conforms to the relative density law of the defect category distribution represented by the target defect distribution density map.

[0100] When the relative density of the defect category distribution represented by the first defect distribution density map conforms to the law, the defect rate of the production line to which the workpiece to be tested belongs is detected. The defect rate refers to the ratio of defective products to all products in a certain period of time, which is a key indicator for measuring production quality.

[0101] If the defect rate is greater than the defect rate threshold, it indicates that the proportion of defective products to all products during the production period of the workpiece to be tested is relatively high. Therefore, it is necessary to redetermine the defect distribution density of the first defect distribution density map, compare the defect distribution density of the first defect distribution density map with the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map, and re-determine the relative density law of the defect category distribution of the workpiece to be tested.

[0102] In some embodiments, after obtaining the defect detection results of the workpiece to be tested, the workpiece defect detection method may further include:

[0103] If the defect detection results of the workpiece under test are confirmed to be correct, the production process of the workpiece under test shall be inspected.

[0104] Assuming the manufacturing process of the workpiece under test is correct, the target defect distribution density map is regenerated.

[0105] In this embodiment, if the defect detection result of the workpiece under test is determined to be correct, it indicates that the defect detection model has performed defect detection without error and there is no need to retrain the defect detection model. However, it is necessary to check the production process of the workpiece under test to determine whether the relative density of the defect category distribution of the workpiece under test does not conform to the corresponding rule due to an error in the production process of the workpiece under test.

[0106] It should be noted that when the production process of the workpiece under test is determined to be correct, it indicates that the relative density law of the defect category distribution represented by the target defect distribution density map may not be comprehensive or accurate enough. The target defect distribution density map is then regenerated to correct the relative density law of the defect category distribution represented by the target defect distribution density map.

[0107] In practice, the first workpiece image and the first defect distribution density map of the workpiece to be tested can be introduced into the sample training set to supplement and correct the relative density law of the defect category distribution represented by the target defect distribution density map.

[0108] In some embodiments, determining that the defect distribution density of the first defect distribution density map is greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map may include:

[0109] Obtain the coordinate information of the first defect in the first defect distribution density map, and obtain the coordinate information of the target defect of the corresponding type in the target defect distribution density map;

[0110] Based on the coordinate information of the first defect and the coordinate information of the target defect, the defect coordinate difference is determined;

[0111] If the defect coordinate difference is greater than the target threshold, the defect distribution density of the first defect distribution density map is determined to be greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map.

[0112] Obtain the first defect coordinate information of each type of defect in the first defect distribution density map, and obtain the corresponding type of defect in the target defect distribution density map. Calculate the difference between the coordinate information of the same type of defect in the first defect distribution density map and the target defect distribution density map. Use the defect coordinate difference to determine whether the defect distribution density of the first defect distribution density map is greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map.

[0113] It is understandable that defects are distributed at different locations on the workpiece to be tested. Both the first defect coordinate information and the target defect coordinate information include multiple coordinate values. The difference between the first defect coordinate information and the target defect coordinate information can be the standard deviation of multiple coordinate values. If the standard deviation is greater than the target threshold, it is determined whether the defect distribution density of the first defect distribution density map is greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map, thus triggering an early warning.

[0114] In some embodiments, the target defect distribution density map is generated through the following steps:

[0115] Multiple sample images from the sample image set are input into the defect detection model for training, and multiple sample defect information output by the defect detection model is obtained. The multiple sample defect information corresponds one-to-one with the multiple sample images.

[0116] Based on multiple sample images and multiple sample defect information, a target defect distribution density map is generated. The sample defect information includes the category information and location information of various defects in the sample images.

[0117] In this embodiment, each coordinate of each sample image is traversed, and the sum of the number of a certain type of defect including this coordinate point in each sample image is counted. Then, the sum is divided by the number of sample images in the sample image set to obtain the relative density of the distribution of that type of defect. All defect categories in the sample image set are traversed to generate a target defect distribution density map.

[0118] The following is a specific example.

[0119] like Figure 2 As shown, the flowchart on the left represents the training phase of the defect detection model, and the flowchart on the right represents the application phase of the defect detection model.

[0120] The training phase includes:

[0121] Step 1: Label the sample images for defects, train the defect detection model, and test the defect detection model.

[0122] Step 2: Use the defect detection model to perform image defect detection, generate a target defect distribution density map, set the conditions for determining defect-free areas based on the target defect distribution density map, determine defect-free areas, and automatically generate mask areas.

[0123] Step 3: Determine whether the mask area meets the production line requirements. If the mask area does not meet the production line requirements, reset the conditions for determining the defect-free area, regenerate the target defect distribution density map, determine the defect-free area, and automatically generate the mask area.

[0124] Once the masked area meets the production line requirements, continue training the defect detection model.

[0125] The application phase includes:

[0126] Step 4: Deploy the defect detection model and the corresponding target defect distribution density map on the production line, and set the warning threshold for judging abnormal defect distribution, that is, the defect distribution density threshold of each type of defect in the target defect distribution density map.

[0127] Step 5: Production line inspection. When a production cycle is reached, check whether the defect distribution density of the workpiece under test is normal, that is, determine whether the defect distribution density of the defect distribution density map of the workpiece under test is greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map.

[0128] An early warning is issued when the defect distribution density of the first defect distribution density map of the workpiece under test is greater than the defect distribution density threshold of the corresponding 5 types of defects in the target defect distribution density map, that is, when the defect distribution density of the workpiece under test is abnormal.

[0129] Step 6: After the warning is issued, image analysis is performed based on the defect detection results of the workpiece to be tested. If the detection results are incorrect, the defect detection model is retrained.

[0130] Step 7: When the test results are correct, determine whether the production process of the production line is normal.

[0131] Under normal production conditions, a new defect distribution density map is generated for the workpiece to be tested, and the defect distribution density threshold is re-evaluated.

[0132] If the production process is abnormal, modify the production process and continue to perform curve detection on other workpieces.

[0133] The workpiece defect detection method of this application analyzes the relative density law of the distribution of image defect categories, automatically generates a mask area, and can detect results that do not conform to the distribution relative density law of defect categories during the detection process.

[0134] Providing early warnings and timely confirmation of the reliability of test results helps users to conduct timely testing of the production process, effectively improving the production efficiency of the 5-liter production line.

[0135] The workpiece defect detection method provided in this application can be executed by a workpiece defect detection device. This application uses a workpiece defect detection device executing the workpiece defect detection method as an example to illustrate the workpiece defect detection device provided in this application.

[0136] This application also provides a workpiece defect detection device.

[0137] 0 as Figure 3 As shown, the workpiece defect detection device includes:

[0138] The acquisition module 310 is used to acquire a first workpiece image of the workpiece to be tested;

[0139] Processing module 320 is used to input the first workpiece image into the defect detection model and obtain the first defect information output by the defect detection model;

[0140] The process of inputting the first workpiece image into the defect detection model and obtaining the first defect information output by the defect detection model further includes:

[0141] The masked area of ​​the first workpiece image is masked to obtain the second workpiece image. The masked area of ​​the first workpiece image is determined based on the target defect distribution density map.

[0142] Defect detection is performed on the second workpiece image to obtain the first defect information;

[0143] The defect detection model is trained based on the sample image set and its corresponding sample labels, and the target defect 0 distribution density map is used to characterize the relative density of the distribution of each defect category in the sample image set.

[0144] According to the workpiece defect detection device provided in the embodiments of this application, the relative density law of the distribution of image defect categories is analyzed by the target defect distribution density map, the mask area in the image is accurately located, the defect detection efficiency is effectively improved, and the mask area is avoided from interfering with the detection results.

[0145] In some embodiments, after obtaining the first defect information output by the defect detection model, the processing module 320 is further configured to obtain a first defect distribution density map based on the first defect information. The first defect distribution density map is used to characterize the relative density of the distribution of each defect category in the first workpiece image.

[0146] If the defect distribution density of the first defect distribution density map is greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map, the defect detection result of the workpiece to be tested is obtained based on the first defect information.

[0147] If the defect detection results of the workpiece under test are found to be incorrect, the defect detection model is retrained.

[0148] In some embodiments, after obtaining a first defect distribution density map based on the first defect information, the processing module 320 is further configured to, if it is determined that the defect distribution density of the first defect distribution density map is less than or equal to the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map,

[0149] Obtain the defect rate of the production line to which the workpiece to be tested belongs;

[0150] If the defect rate is determined to be greater than the defect rate threshold, the defect distribution density of the first defect distribution density map is redefined.

[0151] In some embodiments, after obtaining the defect detection result of the workpiece to be tested, the processing module 320 is further configured to inspect the production process of the workpiece to be tested if it is determined that the defect detection result of the workpiece to be tested is correct.

[0152] Assuming the manufacturing process of the workpiece under test is correct, the target defect distribution density map is regenerated.

[0153] In some embodiments, the processing module 320 is used to obtain the first defect coordinate information of the first defect distribution density map and the target defect coordinate information of the corresponding type of defect in the target defect distribution density map;

[0154] Based on the coordinate information of the first defect and the coordinate information of the target defect, the defect coordinate difference is determined;

[0155] If the defect coordinate difference is greater than the target threshold, the defect distribution density of the first defect distribution density map is determined to be greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map.

[0156] In some embodiments, the processing module 320 is configured to generate a target defect distribution density map through the following steps:

[0157] Multiple sample images from the sample image set are input into the defect detection model for training, and multiple sample defect information output by the defect detection model is obtained. The multiple sample defect information corresponds one-to-one with the multiple sample images.

[0158] Based on multiple sample images and multiple sample defect information, a target defect distribution density map is generated. The sample defect information includes the category information and location information of various defects in the sample images.

[0159] The workpiece defect detection device in this application embodiment can be an electronic device or a component of an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the specific type of device.

[0160] The workpiece defect detection device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0161] The workpiece defect detection device provided in this application embodiment can achieve... Figures 1 to 2 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0162] In some embodiments, such as Figure 4 As shown, this application embodiment also provides an electronic device 400, including a processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401. When the program is executed by the processor 401, it implements the various processes of the above-described workpiece defect detection method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0163] It should be noted that the electronic devices in the embodiments of this application include the aforementioned mobile electronic devices and non-mobile electronic devices.

[0164] Figure 5 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0165] The electronic device 500 includes, but is not limited to, components such as: radio frequency unit 501, network module 502, audio output unit 503, input unit 504, sensor 505, display unit 506, user input unit 507, interface unit 508, memory 509, and processor 510.

[0166] Those skilled in the art will understand that the electronic device 500 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 510 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0167] In this embodiment of the application, the input unit 504 is a camera, used to acquire a first workpiece image of the workpiece to be tested.

[0168] Processor 510 is used to input the first workpiece image into the defect detection model and obtain the first defect information output by the defect detection model;

[0169] The process of inputting the first workpiece image into the defect detection model to obtain the first defect information output by the defect detection model further includes:

[0170] The masked area of ​​the first workpiece image is masked to obtain the second workpiece image. The masked area of ​​the first workpiece image is determined based on the target defect distribution density map.

[0171] Defect detection is performed on the second workpiece image to obtain the first defect information;

[0172] The defect detection model is trained based on the sample image set and its corresponding sample labels, and the target defect distribution density map is used to characterize the relative density of the distribution of each defect category in the sample image set.

[0173] According to the electronic device provided in the embodiments of this application, the relative density law of the distribution of image defect categories is analyzed by the target defect distribution density map, the mask area in the image is accurately located, the defect detection efficiency is effectively improved, and the mask area is avoided from interfering with the detection results.

[0174] In some embodiments, the processor 510 is further configured to obtain a first defect distribution density map based on the first defect information, the first defect distribution density map being used to characterize the relative density of the distribution of each defect category in the first workpiece image;

[0175] If the defect distribution density of the first defect distribution density map is greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map, the defect detection result of the workpiece to be tested is obtained based on the first defect information.

[0176] If the defect detection results of the workpiece under test are found to be incorrect, the defect detection model is retrained.

[0177] In some embodiments, the processor 510 is further configured to, when determining that the defect distribution density of the first defect distribution density map is less than or equal to the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map,

[0178] Obtain the defect rate of the production line to which the workpiece to be tested belongs;

[0179] If the defect rate is determined to be greater than the defect rate threshold, the defect distribution density of the first defect distribution density map is redefined.

[0180] In some embodiments, the processor 510 is also configured to inspect the manufacturing process of the workpiece under test if the defect detection result of the workpiece under test is determined to be correct.

[0181] Assuming the manufacturing process of the workpiece under test is correct, the target defect distribution density map is regenerated.

[0182] In some embodiments, the processor 510 is further configured to obtain first defect coordinate information of the first defect distribution density map and target defect coordinate information of the corresponding type of defect in the target defect distribution density map;

[0183] Based on the coordinate information of the first defect and the coordinate information of the target defect, the defect coordinate difference is determined;

[0184] If the defect coordinate difference is greater than the target threshold, the defect distribution density of the first defect distribution density map is determined to be greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map.

[0185] In some embodiments, the processor 510 is further configured to generate a target defect distribution density map by means of the following steps:

[0186] Multiple sample images from the sample image set are input into the defect detection model for training, and multiple sample defect information output by the defect detection model is obtained. The multiple sample defect information corresponds one-to-one with the multiple sample images.

[0187] Based on multiple sample images and multiple sample defect information, a target defect distribution density map is generated. The sample defect information includes the category information and location information of various defects in the sample images.

[0188] It should be understood that, in this embodiment, the input unit 504 may include a graphics processing unit (GPU) 5041 and a microphone 5042. The GPU 5041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 506 may include a display panel 5061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 507 includes at least one of a touch panel 5071 and other input devices 5072. The touch panel 5071 is also called a touch screen. The touch panel 5071 may include a touch detection device and a touch controller. Other input devices 5072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0189] The memory 509 can be used to store software programs and various data. The memory 509 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 509 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 509 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0190] Processor 510 may include one or more processing units; processor 510 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 510.

[0191] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described workpiece defect detection method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0192] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0193] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described workpiece defect detection method.

[0194] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0195] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described workpiece defect detection method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0196] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0197] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0198] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0199] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0200] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0201] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for detecting defects in a workpiece, characterized in that, include: Acquire the first workpiece image of the workpiece to be tested; The first workpiece image is input into the defect detection model to obtain the first defect information output by the defect detection model. The step of inputting the first workpiece image into the defect detection model to obtain the first defect information output by the defect detection model further includes: The masked area of ​​the first workpiece image is masked to obtain the second workpiece image. The masked area of ​​the first workpiece image is determined based on the target defect distribution density map. Defect detection is performed on the second workpiece image to obtain the first defect information; The defect detection model is trained based on a set of sample images and their corresponding sample labels, and the target defect distribution density map is used to characterize the relative density of each defect category in the set of sample images. After obtaining the first defect information output by the defect detection model, the method further includes: Based on the first defect information, a first defect distribution density map is obtained. The first defect distribution density map is used to characterize the relative density of each defect category in the first workpiece image. If the defect distribution density of the first defect distribution density map is greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map, the defect detection result of the workpiece to be tested is obtained based on the first defect information. If the defect detection result of the workpiece under test is determined to be incorrect, the defect detection model is retrained.

2. The workpiece defect detection method according to claim 1, characterized in that, After obtaining the first defect distribution density map based on the first defect information, the method further includes: If the defect distribution density of the first defect distribution density map is less than or equal to the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map, the defect rate of the production line to which the workpiece to be tested belongs is obtained. If the defect rate is determined to be greater than the defect rate threshold, the defect distribution density of the first defect distribution density map is re-determined.

3. The workpiece defect detection method according to claim 1, characterized in that, After obtaining the defect detection result of the workpiece to be tested, the method further includes: If the defect detection result of the workpiece under test is confirmed to be correct, the production process of the workpiece under test shall be inspected. If the manufacturing process of the workpiece under test is confirmed to be correct, the target defect distribution density map is regenerated.

4. The workpiece defect detection method according to claim 1, characterized in that, The step of determining that the defect distribution density of the first defect distribution density map is greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map includes: Obtain the first defect coordinate information of the first defect distribution density map, and obtain the target defect coordinate information of the corresponding type of defect in the target defect distribution density map; Based on the first defect coordinate information and the target defect coordinate information, the defect coordinate difference is determined; If the defect coordinate difference is determined to be greater than the target threshold, the defect distribution density of the first defect distribution density map is determined to be greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map.

5. The workpiece defect detection method according to any one of claims 1-4, characterized in that, The target defect distribution density map is generated through the following steps: Multiple sample images from the sample image set are respectively input into the defect detection model for training, and multiple sample defect information output by the defect detection model is obtained. The multiple sample defect information corresponds one-to-one with the multiple sample images. Based on the multiple sample images and the multiple sample defect information, a target defect distribution density map is generated. The sample defect information includes the category information and location information of various defects in the sample images.

6. A workpiece defect detection device, characterized in that, include: The acquisition module is used to acquire the first workpiece image of the workpiece to be tested; The processing module is used to input the first workpiece image into the defect detection model and obtain the first defect information output by the defect detection model; The step of inputting the first workpiece image into the defect detection model to obtain the first defect information output by the defect detection model further includes: The masked area of ​​the first workpiece image is masked to obtain the second workpiece image. The masked area of ​​the first workpiece image is determined based on the target defect distribution density map. Defect detection is performed on the second workpiece image to obtain the first defect information; The defect detection model is trained based on a set of sample images and their corresponding sample labels, and the target defect distribution density map is used to characterize the relative density of each defect category in the set of sample images. After obtaining the first defect information output by the defect detection model, the method further includes: Based on the first defect information, a first defect distribution density map is obtained. The first defect distribution density map is used to characterize the relative density of each defect category in the first workpiece image. If the defect distribution density of the first defect distribution density map is greater than the defect distribution density threshold of the corresponding type of defect in the target defect distribution density map, the defect detection result of the workpiece to be tested is obtained based on the first defect information. If the defect detection result of the workpiece under test is determined to be incorrect, the defect detection model is retrained.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the workpiece defect detection method as described in any one of claims 1-5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the workpiece defect detection method as described in any one of claims 1-5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the workpiece defect detection method as described in any one of claims 1-5.

Images