Product defect detection method, system and storage medium

Abstract

The present application discloses a product defect detection method, system and storage medium, which relates to image recognition technology and includes the following steps: obtaining three-dimensional image information of an object to be detected; obtaining a two-dimensional image and depth data of the object to be detected based on the three-dimensional image information; converting the depth data into a grayscale image; performing defect detection based on the grayscale image to obtain a first detection result; inputting the two-dimensional image into an image detection model to obtain a second detection result; inputting the three-dimensional image information into a three-dimensional image detection model to obtain a third detection result; when at least one of the first detection result, the second detection result and the third detection result indicates that the object to be detected has a defect, generating defect prompt information. The present application can improve the accuracy of defect detection.

Description

Technical Field

[0001] The present application relates to image recognition technology, and in particular to a product defect detection method, system and storage medium. Background Art

[0002] In the prior art, when production equipment detects defects on products, image detection technology is often used. The current image detection technology is mainly based on two-dimensional image detection technology, which has insufficient detection accuracy for defects such as scratches with unclear colors, and is prone to missed detection. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention provides a product defect detection method, system and storage medium to improve the detection accuracy of defects.

[0004] On the one hand, an embodiment of the present application provides a product defect detection method, comprising the following steps:

[0005] Acquire three-dimensional image information of the object to be measured, wherein the three-dimensional image information is collected by a three-dimensional camera;

[0006] Obtaining a two-dimensional image and depth data of the object to be measured according to the three-dimensional image information;

[0007] Converting the depth data into a grayscale image;

[0008] Perform defect detection according to the grayscale image to obtain a first detection result;

[0009] Inputting the two-dimensional image into an image detection model to obtain a second detection result;

[0010] Inputting the three-dimensional image information into a three-dimensional image detection model to obtain a third detection result;

[0011] When at least one of the first detection result, the second detection result and the third detection result indicates that the object to be detected has a defect, defect prompt information is generated.

[0012] In some embodiments, the three-dimensional image information is preprocessed, and the preprocessing process is to remove image information other than the object to be measured.

[0013] In some embodiments, the depth data corresponds to the two-dimensional image, and each pixel in the two-dimensional image corresponds to a depth data;

[0014] The step of converting the depth data into a grayscale image is specifically as follows:

[0015] The depth data corresponding to each pixel in the two-dimensional image is converted into a grayscale value to form a grayscale image.

[0016] In some embodiments, the converting of the depth data corresponding to each pixel in the two-dimensional image into a grayscale value is specifically as follows:

[0017] Subtract each depth data from the shallowest depth data;

[0018] The shallowest depth data is used as the reference grayscale to determine the grayscale corresponding to all depth data.

[0019] In some embodiments, the image detection model is obtained by:

[0020] Obtaining a labeled defect image as a first training sample;

[0021] The convolutional neural network is trained using the first training sample until a training condition is met.

[0022] In some embodiments, the image detection model is obtained by:

[0023] Acquire the labeled three-dimensional image information of the object to be tested as a second training sample;

[0024] The second training sample is split into a two-dimensional image and a depth input, and input into the three-dimensional image detection model for training until the training conditions are met.

[0025] In another aspect, the present embodiment discloses a product defect detection system, including:

[0026] A three-dimensional camera, used to collect three-dimensional image information of the object to be measured;

[0027] The processing module is used to execute the product defect detection method.

[0028] In another aspect, the present embodiment discloses a product defect detection system, including:

[0029] An acquisition unit, used for acquiring three-dimensional image information of the object to be measured, wherein the three-dimensional image information is collected by a three-dimensional camera;

[0030] A data decomposition unit, used to obtain a two-dimensional image and depth data of the object to be measured according to the three-dimensional image information;

[0031] A conversion unit, used to convert the depth data into a grayscale image;

[0032] A first detection unit, used for performing defect detection according to the grayscale image to obtain a first detection result;

[0033] A second detection unit, used for inputting the two-dimensional image into an image detection model to obtain a second detection result;

[0034] A third detection unit, used for inputting the three-dimensional image information into a three-dimensional image detection model to obtain a third detection result;

[0035] The prompting unit is used to generate defect prompting information when at least one of the first detection result, the second detection result and the third detection result indicates that the object to be detected has a defect.

[0036] In another aspect, the present embodiment discloses a product defect detection system, including:

[0037] Memory, used to store programs;

[0038] A processor is used to load the program to execute the product defect detection method.

[0039] On the other hand, the present embodiment discloses a storage medium storing a program, wherein the program implements the product defect detection method when executed by a processor.

[0040] The embodiment of the present application collects three-dimensional image information based on a three-dimensional camera, and separates the three-dimensional image information into a two-dimensional image and depth data, and converts the depth data into a grayscale image. The use of the grayscale image for defect detection actually utilizes the statistical characteristics of the depth data, and the use of a deep neural network model to identify two-dimensional image and three-dimensional image data utilizes the model to mine hidden data features, thereby realizing defect detection from three dimensions with higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 It is a flow chart of a product defect detection method. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described through the implementation method with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0044] In the description of the present invention, "several" means more than one, "many" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0045] In the description of the present invention, unless otherwise clearly defined, words such as "setting" should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meaning of the above words in the present invention in combination with the specific content of the technical solution.

[0046] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0047] Reference Figure 1 This embodiment discloses a product defect detection method, comprising the following steps:

[0048] Step 101: Acquire three-dimensional image information of an object to be measured, wherein the three-dimensional image information is collected by a three-dimensional camera.

[0049] It is understandable that the 3D camera exists as part of the product production line. When the product is transferred to a specific station, the 3D camera shoots it. Then, recognition is performed based on the captured 3D data to determine whether the product has defects. In this embodiment, the object to be tested generally refers to the product or a part of the product or a surface of the product.

[0050] Step 102: Obtain a two-dimensional image and depth data of the object to be measured according to the three-dimensional image information.

[0051] Three-dimensional image information is generally composed of multiple points, which contain the color information and depth information of the point. In fact, three-dimensional image information can be understood as a point cloud on the surface of the object to be measured. Therefore, the three-dimensional image information can be split to obtain a two-dimensional image and the depth data corresponding to each pixel in the two-dimensional image.

[0052] Step 103: Convert the depth data into a grayscale image.

[0053] In this embodiment, there are many ways to convert the depth data, for example, directly expressing the depth data as the corresponding grayscale, for example, the depth is 100, and the corresponding grayscale is 100. However, this method does not make the depth change obvious due to the large distance between the camera and the object to be measured. Therefore, in some embodiments, the depth data corresponds to the two-dimensional image, and each pixel in the two-dimensional image corresponds to a depth data;

[0054] The step of converting the depth data into a grayscale image is specifically as follows:

[0055] The depth data corresponding to each pixel in the two-dimensional image is converted into a grayscale value to form a grayscale image.

[0056] The depth data corresponding to each pixel in the two-dimensional image is converted into a grayscale value, specifically:

[0057] Subtract each depth data from the shallowest depth data;

[0058] The shallowest depth data is used as the reference grayscale to determine the grayscale corresponding to all depth data.

[0059] In this example, the shallowest depth data is used as the reference grayscale, for example, as 0, and the difference between the deeper depth data and the shallowest depth is used as the grayscale value. This method can highlight the depth difference in the depth data. Then, it can be determined whether there are obvious defects on the surface based on statistical information such as the variance of the grayscale image.

[0060] Step 104: Perform defect detection according to the grayscale image to obtain a first detection result.

[0061] In this step, whether there is a defect is determined based on the grayscale image, which is one of the judgment results.

[0062] Step 105: input the two-dimensional image into an image detection model to obtain a second detection result.

[0063] Next, the two-dimensional image is detected through a neural network model (such as a convolutional neural network model) to obtain a second judgment result. This judgment result is obtained based on the prediction of the trained convolutional neural network model. The result predicted by the neural network model is the probability that the object to be tested has a defect. When the probability is greater than a threshold, it is determined that the object to be tested has a defect.

[0064] The image detection model is obtained in the following way:

[0065] Obtain a labeled defect image as the first training sample; generally, an image is used as an input sample, and whether it is a defect image is used as a label.

[0066] The convolutional neural network is trained using the first training sample until the training condition is met. During the training process, the convolutional neural network parameters are updated using the gradient descent method until the convolutional neural network reaches the training number or the convolutional neural network can converge on the verification set.

[0067] Step 106: Input the three-dimensional image information into a three-dimensional image detection model to obtain a third detection result.

[0068] Specifically, the 3D image information is split into 2D image and depth data (input) before being input into the model, and then the trained 3D image detection model is used to predict whether there are defects. This result is the detection result of the third dimension.

[0069] The image detection model is obtained in the following way:

[0070] Acquire the labeled three-dimensional image information of the object to be tested as a second training sample;

[0071] The second training sample is split into a two-dimensional image and a depth input, and input into the three-dimensional image detection model for training until the training conditions are met. It can be seen that the training process corresponds to the prediction process.

[0072] Step 107: When at least one of the first detection result, the second detection result and the third detection result indicates that the object to be detected has a defect, defect prompt information is generated.

[0073] In this embodiment, no matter which of the three-dimensional detection results shows a defect, a relevant prompt will be generated. These prompts can be reflected on the control terminal of the production / detection equipment, which can be manifested as generating a prompt sound, generating a prompt message, generating a defect report, or performing a screening action on the product.

[0074] The embodiment of the present application collects three-dimensional image information based on a three-dimensional camera, and separates the three-dimensional image information into a two-dimensional image and depth data, and converts the depth data into a grayscale image. The use of the grayscale image for defect detection actually utilizes the statistical characteristics of the depth data, and the use of a deep neural network model to identify two-dimensional image and three-dimensional image data utilizes the model to mine hidden data features, thereby realizing defect detection from three dimensions with higher accuracy.

[0075] In some embodiments, in order to reduce the interference of redundant data (because the range captured by the camera is larger than the range to be detected), the three-dimensional image information is preprocessed, and the preprocessing process is to remove the image information outside the object to be detected. Generally, the target area of ​​the object to be detected is determined by image recognition, and then the data is cropped according to the target area.

[0076] This embodiment discloses a product defect detection system, including:

[0077] A three-dimensional camera, used to collect three-dimensional image information of the object to be measured;

[0078] The processing module is used to execute the product defect detection method.

[0079] This embodiment discloses a product defect detection system, including:

[0080] An acquisition unit, used for acquiring three-dimensional image information of the object to be measured, wherein the three-dimensional image information is collected by a three-dimensional camera;

[0081] A data decomposition unit, used to obtain a two-dimensional image and depth data of the object to be measured according to the three-dimensional image information;

[0082] A conversion unit, used to convert the depth data into a grayscale image;

[0083] A first detection unit, used for performing defect detection according to the grayscale image to obtain a first detection result;

[0084] A second detection unit, used for inputting the two-dimensional image into an image detection model to obtain a second detection result;

[0085] A third detection unit, used for inputting the three-dimensional image information into a three-dimensional image detection model to obtain a third detection result;

[0086] The prompting unit is used to generate defect prompting information when at least one of the first detection result, the second detection result and the third detection result indicates that the object to be detected has a defect.

[0087] This embodiment discloses a product defect detection system, including:

[0088] Memory, used to store programs;

[0089] A processor is used to load the program to execute the product defect detection method.

[0090] On the other hand, the present embodiment discloses a storage medium storing a program, wherein the program implements the product defect detection method when executed by a processor.

[0091] The above-mentioned hardware embodiment collects three-dimensional image information based on a three-dimensional camera, and separates the three-dimensional image information into a two-dimensional image and depth data, and converts the depth data into a grayscale image. The use of the grayscale image for defect detection actually utilizes the statistical characteristics of the depth data, while the use of a deep neural network model to identify two-dimensional image and three-dimensional image data utilizes the model to mine hidden data features, thereby realizing defect detection from three dimensions with higher accuracy.

[0092] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A product defect detection method, characterized in that: The following steps are involved: Acquire three-dimensional image information of the object to be measured, wherein the three-dimensional image information is collected by a three-dimensional camera; A two-dimensional image and depth data of the object to be tested are obtained according to the three-dimensional image information, wherein the depth data corresponds to the two-dimensional image; the depth data is converted into a grayscale image; defect detection is performed according to the grayscale image to obtain a first detection result; the two-dimensional image is input into an image detection model to obtain a second detection result, wherein the image detection model is obtained in the following manner: an annotated defect image is obtained as a first training sample, and a convolutional neural network is trained using the first training sample until a training condition is met; the three-dimensional image information is split into a two-dimensional image and depth data, and then the split two-dimensional image and depth data are input into a three-dimensional image detection model to obtain a third detection result, wherein the three-dimensional image detection model is obtained in the following manner: annotated three-dimensional image information of the object to be tested is obtained as a second training sample; the second training sample is split into a two-dimensional image and a depth input, and the two-dimensional image and depth data are input into a three-dimensional image detection model for training until the training condition is met; when at least one of the first detection result, the second detection result, and the third detection result indicates that the object to be tested has a defect, defect prompt information is generated.

2. The product defect detection method according to claim 1, characterized in that: The three-dimensional image information is preprocessed, and the preprocessing process is to remove image information other than the object to be measured.

3. The product defect detection method according to claim 1, characterized in that: Each pixel in the two-dimensional image corresponds to a depth data; the converting of the depth data into a grayscale image specifically includes: converting the depth data corresponding to each pixel in the two-dimensional image into a grayscale value, thereby forming a grayscale image.

4. The product defect detection method according to claim 3, characterized in that: The depth data corresponding to each pixel in the two-dimensional image is converted into a grayscale value, specifically: subtracting each depth data from the shallowest depth data; taking the shallowest depth data as the reference grayscale, and determining the grayscale corresponding to all the depth data.

5. A product defect detection system, characterized in that: include: A three-dimensional camera is used to collect three-dimensional image information of the object to be tested; a processing module is used to execute the product defect detection method as described in any one of claims 1-4.

6. A product defect detection system, characterized in that: include: An acquisition unit, used for acquiring three-dimensional image information of the object to be measured, wherein the three-dimensional image information is collected by a three-dimensional camera; a data decomposition unit, for obtaining a two-dimensional image and depth data of the object to be tested according to the three-dimensional image information, wherein the depth data corresponds to the two-dimensional image; a conversion unit, for converting the depth data into a grayscale image; a first detection unit, for performing defect detection according to the grayscale image to obtain a first detection result; a second detection unit, for inputting the two-dimensional image into an image detection model to obtain a second detection result, wherein the image detection model is obtained in the following manner: obtaining a labeled defect image as a first training sample, and using the first training sample to train a convolutional neural network until a training condition is met; a third detection unit, for splitting the three-dimensional image information into a two-dimensional image and depth data, and then inputting the split two-dimensional image and depth data into a three-dimensional image detection model to obtain a third detection result, wherein the three-dimensional image detection model is obtained in the following manner: obtaining a labeled three-dimensional image information of the object to be tested as a second training sample; splitting the second training sample into a two-dimensional image and a depth input, and inputting the two-dimensional image and the depth input into a three-dimensional image detection model for training until the training condition is met; and a prompt unit, for generating defect prompt information when at least one of the first detection result, the second detection result, and the third detection result indicates that the object to be tested has a defect.

7. A product defect detection system, characterized in that: include: Memory, used to store programs; A processor, used to load the program to execute the product defect detection method according to any one of claims 1 to 4.

8. A storage medium, characterized in that: It stores a program, and when the program is executed by a processor, the product defect detection method according to any one of claims 1 to 4 is implemented.

Images