Workpiece detection method and device, electronic equipment and storage medium

Abstract

The invention relates to a workpiece detection method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining 3D attitude point clouds of a three-dimensional model of a target workpiece at different attitude points before the online detection of the target workpiece, inputting the 3D attitude point clouds into a pre-trained artificial intelligence generation content model, and obtaining a first generation image; then detecting the first generated image based on a preset defect detection model to obtain a first training sample, so that the defect detection model can be trained in advance according to the first training sample to obtain an optimized detection model for the target workpiece; according to the method, the target workpiece can be subjected to defect detection according to the optimized detection model when the target workpiece is officially on line, so that the problem of false detection of the target workpiece is solved in advance, the detection first pass yield of the target workpiece can be improved, iterative training on the defect detection model is not needed when the target workpiece is officially on line, and the defect detection efficiency is improved.

Description

Technical Field

[0001] This application relates to the technical field of defect detection, and particularly to a workpiece detection method, device, electronic device, and storage medium. Background Art

[0002] Before a workpiece leaves the factory, it needs to be defect-detected. However, due to the diverse shapes of workpieces, differences in shape, photographing angle, occlusion, etc., there is a problem of a high probability of false detection when a new workpiece is put on the line for detection. If the false detection problem cannot be solved in advance, the detection through rate for each new workpiece put on the line is relatively low, and the detection algorithm needs to be frequently iteratively trained to improve the detection through rate, which affects the efficiency of defect detection. Summary of the Invention

[0003] This application provides a workpiece detection method, device, electronic device, and storage medium to solve the technical problem of how to improve the detection through rate of workpieces.

[0004] In a first aspect, this application provides a workpiece detection method, which includes: Obtain the 3D pose point clouds of a target workpiece at different pose points; Input the 3D pose point clouds into a pre-trained artificial intelligence generated content model to obtain a first generated image; wherein, the artificial intelligence generated content model is a model for generating 2D images based on 3D point clouds; Detect the first generated image based on a preset defect detection model to obtain a first training sample; Train the defect detection model based on the first training sample to obtain an optimized detection model for the target workpiece; Perform defect detection on the target workpiece according to the optimized detection model.

[0005] Optionally, the method further includes: obtaining the artificial intelligence generated content model; Wherein, the training process of the artificial intelligence generated content model includes: Sequentially collect target data when a first workpiece is at each point to be collected; wherein, when the first workpiece is at any target point in the points to be collected, collect the 3D point cloud data of the first workpiece based on a 3D camera, collect the real 2D image of the first workpiece based on a 2D camera, and obtain the pose transformation matrix between the 3D point cloud data and the real 2D image; use the 3D point cloud data, the real 2D image, and the pose transformation matrix as the target data at the target point; Perform the following processing on each group of the target data respectively to obtain a second training sample set: perform point cloud voxel filtering on the 3D point cloud data to obtain a downsampled point cloud; calculate the downsampled point cloud according to the pose transformation matrix to obtain a target point cloud unified to the perspective of the real 2D image, and use the target point cloud and the real 2D image as the second training samples; wherein, the second training sample set includes all the second training samples. Train the initial generation model according to the second training sample set to obtain the trained artificial intelligence generated content model.

[0006] Optionally, performing point cloud voxel filtering on the 3D point cloud data to obtain a downsampled point cloud includes: Divide the 3D point cloud data into a grid to obtain a plurality of three-dimensional voxels; For each target three-dimensional voxel in the three-dimensional voxels, perform point cloud voxel filtering processing; wherein, the point cloud voxel filtering processing includes: determining the voxel centroid of the target three-dimensional voxel; using the point closest to the voxel centroid in the target three-dimensional voxel as the representative point of the target three-dimensional voxel; Use all the representative points as the downsampled point cloud.

[0007] Optionally, dividing the 3D point cloud data into a grid to obtain a plurality of three-dimensional voxels includes: According to the formula Determine the maximum voxel side length; where lmax represents the maximum voxel side length, V represents all the points of the current voxel divided according to the voxel side length l, T represents all the voxels divided according to the voxel side length l, p V represents the point closest to the voxel centroid in the current voxel, f represents the curvature feature function, and λ represents the regularization term parameter; represents the square of the two-norm; Divide the 3D point cloud data into a grid according to the maximum voxel side length to obtain a plurality of the three-dimensional voxels; wherein, when dividing the grid according to the maximum voxel side length, the curvature feature loss of all the three-dimensional voxels is the smallest.

[0008] Optionally, training the initial generation model according to the second training sample set to obtain the trained artificial intelligence generated content model includes: Perform the following processing on each second training sample in the second training sample set: convert the target point cloud in the second training sample into a tensor format and perform feature extraction to obtain a feature map; perform upsampling, convolution, and normalization processing on the feature map to obtain a second generated image; use the target point cloud and the real 2D image as the first data pair, and use the target point cloud and the second generated image as the second data pair; Train the initial generation model according to the first set of data pairs and the second set of data pairs until the loss function is minimized, and use the model when the loss function is minimized as the trained artificial intelligence generated content model.

[0009] Optionally, detect the first generated image based on a preset defect detection model to obtain a first training sample, including: Detect the first generated image based on a preset defect detection model to obtain a detection failure image that fails the detection; Obtain a misdetection image after manually reviewing the detection failure image; Repeat the step of obtaining the 3D pose point cloud of the three-dimensional model of the target workpiece at different pose points until the step of obtaining the misdetection image after manually reviewing the detection failure image, until the number of repetitions reaches a preset number; Use all the misdetection images as the first training sample.

[0010] Optionally, train the defect detection model based on the first training sample to obtain an optimized detection model for the target workpiece, including: Train the defect detection model based on all the misdetection images until all the misdetection images can be recognized and passed by the defect detection model, and use the trained defect detection model as the optimized detection model for the target workpiece.

[0011] In a second aspect, the present application provides a workpiece detection device, the device includes: An acquisition module, configured to acquire the 3D pose point cloud of the three-dimensional model of the target workpiece at different pose points; An image generation module, configured to input the 3D pose point cloud into a pre-trained artificial intelligence generated content model to obtain a first generated image; wherein, the artificial intelligence generated content model is a model for generating a 2D image based on 3D point cloud; A simulation detection module, configured to detect the first generated image based on a preset defect detection model to obtain a first training sample; A defect detection model training module, configured to train the defect detection model based on the first training sample to obtain an optimized detection model for the target workpiece; A defect detection module, configured to detect defects of the target workpiece according to the optimized detection model.

[0012] In a third aspect, the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus; The memory is used to store a computer program; A processor, when executing a program stored in a memory, implements the workpiece detection method according to any one of the embodiments of the first aspect.

[0013] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the workpiece detection method according to any one of the embodiments of the first aspect.

[0014] The above technical solutions provided by the embodiments of the present application have the following advantages compared with the prior art: In the method provided by the embodiments of the present application, a 3D pose point cloud of a target workpiece at different pose points is obtained; the 3D pose point cloud is input into a pre-trained artificial intelligence generation content model to obtain a first generated image; wherein, the artificial intelligence generation content model is a model for generating a 2D image based on a 3D point cloud; a first training sample is obtained by detecting the first generated image based on a preset defect detection model; the defect detection model is trained based on the first training sample to obtain an optimized detection model for the target workpiece; and the target workpiece is defect-detected according to the optimized detection model. This method can obtain the 3D pose point cloud of the target workpiece at different pose points before the target workpiece is put on-line for detection, input the 3D pose point cloud into a pre-trained artificial intelligence generation content model to obtain a first generated image, and then detect the first generated image based on a preset defect detection model to obtain a first training sample, so that the defect detection model can be trained in advance according to the first training sample to obtain an optimized detection model for the target workpiece. When the target workpiece is officially put on-line, the target workpiece can be defect-detected according to the optimized detection model. Since the misdetection problem of the target workpiece is solved in advance, the detection throughput rate of the target workpiece can be improved, and it is not necessary to iteratively train the defect detection model when the target workpiece is officially put on-line, thereby improving the defect detection efficiency. Description of the Drawings

[0015] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application and used together with the specification to explain the principles of the present application.

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] One or more embodiments are exemplarily illustrated by the pictures in the corresponding drawings. These exemplary illustrations do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, unless otherwise stated, and the drawings in the figures do not constitute a scale limitation.

[0018] Figure 1 The system architecture diagram of a workpiece detection method provided by an embodiment of the present application; Figure 2 The flowchart of a workpiece detection method provided by an embodiment of the present application; Figure 3 The structural schematic diagram of a workpiece detection device provided by an embodiment of the present application; Figure 4 The structural schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some but not all of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the scope of protection of the present application.

[0020] The following disclosure provides many different embodiments or examples for implementing different structures of the present application. To simplify the disclosure of the present application, components and settings of specific examples are described below. Of course, they are only examples and are not intended to limit the present application. In addition, the present application may repeat reference numerals and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0021] To solve the technical problem of how to improve the detection pass rate of workpieces in the prior art, the present application provides a workpiece detection method, device, electronic device, and storage medium, which can train a defect detection model according to the first training sample before the target workpiece is put on the line for detection, obtain an optimized detection model for the target workpiece, and can perform defect detection on the target workpiece according to the optimized detection model when the target workpiece is officially put on the line. Since the misdetection problem of the target workpiece is solved in advance, the detection pass rate of the target workpiece can be improved.

[0022] The first embodiment of the present application provides a workpiece detection method, which can be applied as Figure 1The system architecture shown, in which at least a data acquisition module 101 and a data processing module 102 are included, and a communication connection is established between the data acquisition module 101 and the data processing module 102. The data acquisition module 101 can acquire 3D pose point clouds of the 3D model of the target workpiece at different pose points, and transmit the 3D pose point clouds to the data processing module 102. The data processing module 102 can input the 3D pose point clouds into a pre-trained artificial intelligence generated content model to obtain a first generated image, detect the first generated image based on a preset defect detection model to obtain a first training sample, and train the defect detection model based on the first training sample to obtain an optimized detection model for the target workpiece, so that the target workpiece can be defect-detected according to the optimized detection model.

[0023] Next, based on this system architecture, the workpiece detection method will be described in detail, as Figure 2 , the workpiece detection method includes: Step 201, acquire 3D pose point clouds of the 3D model of the target workpiece at different pose points.

[0024] The target workpiece refers to a new workpiece that needs to be put on the line for defect detection. The 3D model of the target workpiece can be the design 3D drawing of the target workpiece, and the 3D model includes the complete and regular point cloud data of the target workpiece. The pose of the 3D model can be adjusted in the software, and the 3D pose point clouds of the 3D model at different pose points can be collected. For example, 3D pose point clouds of the 3D model at 200 different pose points can be collected.

[0025] Step 202, input the 3D pose point clouds into a pre-trained artificial intelligence generated content model to obtain a first generated image; wherein, the artificial intelligence generated content model is a model for generating 2D images based on 3D point clouds.

[0026] The artificial intelligence generated content model is the AIGC (Artificial Intelligence Generated Content) generation model. The AIGC generation model can output 2D images based on the input 3D point clouds. In this embodiment, inputting the 3D pose point clouds into the pre-trained AIGC generation model can obtain a first generated image. For example, inputting 200 3D pose point clouds can obtain 200 first generated images.

[0027] In one embodiment, inputting the 3D pose point clouds into a pre-trained artificial intelligence generated content model to obtain a first generated image includes: performing point cloud voxel filtering processing on the 3D pose point clouds to obtain downsampled point cloud data, and inputting the downsampled point cloud data into the artificial intelligence generated content model to obtain a first generated image.

[0028] In this embodiment, to improve the speed of data processing, the 3D pose point cloud can be first downsampled, for example, by performing point cloud voxel filtering to obtain the downsampled point cloud data, and then the downsampled point cloud data is input into the AIGC model to obtain the first generated image. Briefly speaking, point cloud voxel filtering is to divide the 3D pose point cloud into three-dimensional voxel grids, and use the point closest to the center of gravity of the neighboring voxels in the current voxel to replace all points in the current voxel. Point cloud voxel filtering is also applied in the subsequent training process of the AIGC model, which will not be described in detail here.

[0029] In one embodiment, the method further includes: obtaining an artificial intelligence generated content model.

[0030] Among them, the training process of the artificial intelligence generated content model includes: sequentially collecting target data when the first workpiece is at each point to be collected. Among them, when the first workpiece is at any target point in the points to be collected, based on the 3D camera, collect the 3D point cloud data of the first workpiece, based on the 2D camera, collect the real 2D image of the first workpiece, and obtain the pose transformation matrix between the 3D point cloud data and the real 2D image. The 3D point cloud data, the real 2D image, and the pose transformation matrix are used as the target data at the target point. Each group of target data is processed as follows to obtain the second training sample set: perform point cloud voxel filtering on the 3D point cloud data to obtain the downsampled point cloud, calculate the downsampled point cloud according to the pose transformation matrix to obtain the target point cloud unified to the perspective of the real 2D image, and use the target point cloud and the real 2D image as the second training sample, where the second training sample set includes all second training samples. Train the initial generation model according to the second training sample set to obtain the trained artificial intelligence generated content model.

[0031] In this embodiment, collecting the target data when the first workpiece is at each point to be collected is data collection in the real scenario. For example, collecting the target data of the first workpiece undergoing defect detection on the production line to train the AIGC generation model based on the collected target data. The points to be collected may include 80 different pose points. When the first workpiece is at any target point, based on a 3D camera, collect the 3D point cloud data of the first workpiece, based on a 2D camera, collect the real 2D image of the first workpiece, and obtain the pose transformation matrix between the 3D point cloud data and the real 2D image. Take the 3D point cloud data, the real 2D image, and the pose transformation matrix as the target data at the target point. Repeat the above steps until the target data of the first workpiece at each point to be collected is collected in sequence. After collecting the target data at each point, perform point cloud voxel filtering on the 3D point cloud data in the target data to obtain the downsampled point cloud, and calculate the downsampled point cloud according to the pose transformation matrix to obtain the target point cloud unified to the perspective of the real 2D image. Take the target point cloud and the real 2D image as the second training samples. After processing each target data in sequence, take all the obtained second training samples as the second training sample set, and train the initial generation model according to the second training sample set to obtain the trained AIGC generation model.

[0032] In one embodiment, performing point cloud voxel filtering on the 3D point cloud data to obtain the downsampled point cloud includes: dividing the 3D point cloud data into grids to obtain a plurality of three-dimensional voxels; for each target three-dimensional voxel in the three-dimensional voxels, perform point cloud voxel filtering processing; wherein, the point cloud voxel filtering processing includes: determining the voxel centroid of the target three-dimensional voxel; taking the point closest to the voxel centroid in the target three-dimensional voxel as the representative point of the target three-dimensional voxel; taking all the representative points as the downsampled point cloud.

[0033] In this embodiment, the three-dimensional voxels obtained by grid division can be subjected to point cloud voxel filtering through the following formula (1).

[0034] Formula (1) wherein, V represents all the points in the current voxel; g V represents the voxel centroid of the current voxel; p V represents the point closest to the voxel centroid in the current voxel; p is any point in the current voxel, and the current voxel refers to the three-dimensional voxel currently undergoing point cloud voxel filtering.

[0035] In this embodiment, when performing point cloud voxel filtering, taking the point closest to the voxel centroid in the target three-dimensional voxel as the representative point of the target three-dimensional voxel. Since the point closest to the voxel centroid is itself a point in the original 3D point cloud data, it can better retain the features of the 3D point cloud data.

[0036] In one embodiment, the 3D point cloud data is meshed to obtain a plurality of three-dimensional voxels, including: Determine the maximum voxel side length according to the following formula (2), and mesh the 3D point cloud data according to the maximum voxel side length to obtain a plurality of three-dimensional voxels; wherein, when meshing according to the maximum voxel side length, the curvature feature loss of all three-dimensional voxels is minimized.

[0037] Formula (2) wherein, lmax represents the maximum voxel side length, V represents all points of the current voxel obtained by dividing according to the voxel side length l, T represents all voxels obtained by dividing according to the voxel side length l, p V represents the point closest to the voxel centroid in the current voxel, f represents the curvature feature function, and λ represents the regularization term parameter; represents the square of the second norm, which is used to reflect the difference between the curvature feature of the sampled point cloud and the curvature feature of the original point cloud.

[0038] In this embodiment, since the amount of point cloud data is large and there are many features, which has a great impact on the training speed, the selection of the voxel side length lmax is crucial. The method of formula (2) is used to determine the specific voxel side length, which can maximize point cloud downsampling on the basis of retaining the three-dimensional point cloud features and improve the training speed of the AIGC model.

[0039] For example, the curvature feature of the current voxel V is [0.2, 0.1, 0.8, 0.5], where the first value is the curvature mean and the latter three values are the normal vectors. The curvature feature of the voxel centroid point after voxel filtering of the current voxel is [0.3, 0.2, 0.7, 0.6], where the first value is the curvature and the latter three values are the normal vectors. Then the curvature feature loss of the current voxel can be expressed as: (0.2 - 0.3) 2 + (0.1 - 0.2) 2 + (0.8 - 0.7) 2 + (0.5 - 0.6) 2 , and the entire formula (2) is to calculate the side length value l that minimizes the curvature feature loss of all voxels within the set side length range, and use it as the maximum voxel side length for point cloud voxel filtering.

[0040] In one embodiment, the initial generative model is trained according to the second training sample set to obtain a trained artificial intelligence generated content model, including: for each second training sample in the second training sample set, the following processing is performed: converting the target point cloud in the second training sample into a tensor format and performing feature extraction to obtain a feature map, performing upsampling, convolution, and normalization on the feature map to obtain a second generated image, using the target point cloud and the real 2D image as the first data pair, and using the target point cloud and the second generated image as the second data pair. The initial generative model is trained according to the first data pair set and the second data pair set until the loss function is minimized, and the model when the loss function is minimized is used as the trained artificial intelligence generated content model.

[0041] In this embodiment, the Generative Adversarial Network (GAN) model can be used as the AIGC generation algorithm, and the training process is mainly divided into an encoding process, a decoding process, and a discrimination process.

[0042] a. Encoding process: Convert the XYZ coordinate information of the target point cloud in the second training sample set into a tensor format as the input, and with the help of the PointNet++ deep neural network, extract features layer by layer through the Set Abstraction module, and finally output a feature map with a fixed dimension.

[0043] b. Decoding process: Perform operations such as upsampling, convolution, and normalization on the feature map output by the encoding process, and output a second generated image with a specific size.

[0044] c. Discrimination process: Use the target point cloud and the real 2D image as the first data pair, and use the target point cloud and the second generated image as the second data pair. Using these two data pairs as the input, use the discriminator to perform discrimination, and the loss in the discrimination process is shown in formula (3).

[0045] Formula (3) where x i is the three-dimensional target point cloud (the complete three-dimensional point cloud data of the i-th input), y i is the real 2D image, G(x i ) is the second generated image, L D represents the loss function value, N is the number of second training samples, is the discriminator function, whose value is the authenticity probability of the input data pair, ranging from 0 to 1, and log is the logarithm of the authenticity probability. The meaning of the entire formula (3) is to minimize the loss function and save the AIGC generation model with the minimum L D loss. At this time, the generated image inferred by the AIGC generation model is closest to the real image in the dataset.

[0046] Step 203: Detect the first generated image based on a preset defect detection model to obtain a first training sample.

[0047] The preset defect detection model can be the defect detection model currently used in the production line. If the defect detection model is directly used to detect defects in the target workpiece, there will be a problem of high false detection probability. Therefore, the first generated image can be simulated and detected first to obtain a first training sample, and then the defect detection model can be trained based on the first training sample to obtain an optimized detection model suitable for detecting defects in the target workpiece. Among them, the first training sample can be a misdetection image determined to be misdetected after manual review of the detection failure image.

[0048] In one embodiment, detecting the first generated image based on a preset defect detection model to obtain a first training sample includes: detecting the first generated image based on a preset defect detection model to obtain a detection failure image that fails the detection, obtaining a misdetection image after manual review of the detection failure image, and repeating the steps of obtaining the 3D pose point cloud of the three-dimensional model of the target workpiece at different pose points until the misdetection image after manual review of the detection failure image is obtained, until the number of repetitions reaches a preset number, and taking all misdetection images as the first training sample.

[0049] In this embodiment, the first generated image is detected based on a preset defect detection model to obtain a detection failure image that fails the detection (NG). The detection failure image is manually reviewed, and all misdetection images caused by modeling, occlusion, angle photography, etc. are picked out. To increase the quantity and quality of the first training sample, steps 201 to the step of obtaining the misdetection image can be repeatedly executed in a loop multiple times, for example, thousands of times can be repeated, and all obtained misdetection images are taken as the first training sample. In this embodiment, detecting the first generated image based on a preset defect detection model is equivalent to simulating the detection of the target workpiece based on the defect detection model. It should be noted that the purpose of training the AIGC generation model in the above embodiment is to provide new training samples for the defect detection model and improve the straight-through rate of the defect detection model when detecting the target workpiece.

[0050] Step 204: Train the defect detection model based on the first training sample to obtain an optimized detection model for the target workpiece.

[0051] Before the online detection of the target workpiece, the defect detection model can be trained in advance according to the first training sample to obtain an optimized detection model for the target workpiece. The defect detection model can be an AI (Artificial Intelligence) defect detection model. The optimized detection model trained based on the first training sample can solve the misdetection problem of the target workpiece. Therefore, when the target workpiece is officially put into production, the optimized detection model can be used to detect the defects of the target workpiece, which can improve the detection pass rate of the target workpiece and does not require iterative training of the defect detection model when the target workpiece is officially put into production, thus improving the defect detection efficiency.

[0052] In one embodiment, training the defect detection model based on the first training sample to obtain an optimized detection model for the target workpiece includes: training the defect detection model based on all misdetection images until all misdetection images can be recognized and passed by the defect detection model, and using the trained defect detection model as the optimized detection model for the target workpiece.

[0053] In this embodiment, training the defect detection model with all the misdetection images obtained from the simulated detection can perform iterative training on the defect detection model before the official online defect detection of the target workpiece. The optimized detection model obtained after training can improve the pass rate when detecting the target workpiece. Moreover, all iterative training is completed before the target workpiece is officially put into production, and there is no need to optimize the model after the official launch, thus improving the defect detection efficiency.

[0054] Step 205, detecting the defects of the target workpiece according to the optimized detection model.

[0055] This method can obtain the 3D pose point cloud of the 3D model of the target workpiece at different pose points before the online detection of the target workpiece, input the 3D pose point cloud into a pre-trained artificial intelligence generated content model to obtain the first generated image, and then detect the first generated image based on a preset defect detection model to obtain the first training sample. Thus, the defect detection model can be trained in advance according to the first training sample to obtain an optimized detection model for the target workpiece. When the target workpiece is officially put into production, the optimized detection model can be used to detect the defects of the target workpiece. Since the misdetection problem of the target workpiece is solved in advance, the detection pass rate of the target workpiece can be improved, and there is no need to perform iterative training on the defect detection model when the target workpiece is officially put into production, thus improving the defect detection efficiency.

[0056] In a specific embodiment, the workpiece detection method includes: Step S1, obtaining the real 2D image of the workpiece on the production line and the corresponding complete three-dimensional point cloud to construct a training data set.

[0057] Each model of the product has a 3D digital model (three-dimensional model) of the same proportional size, and this 3D digital model has complete and regular point cloud data. The 3D camera is fixedly set to obtain the three-dimensional point cloud data of the workpiece. The 2D camera is bound to the robotic arm and can obtain 2D images at different positions. Suppose the data of the first position of the workpiece is being collected now. Then the 3D camera will collect the three-dimensional point cloud data of this position, and at the same time, the 2D camera will also collect the real 2D image of this position.

[0058] At this time, the complete three-dimensional point cloud in the 2D image view can be obtained. The calculations to be done at this time are: transform the three-dimensional point cloud data obtained by the 3D camera through the transformation between the 3D camera coordinate system and the robotic arm coordinate system (the transformation of these coordinate systems in the calculation process requires the use of corresponding pose transformation matrices, and these pose transformation matrices can be directly obtained through calibration), unify it to the 2D camera view, and then with the help of camera parameters such as the depth of field of the 2D camera, use the principle of pinhole imaging to obtain the pose point cloud corresponding to the 2D image. Then, perform point cloud matching between the pose point cloud and the 3D digital model to obtain the complete three-dimensional point cloud corresponding to the 2D image of the first position.

[0059] For example, if 80 positions need to be collected for the current model of the workpiece, then 80 sets of data (real 2D images and corresponding complete three-dimensional point clouds) can be obtained according to the above method, and these data are used to construct the training data set.

[0060] Step S2: Use the training data set to train the AIGC generation model. Input the three-dimensional point cloud to this model, and it can generate the corresponding 2D image.

[0061] The trained AIGC generation model can accurately generate the corresponding 2D image according to the input three-dimensional point cloud.

[0062] Step S3: According to the three-dimensional model of the target workpiece to be put on the line for defect detection, obtain the pose point clouds of 200 three-dimensional models, input them to the AIGC generation model, and obtain 200 generated images.

[0063] Step S4: Use the AI defect detection model to infer 200 generated images and pick out the misdetected images.

[0064] Step S5: Collect all the misdetected images.

[0065] Step S6: Train the misdetected images with the AI defect detection model, optimize the misdetection situation, and improve the AI detection pass rate.

[0066] In this embodiment, a simulation evaluation is carried out before the target workpiece is put on the line, and the AI defect detection model is iteratively trained in advance according to the misdetected images of the simulation evaluation, which can effectively control the probability of misdetection of the AI defect detection model when the target workpiece is officially put on the line and improve the detection pass rate.

[0067] Based on the same inventive concept, the second embodiment of the present application provides a workpiece detection device, as Figure 3 The device includes: An acquisition module 301, configured to acquire 3D pose point clouds of a three-dimensional model of a target workpiece at different pose points; An image generation module 302, configured to input the 3D pose point cloud into a pre-trained artificial intelligence content generation model to obtain a first generated image; wherein, the artificial intelligence content generation model is a model for generating 2D images based on 3D point clouds; A simulation detection module 303, configured to detect the first generated image based on a preset defect detection model to obtain a first training sample; A defect detection model training module 304, configured to train the defect detection model based on the first training sample to obtain an optimized detection model for the target workpiece; A defect detection module 305, configured to perform defect detection on the target workpiece according to the optimized detection model.

[0068] Before the target workpiece is put on the line for detection, the device can acquire 3D pose point clouds of the three-dimensional model of the target workpiece at different pose points, input the 3D pose point cloud into a pre-trained artificial intelligence content generation model to obtain a first generated image, and then detect the first generated image based on a preset defect detection model to obtain a first training sample. Thus, the defect detection model can be trained in advance according to the first training sample to obtain an optimized detection model for the target workpiece. When the target workpiece is officially put on the line, the optimized detection model can be used to perform defect detection on the target workpiece. Since the problem of false detection of the target workpiece is solved in advance, the detection pass rate of the target workpiece can be improved, and it is not necessary to perform iterative training on the defect detection model when the target workpiece is officially put on the line, thereby improving the defect detection efficiency.

[0069] As Figure 4 shown, an embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113, and a communication bus 114. Among them, the processor 111, the communication interface 112, and the memory 113 complete mutual communication through the communication bus 114. The memory 113 is used to store a computer program; In an embodiment of the present application, when the processor 111 is configured to execute the program stored on the memory 113, it implements the workpiece detection method provided by any one of the foregoing method embodiments.

[0070] The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0071] The communication interface is used for communication between the above terminal and other devices.

[0072] The memory may include a Random Access Memory (RAM), or may also include a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0073] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0074] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the workpiece detection method provided in any one of the foregoing method embodiments.

[0075] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0076] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the related technology can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0077] It should be understood that the terms used herein are for the purpose of describing specific example embodiments only and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" as used herein may also include the plural forms. The terms "comprising", "including", "containing", and "having" are inclusive and thus specify the presence of the stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or their combinations. The method steps, processes, and operations described herein are not to be construed as necessarily requiring them to be performed in the particular order described or illustrated, unless the order of execution is explicitly stated. It should also be understood that additional or alternative steps may be used.

[0078] It should be understood that the specific embodiments described herein are merely for explaining the present application and are not used to limit the present application. In the description, the suffixes such as "module", "component", or "unit" used to represent elements are only for the convenience of explaining the present application and have no specific meaning in themselves. Therefore, "module", "component", or "unit" can be used interchangeably.

[0079] The above are only specific embodiments of the present application, enabling those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A workpiece detection method, characterized in that, The method includes: Obtaining 3D pose point clouds of a target workpiece at different pose points; Inputting the 3D pose point clouds into a pre-trained artificial intelligence generated content model to obtain a first generated image; wherein, the artificial intelligence generated content model is a model for generating 2D images based on 3D point clouds; Detecting the first generated image based on a preset defect detection model to obtain a first training sample; Training the defect detection model based on the first training sample to obtain an optimized detection model for the target workpiece; Performing defect detection on the target workpiece according to the optimized detection model.

2. The method according to claim 1, wherein The method further includes: obtaining the artificial intelligence generated content model; Wherein, the training process of the artificial intelligence generated content model includes: Sequentially collecting target data when a first workpiece is at each point to be collected; wherein, when the first workpiece is at any target point among the points to be collected, 3D point cloud data of the first workpiece is collected based on a 3D camera, a real 2D image of the first workpiece is collected based on a 2D camera, and a pose transformation matrix between the 3D point cloud data and the real 2D image is obtained; the 3D point cloud data, the real 2D image, and the pose transformation matrix are used as the target data at the target point; Performing the following processing on each group of the target data respectively to obtain a second training sample set: performing point cloud voxel filtering on the 3D point cloud data to obtain a downsampled point cloud; calculating the downsampled point cloud according to the pose transformation matrix to obtain a target point cloud unified to the perspective of the real 2D image, and using the target point cloud and the real 2D image as a second training sample; wherein, the second training sample set includes all the second training samples; Training an initial generation model according to the second training sample set to obtain the trained artificial intelligence generated content model.

3. The method according to claim 2, wherein Performing point cloud voxel filtering on the 3D point cloud data to obtain a downsampled point cloud, including: Dividing the 3D point cloud data into a grid to obtain a plurality of three-dimensional voxels; For each target three-dimensional voxel in the three-dimensional voxels, performing point cloud voxel filtering processing; wherein, the point cloud voxel filtering processing includes: determining the voxel centroid of the target three-dimensional voxel; using the point closest to the voxel centroid in the target three-dimensional voxel as the representative point of the target three-dimensional voxel; Using all the representative points as the downsampled point cloud.

4. The method according to claim 3, characterized in that Dividing the 3D point cloud data into a grid to obtain a plurality of three-dimensional voxels, including: According to the formula to determine the maximum voxel side length; where, lmax represents the maximum voxel side length, V represents all points of the current voxel divided according to the voxel side length l, T represents all voxels divided according to the voxel side length l, p V represents the point closest to the voxel centroid in the current voxel, f represents the curvature feature function, and λ represents the regularization term parameter; represents the square of the second norm; Dividing the 3D point cloud data into a grid according to the maximum voxel side length to obtain a plurality of the three-dimensional voxels; wherein, when dividing the grid according to the maximum voxel side length, the curvature feature loss of all the three-dimensional voxels is the smallest.

5. The method according to claim 2, wherein Training an initial generation model according to the second training sample set to obtain the trained artificial intelligence generated content model, including: For each second training sample in the second training sample set, the following processing is performed: converting the target point cloud in the second training sample into a tensor format and performing feature extraction to obtain a feature map; performing upsampling, convolution, and normalization on the feature map to obtain a second generated image; using the target point cloud and the real 2D image as a first data pair, and using the target point cloud and the second generated image as a second data pair; Training the initial generation model according to the first data pair set and the second data pair set until the loss function is minimized, and using the model when the loss function is minimized as the trained artificial intelligence generated content model.

6. The method according to claim 1, wherein Detecting the first generated image based on a preset defect detection model to obtain a first training sample, including: Detecting the first generated image based on a preset defect detection model to obtain a detection failure image that fails the detection; Obtaining a misdetection image after manually reviewing the detection failure image; Repeating the steps of obtaining the 3D pose point cloud of the three-dimensional model of the target workpiece at different pose points until the step of obtaining the misdetection image after manually reviewing the detection failure image, until the number of repetitions reaches a preset number; Using all the misdetection images as the first training sample.

7. The method according to claim 6, wherein Training the defect detection model based on the first training sample to obtain the optimized detection model of the target workpiece, including: Training the defect detection model based on all the misdetection images until all the misdetection images can be recognized and passed by the defect detection model, and using the trained defect detection model as the optimized detection model of the target workpiece.

8. A workpiece detection device, characterized in that, The device includes: An acquisition module, configured to acquire the 3D pose point cloud of the three-dimensional model of the target workpiece at different pose points; An image generation module, configured to input the 3D pose point cloud into a pre-trained artificial intelligence generated content model to obtain a first generated image; wherein, the artificial intelligence generated content model is a model for generating a 2D image based on a 3D point cloud; A simulation detection module, configured to detect the first generated image based on a preset defect detection model to obtain a first training sample; A defect detection model training module, configured to train the defect detection model based on the first training sample to obtain the optimized detection model of the target workpiece; A defect detection module, configured to detect defects of the target workpiece according to the optimized detection model.

9. An electronic device, characterized in that, Including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus; The memory is used for storing a computer program; When the processor is configured to execute the program stored in the memory, it implements the workpiece detection method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when executed by the processor, implements the workpiece detection method according to any one of claims 1-7.

Images