Workpiece detection method, device, electronic equipment and storage medium

Abstract

The present application relates to a workpiece detection method, device, electronic device and storage medium. The workpiece detection method can obtain a 3D posture point cloud of a three-dimensional model of a target workpiece at different posture points before the target workpiece is put on line for inspection, input the 3D posture point cloud into a pre-trained artificial intelligence generated content model to obtain a first generated image, and then detect the first generated image based on a preset defect detection model to obtain a first training sample, so that the defect detection model can be trained in advance according to the first training sample to obtain an optimized detection model for the target workpiece. When the target workpiece is officially put on line, the target workpiece can be inspected for defects according to the optimized detection model. Since the problem of false detection of the target workpiece is solved in advance, the detection pass rate of the target workpiece can be improved, and there is no need to iteratively train the defect detection model when the target workpiece is officially put on line, thereby improving the defect detection efficiency.

Description

Technical Field

[0001] The present application relates to the field of defect detection technology, and in particular to a workpiece detection method, device, electronic device and storage medium. Background Art

[0002] Before a workpiece leaves the factory, it needs to be inspected for defects. However, due to the diverse shapes of workpieces, differences in shape, camera angle, occlusion, etc., there is a high probability of false detection when new workpieces are put online for inspection. If the false detection problem cannot be solved in advance, the pass rate of each new workpiece put online will be low. The detection algorithm needs to be frequently iterated and trained to improve the pass rate, which affects the efficiency of defect detection. Summary of the Invention

[0003] The present application provides a workpiece detection method, device, electronic device and storage medium to solve the technical problem of how to improve the detection pass rate of workpieces.

[0004] In a first aspect, the present application provides a workpiece detection method, the method comprising:

[0005] Obtain 3D pose point clouds of the target workpiece's 3D model at different pose points;

[0006] Inputting the 3D pose point cloud into a pre-trained AI-generated content model to obtain a first generated image; wherein the AI-generated content model is a model for generating 2D images based on 3D point clouds;

[0007] Detecting the first generated image based on a preset defect detection model to obtain a first training sample;

[0008] Training the defect detection model based on the first training sample to obtain an optimized detection model for the target workpiece;

[0009] Defect detection is performed on the target workpiece according to the optimized detection model.

[0010] Optionally, the method further comprises: obtaining the artificial intelligence generated content model;

[0011] The training process of the artificial intelligence-generated content model includes:

[0012] sequentially collecting target data of the first workpiece when it is at each of the points to be collected; wherein, when the first workpiece is at any target point among the points to be collected, collecting 3D point cloud data of the first workpiece using a 3D camera, collecting a real 2D image of the first workpiece using a 2D camera, and obtaining a pose transformation matrix between the 3D point cloud data and the real 2D image; and using the 3D point cloud data, the real 2D image, and the pose transformation matrix as the target data at the target point;

[0013] Performing the following processing on each group of the target data to obtain a second training sample set: performing point cloud voxel filtering on the 3D point cloud data to obtain a downsampled point cloud; calculating the downsampled point cloud according to the pose transformation matrix to obtain a target point cloud unified to the perspective of the real 2D image, and using the target point cloud and the real 2D image as second training samples; wherein the second training sample set includes all the second training samples;

[0014] The initial generation model is trained according to the second training sample set to obtain the trained artificial intelligence generated content model.

[0015] Optionally, performing point cloud voxel filtering on the 3D point cloud data to obtain a downsampled point cloud includes:

[0016] Meshing the 3D point cloud data to obtain a plurality of three-dimensional voxels;

[0017] For each target 3D voxel in the 3D voxels, performing a point cloud voxel filtering process; wherein the point cloud voxel filtering process includes: determining the voxel centroid of the target 3D voxel; and using the point in the target 3D voxel that is closest to the voxel centroid as a representative point of the target 3D voxel;

[0018] All the representative points are used as the downsampled point cloud.

[0019] Optionally, the 3D point cloud data is meshed to obtain a plurality of three-dimensional voxels, including:

[0020] According to the formula Determine the maximum voxel side length; where lmax represents the maximum voxel side length, V represents all points of the current voxel divided by the voxel side length l, T represents all voxels divided by the voxel side length l, and p V represents the point closest to the voxel center in the current voxel, f represents the curvature characteristic function, and λ represents the regularization term parameter; represents the square of the two-norm;

[0021] The 3D point cloud data is meshed according to the maximum voxel side length to obtain a plurality of the three-dimensional voxels; wherein, when meshing is performed according to the maximum voxel side length, the curvature feature loss of all the three-dimensional voxels is minimized.

[0022] Optionally, training the initial generation model according to the second training sample set to obtain the trained artificial intelligence-generated content model includes:

[0023] The following processing is performed on each second training sample in the second training sample set: the target point cloud in the second training sample is converted into a tensor format and feature extracted to obtain a feature map; the feature map is upsampled, convolved, and normalized to obtain a second generated image; the target point cloud and the real 2D image are used as a first data pair, and the target point cloud and the second generated image are used as a second data pair;

[0024] The initial generation model is trained according to the first data pair set and the second data pair set until the loss function is minimized, and the model with the minimum loss function is used as the trained artificial intelligence generated content model.

[0025] Optionally, detecting the first generated image based on a preset defect detection model to obtain a first training sample includes:

[0026] Detecting the first generated image based on a preset defect detection model to obtain a detection failure image that fails the detection;

[0027] Obtaining a false detection image after manual review of the detection failed image;

[0028] Repeating the steps of obtaining the 3D posture point cloud of the three-dimensional model of the target workpiece at different posture points, and obtaining the false detection image after manual review of the failed detection image, until the number of repetitions reaches a preset number;

[0029] All the false detection images are used as the first training samples.

[0030] Optionally, training the defect detection model based on the first training sample to obtain an optimized detection model for the target workpiece includes:

[0031] The defect detection model is trained based on all the false detection images until all the false detection images can be recognized by the defect detection model, and the trained defect detection model is used as the optimized detection model for the target workpiece.

[0032] In a second aspect, the present application provides a workpiece detection device, comprising:

[0033] An acquisition module is used to obtain 3D posture point clouds of the three-dimensional model of the target workpiece at different posture points;

[0034] An image generation module, configured to input the 3D pose point cloud into a pre-trained AI-generated content model to obtain a first generated image; wherein the AI-generated content model is a model for generating 2D images based on 3D point clouds;

[0035] a simulation detection module, configured to detect the first generated image based on a preset defect detection model to obtain a first training sample;

[0036] a defect detection model training module, configured to train the defect detection model based on the first training sample to obtain an optimized detection model for the target workpiece;

[0037] A defect detection module is used to perform defect detection on the target workpiece according to the optimized detection model.

[0038] In a third aspect, the present application provides an electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0039] Memory for storing computer programs;

[0040] The processor is configured to implement the workpiece detection method described in any one of the embodiments of the first aspect when executing the program stored in the memory.

[0041] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the workpiece detection method as described in any one of the embodiments of the first aspect is implemented.

[0042] The above-mentioned technical solution provided by the embodiment of the present application has the following advantages compared with the existing technology: the method provided by the embodiment of the present application obtains the 3D posture point cloud of the three-dimensional model of the target workpiece at different posture points; inputs the 3D posture point cloud into a pre-trained artificial intelligence generated content model to obtain a first generated image; wherein, the artificial intelligence generated content model is a model for generating 2D images based on 3D point clouds; the first generated image is detected based on a preset defect detection model to obtain a first training sample; the defect detection model is trained based on the first training sample to obtain an optimized detection model of the target workpiece; and defect detection is performed on the target workpiece according to the optimized detection model. This method can obtain 3D posture point clouds of the three-dimensional model of the target workpiece at different posture points before the target workpiece is put on line for inspection, input the 3D posture point clouds into a pre-trained artificial intelligence generated content model to obtain a first generated image, and then detect the first generated image based on a preset defect detection model to obtain a first training sample, so that the defect detection model can be trained in advance according to the first training sample to obtain an optimized detection model for the target workpiece. When the target workpiece is officially put on line, the target workpiece can be inspected for defects according to the optimized detection model. Since the problem of false detection of the target workpiece is solved in advance, the inspection pass rate of the target workpiece can be improved, and there is no need to iteratively train the defect detection model when the target workpiece is officially put on line, thereby improving the defect detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0045] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0046] Figure 1 A system architecture diagram of a workpiece detection method provided in one embodiment of the present application;

[0047] Figure 2 A schematic diagram of a workpiece detection method according to an embodiment of the present application;

[0048] Figure 3 A schematic structural diagram of a workpiece detection device provided in one embodiment of the present application;

[0049] Figure 4 A schematic structural diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0050] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0051] The disclosure below provides many different embodiments or examples for implementing different structures of the present application. In order to simplify the disclosure of the present application, the components and settings of specific examples are described below. Of course, these are merely examples and are not intended to limit the present application. In addition, the present application may repeat reference numbers and / or letters in different examples. Such repetition is for the purpose of simplicity and clarity and does not in itself indicate the relationship between the various embodiments and / or settings discussed.

[0052] In order to solve the technical problem of how to improve the inspection pass rate of workpieces in the existing technology, the present application provides a workpiece detection method, device, electronic device and storage medium, which can train a defect detection model according to a first training sample before the target workpiece is put on line for inspection to obtain an optimized detection model for the target workpiece. When the target workpiece is officially put on line, defect detection can be performed on the target workpiece according to the optimized detection model. Since the problem of false detection of the target workpiece is solved in advance, the inspection pass rate of the target workpiece can be improved.

[0053] The first embodiment of the present application provides a workpiece detection method, which can be applied to Figure 1 The system architecture shown includes at least a data acquisition module 101 and a data processing module 102, which establish a communication connection. The data acquisition module 101 can acquire a 3D pose point cloud of the three-dimensional model of the target workpiece at different pose points and transmit the 3D pose point cloud to the data processing module 102. The data processing module 102 can input the 3D pose point cloud into a pre-trained artificial intelligence generated content model to obtain a first generated image. The first generated image is then detected based on a preset defect detection model to obtain a first training sample. The defect detection model is then trained based on the first training sample to obtain an optimized detection model for the target workpiece, so that defect detection can be performed on the target workpiece based on the optimized detection model.

[0054] Next, based on the system architecture, the workpiece detection method is described in detail. Figure 2 The workpiece detection method includes:

[0055] Step 201 : Acquire 3D point clouds of a three-dimensional model of a target workpiece at different posture points.

[0056] The target workpiece refers to a new workpiece that requires on-line defect inspection. The 3D model of the target workpiece can be a 3D design drawing of the target workpiece, which includes complete and regular point cloud data of the target workpiece. The software can adjust the pose of the 3D model and collect 3D pose point clouds of the 3D model at different pose points. For example, 3D pose point clouds of the 3D model can be collected at 200 different pose points.

[0057] In step 202 , the 3D pose point cloud is input into a pre-trained artificial intelligence generated content model to obtain a first generated image; wherein the artificial intelligence generated content model is a model for generating 2D images based on 3D point clouds.

[0058] The AIGC (Artificial Intelligence Generated Content) model can output a 2D image based on a 3D point cloud input. In this embodiment, a 3D pose point cloud is input into a pre-trained AIGC model to generate a first generated image. For example, inputting 200 3D pose point clouds can generate 200 first generated images.

[0059] In one embodiment, a 3D pose point cloud is input into a pre-trained artificial intelligence generated content model to obtain a first generated image, including: performing point cloud voxel filtering on the 3D pose point cloud to obtain downsampled point cloud data, and inputting the downsampled point cloud data into the artificial intelligence generated content model to obtain the first generated image.

[0060] In this embodiment, to increase data processing speed, the 3D pose point cloud can be first downsampled, for example, by performing point cloud voxel filtering to obtain downsampled point cloud data. This downsampled point cloud data is then input into the AIGC model to obtain the first generated image. Point cloud voxel filtering can be simply described as dividing the 3D pose point cloud into a three-dimensional voxel grid and replacing all points in the current voxel with the point closest to the voxel's centroid. Point cloud voxel filtering is also applied during the subsequent AIGC model training process and will not be described in detail here.

[0061] In one embodiment, the method further includes: obtaining an artificial intelligence-generated content model.

[0062] The training process of the AI-generated content model includes sequentially collecting target data of the first workpiece at each target location. When the first workpiece is at any target location among the target locations, 3D point cloud data of the first workpiece is collected using a 3D camera, a real 2D image of the first workpiece is collected using a 2D camera, and a pose transformation matrix between the 3D point cloud data and the real 2D image is obtained. The 3D point cloud data, the real 2D image, and the pose transformation matrix are used as the target data at the target location. Each set of target data is processed as follows to obtain a second training sample set: voxel filtering is performed on the 3D point cloud data to obtain a downsampled point cloud. The downsampled point cloud is calculated based on the pose transformation matrix to obtain a target point cloud unified to the perspective of the real 2D image. The target point cloud and the real 2D image are used as second training samples. The second training sample set includes all second training samples. The initial generative model is trained based on the second training sample set to obtain a trained AI-generated content model.

[0063] In this embodiment, the target data collected when the first workpiece is at each point to be collected is data collected in a real scene. For example, the target data of the first workpiece undergoing defect inspection on the production line is collected to train the AIGC generation model based on the collected target data. The points to be collected may include, for example, 80 points with different postures. Then, when the first workpiece is at any target point, the 3D point cloud data of the first workpiece is collected using a 3D camera, the real 2D image of the first workpiece is collected using a 2D camera, and the pose transformation matrix between the 3D point cloud data and the real 2D image is obtained. The 3D point cloud data, the real 2D image, and the pose transformation matrix are used as the target data at the target point. The above steps are repeated until the target data of the first workpiece at each point to be collected is collected in sequence. After completing the collection of the target data for each point, the 3D point cloud data in the target data is subjected to point cloud voxel filtering to obtain a downsampled point cloud. The downsampled point cloud is then calculated based on the pose transformation matrix to obtain a target point cloud unified to the perspective of the real 2D image. The target point cloud and the real 2D image are used as the second training samples. After processing each target data in sequence, all the obtained second training samples are used as the second training sample set, and the initial generation model is trained according to the second training sample set to obtain the trained AIGC generation model.

[0064] In one embodiment, point cloud voxel filtering is performed on 3D point cloud data to obtain a downsampled point cloud, including: gridding the 3D point cloud data to obtain multiple three-dimensional voxels; performing point cloud voxel filtering processing on each target three-dimensional voxel in the three-dimensional voxels; wherein the point cloud voxel filtering processing includes: determining the voxel centroid of the target three-dimensional voxel; using the point of the voxel centroid closest to the target three-dimensional voxel as the representative point of the target three-dimensional voxel; and using all representative points as the downsampled point cloud.

[0065] In this embodiment, point cloud voxel filtering can be performed on the three-dimensional voxels obtained by grid division using the following formula (1).

[0066] Formula (1)

[0067] Among them, V represents all points in the current voxel; g V Indicates the voxel center of the current voxel; p V Represents the point closest to the voxel center in the current voxel; p is any point in the current voxel, and the current voxel refers to the three-dimensional voxel currently performing point cloud voxel filtering.

[0068] In this embodiment, when performing point cloud voxel filtering, the point of the nearest voxel centroid in the target three-dimensional voxel is used as the representative point of the target three-dimensional voxel. Since the point of the nearest voxel centroid itself is a point in the original 3D point cloud data, the characteristics of the 3D point cloud data can be better preserved.

[0069] In one embodiment, 3D point cloud data is meshed to obtain a plurality of 3D voxels, including:

[0070] The maximum voxel side length is determined according to the following formula (2), and the 3D point cloud data is meshed according to the maximum voxel side length to obtain multiple three-dimensional voxels. Among them, when meshing according to the maximum voxel side length, the curvature feature loss of all three-dimensional voxels is minimized.

[0071] Formula (2)

[0072] Among them, lmax represents the maximum voxel side length, V represents all points of the current voxel divided according to the voxel side length l, T represents all voxels divided according to the voxel side length l, p V represents the point closest to the voxel center in the current voxel, f represents the curvature characteristic function, and λ represents the regularization term parameter; It represents the square of the second norm, which is used to reflect the difference between the curvature characteristics of the sampled point cloud and the original point cloud.

[0073] In this embodiment, since the point cloud data volume is large and has many features, it has a great impact on the training speed. Therefore, the selection of the voxel side length lmax is crucial. The method of formula (2) is used to determine the specific voxel side length. On the basis of retaining the three-dimensional point cloud features, the point cloud downsampling is achieved to the greatest extent, thereby improving the AIGC model training speed.

[0074] For example, the curvature feature of the current voxel V is [0.2, 0.1, 0.8, 0.5], where the first value is the mean curvature and the next three values ​​are the normal vector. The curvature feature of the voxel center after voxel filtering is [0.3, 0.2, 0.7, 0.6], where the first value is the curvature and the next three values ​​are the normal vector. The curvature feature loss of the current voxel can be expressed as: (0.2-0.3) 2 + (0.1-0.2) 2 + (0.8-0.7) 2 + (0.5-0.6) 2 , the whole formula (2) is to calculate the side length value l that minimizes the loss of all voxel curvature features within the set side length range, and use it as the maximum voxel side length for point cloud voxel filtering.

[0075] In one embodiment, the initial generative model is trained based on a second training sample set to obtain a trained AI-generated content model, including: performing the following processing on each second training sample in the second training sample set: converting the target point cloud in the second training sample into a tensor format and performing feature extraction to obtain a feature map; upsampling, convolution, and normalizing the feature map to obtain a second generated image; using the target point cloud and the real 2D image as a first data pair, and using the target point cloud and the second generated image as a second data pair. The initial generative model is trained based on the first data pair set and the second data pair set until the loss function is minimized, and the model with the minimum loss function is used as the trained AI-generated content model.

[0076] In this embodiment, a generative adversarial network model (GAN model for short) can be used as the AIGC generation algorithm, and the training process is mainly divided into an encoding process, a decoding process, and a discrimination process.

[0077] a. Encoding process: The XYZ coordinate information of the target point cloud in the second training sample set is converted into a tensor format as input. With the help of the PointNet++ deep neural network, the set abstraction module is used to extract features hierarchically, and finally a fixed-dimensional feature map is output.

[0078] b. Decoding process: Upsampling, convolution, and normalization are performed on the feature map output by the encoding process to output a second generated image of a specific size.

[0079] c. Discrimination process: The target point cloud and the real 2D image are taken as the first data pair, and the target point cloud and the second generated image are taken as the second data pair. These two data pairs are used as input and the discriminator is used for discrimination. The loss of the discrimination process is shown in formula (3).

[0080] Formula (3)

[0081] Among them, x i is the three-dimensional target point cloud (the complete three-dimensional point cloud data of the i-th input), y i is a real 2D image, G(x i ) is the second generated image, L D Represents the loss function value, N is the number of second training samples, is the discriminator function, whose value is the authenticity probability of the input data pair, the size is 0~1, and log is the logarithm of the authenticity probability. The meaning of the entire formula (3) is to minimize the loss function and save L D The AIGC generation model with the smallest loss. At this time, the generated image inferred by the AIGC generation model is closest to the real image in the dataset.

[0082] Step 203 : Detect the first generated image based on a preset defect detection model to obtain a first training sample.

[0083] The preset defect detection model can be the defect detection model currently used in the production line. If the defect detection model is directly used to detect defects on the target workpiece, there will be a high probability of false detection. Therefore, it is possible to first simulate the detection of the first generated image to obtain a first training sample. The defect detection model is then trained based on the first training sample to obtain an optimized detection model suitable for defect detection of the target workpiece. The first training sample can be a false detection image that is determined to be a false detection after manual review of the failed detection image.

[0084] In one embodiment, the first generated image is detected based on a preset defect detection model to obtain a first training sample, including: detecting the first generated image based on the preset defect detection model to obtain a failed detection image that does not pass the detection, obtaining a false detection image after manual review of the failed detection image, repeatedly performing the steps of obtaining a 3D posture point cloud of the three-dimensional model of the target workpiece at different posture points, and obtaining a false detection image after manual review of the failed detection image, until the number of repetitions reaches a preset number, and using all false detection images as the first training sample.

[0085] In this embodiment, the first generated image is inspected based on a preset defect detection model to obtain a failed detection image that does not pass the inspection (NG). The failed detection image is manually reviewed to pick out all false detection images caused by shape, occlusion, angle shooting, etc. To improve the quantity and quality of the first training samples, step 201 to the step of obtaining false detection images can be repeatedly executed multiple times. For example, it can be repeated thousands of times, and all false detection images obtained are used as the first training samples. In this embodiment, the first generated image is inspected based on the preset defect detection model, which is equivalent to performing a simulated inspection of the target workpiece based on the defect detection model. It should be noted that the purpose of training the AIGC generation model in the above embodiment is to provide new training samples for the defect detection model and improve the pass rate of the defect detection model when detecting the target workpiece.

[0086] Step 204 : training the defect detection model based on the first training sample to obtain an optimized detection model for the target workpiece.

[0087] Before the target workpiece goes online for inspection, the defect detection model is pre-trained based on the first training sample to obtain an optimized detection model for the target workpiece. The defect detection model can be an AI (Artificial Intelligence) defect detection model. The optimized detection model trained based on the first training sample can resolve the problem of false detection of the target workpiece. When the target workpiece is officially launched, defect detection can be performed on the target workpiece based on the optimized detection model, thereby improving the target workpiece's inspection pass rate and eliminating the need for iterative training of the defect detection model when the target workpiece is officially launched, thereby improving defect detection efficiency.

[0088] In one embodiment, a defect detection model is trained based on a first training sample to obtain an optimized detection model for the target workpiece, including: training the defect detection model based on all false detection images until all false detection images can be recognized by the defect detection model, and using the trained defect detection model as the optimized detection model for the target workpiece.

[0089] In this embodiment, the defect detection model is trained with all the false detection images obtained by the simulated detection. The defect detection model can be iteratively trained before the target workpiece is officially put online for defect detection. The optimized detection model obtained after the training is completed can improve the pass rate when detecting the target workpiece. Moreover, all iterative training is completed before the target workpiece is officially put online, and there is no need to optimize the model after it is officially put online, thereby improving the efficiency of defect detection.

[0090] Step 205 : performing defect detection on the target workpiece according to the optimized detection model.

[0091] This method can obtain 3D posture point clouds of the three-dimensional model of the target workpiece at different posture points before the target workpiece is put on line for inspection, input the 3D posture point clouds into a pre-trained artificial intelligence generated content model to obtain a first generated image, and then detect the first generated image based on a preset defect detection model to obtain a first training sample, so that the defect detection model can be trained in advance according to the first training sample to obtain an optimized detection model for the target workpiece. When the target workpiece is officially put on line, the target workpiece can be inspected for defects according to the optimized detection model. Since the problem of false detection of the target workpiece is solved in advance, the inspection pass rate of the target workpiece can be improved, and there is no need to iteratively train the defect detection model when the target workpiece is officially put on line, thereby improving the defect detection efficiency.

[0092] In a specific embodiment, the workpiece detection method includes:

[0093] Step S1: Obtain the real 2D image of the workpiece on the production line and the corresponding complete 3D point cloud to construct a training dataset.

[0094] Each product model has a 3D digital model (three-dimensional model) of the same scale, which contains complete and regular point cloud data. A fixed 3D camera is used to acquire 3D point cloud data of the workpiece. A 2D camera, attached to the robotic arm, can capture 2D images of different points. For example, if data is currently being collected for the first point on the workpiece, the 3D camera will capture the 3D point cloud data for that point, while the 2D camera will also capture the actual 2D image of that point.

[0095] At this point, a complete three-dimensional point cloud can be obtained from the perspective of the 2D image. The calculation that needs to be done at this time is: the three-dimensional point cloud data obtained by the 3D camera is transformed through the 3D camera coordinate system and the robotic arm coordinate system (the transformation of these coordinate systems during the calculation process requires the use of corresponding pose transformation matrices, which can be directly obtained through calibration), and unified to the 2D camera perspective. Then, with the help of camera parameters such as the 2D camera depth of field, the pinhole imaging principle is used to obtain the pose point cloud corresponding to the 2D image. Then, the pose point cloud and the 3D digital model are matched to obtain the complete three-dimensional point cloud corresponding to the 2D image of the first point.

[0096] For example, if the current model of the workpiece requires the collection of 80 points, then according to the above method, 80 sets of data (real 2D images and corresponding complete 3D point clouds) can be obtained, and these data are used to construct the training data set.

[0097] In step S2, the AIGC generation model is trained using the training data set. The model is fed with a 3D point cloud to generate a corresponding 2D image.

[0098] The trained AIGC generation model can accurately generate corresponding 2D images based on the input 3D point cloud.

[0099] In step S3, based on the three-dimensional model of the target workpiece to be put online for defect detection, 200 posture point clouds of the three-dimensional model are obtained and input into the AIGC generation model to obtain 200 generated images.

[0100] In step S4, the AI ​​defect detection model is used to infer 200 generated images and to select falsely detected images.

[0101] Step S5: Collect all misdetected images.

[0102] Step S6: The AI ​​defect detection model is trained on false positive images to optimize false positives and improve the AI ​​detection pass rate.

[0103] This embodiment performs a simulation evaluation before the target workpiece goes online, and iteratively trains the AI ​​defect detection model in advance based on the false detection images from the simulation evaluation. This can effectively control the probability of false detection by the AI ​​defect detection model when the target workpiece is officially put online, thereby improving the detection pass rate.

[0104] Based on the same technical concept, the second embodiment of the present application provides a workpiece detection device, such as Figure 3 , the device comprises:

[0105] An acquisition module 301 is used to acquire a 3D pose point cloud of a three-dimensional model of a target workpiece at different pose points;

[0106] An image generation module 302 is configured to input the 3D pose point cloud into a pre-trained AI-generated content model to obtain a first generated image; wherein the AI-generated content model is a model for generating 2D images based on 3D point clouds;

[0107] A simulation detection module 303 is configured to detect the first generated image based on a preset defect detection model to obtain a first training sample;

[0108] A defect detection model training module 304 is configured to train the defect detection model based on the first training sample to obtain an optimized detection model for the target workpiece;

[0109] The defect detection module 305 is configured to perform defect detection on the target workpiece according to the optimized detection model.

[0110] Before the target workpiece is put on line for inspection, the device can obtain the 3D posture point cloud of the three-dimensional model of the target workpiece at different posture points, input the 3D posture point cloud into a pre-trained artificial intelligence generated content model to obtain a first generated image, and then detect the first generated image based on a preset defect detection model to obtain a first training sample, so that the defect detection model can be trained in advance according to the first training sample to obtain an optimized detection model for the target workpiece. When the target workpiece is officially put on line, the target workpiece can be inspected for defects according to the optimized detection model. Since the problem of false detection of the target workpiece is solved in advance, the pass rate of the target workpiece inspection can be improved, and there is no need to iteratively train the defect detection model when the target workpiece is officially put on line, thereby improving the defect detection efficiency.

[0111] like Figure 4 As shown, an embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0112] Memory 113, for storing computer programs;

[0113] In one embodiment of the present application, the processor 111 is configured to implement the workpiece detection method provided by any one of the aforementioned method embodiments when executing a program stored in the memory 113 .

[0114] The communication bus mentioned in the terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0115] The communication interface is used for communication between the above terminal and other devices.

[0116] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0117] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0118] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the workpiece detection method provided by any of the aforementioned method embodiments is implemented.

[0119] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0120] Through the description of the above embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a general hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the relevant technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0121] It should be understood that the terms used herein are for the purpose of describing specific example embodiments only and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "comprise", "include", "contain" and "have" are inclusive and therefore specify the presence of stated features, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring them to be performed in the specific order described or illustrated, unless the order of execution is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0122] It should be understood that the specific embodiments described herein are intended only to illustrate the present application and are not intended to limit the present application. In the description, suffixes such as "module," "component," or "unit" used to represent elements are used solely to facilitate the description of the present application and have no specific meaning. Therefore, "module," "component," or "unit" may be used interchangeably.

[0123] The foregoing is merely a list of specific embodiments of the present application, intended to enable those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the broadest scope consistent with the principles and novel features of the present application.

Claims

1. A workpiece detection method, characterized in that: The method comprises: Obtain 3D pose point clouds of the target workpiece's 3D model at different pose points; Inputting the 3D pose point cloud into a pre-trained AI-generated content model to obtain a first generated image; wherein the AI-generated content model is a model for generating 2D images based on 3D point clouds; Detecting the first generated image based on a preset defect detection model to obtain a first training sample; Training the defect detection model based on the first training sample to obtain an optimized detection model for the target workpiece; Performing defect detection on the target workpiece according to the optimized detection model; The method further comprises: obtaining the artificial intelligence generated content model; The training process of the artificial intelligence-generated content model includes: sequentially collecting target data of the first workpiece when it is at each of the points to be collected; wherein, when the first workpiece is at any target point among the points to be collected, collecting 3D point cloud data of the first workpiece using a 3D camera, collecting a real 2D image of the first workpiece using a 2D camera, and obtaining a pose transformation matrix between the 3D point cloud data and the real 2D image; and using the 3D point cloud data, the real 2D image, and the pose transformation matrix as the target data at the target point; Performing the following processing on each group of the target data to obtain a second training sample set: performing point cloud voxel filtering on the 3D point cloud data to obtain a downsampled point cloud; calculating the downsampled point cloud according to the pose transformation matrix to obtain a target point cloud unified to the perspective of the real 2D image, and using the target point cloud and the real 2D image as second training samples; wherein the second training sample set includes all the second training samples; Training the initial generation model according to the second training sample set to obtain the trained artificial intelligence generated content model; The step of performing point cloud voxel filtering on the 3D point cloud data to obtain a downsampled point cloud comprises: Meshing the 3D point cloud data to obtain a plurality of three-dimensional voxels; For each target 3D voxel in the 3D voxels, performing a point cloud voxel filtering process; wherein the point cloud voxel filtering process includes: determining the voxel centroid of the target 3D voxel; and using the point in the target 3D voxel that is closest to the voxel centroid as a representative point of the target 3D voxel; Taking all the representative points as the downsampled point cloud; The 3D point cloud data is meshed to obtain a plurality of three-dimensional voxels, including: According to the formula Determine the maximum voxel side length; where lmax represents the maximum voxel side length, V represents all points of the current voxel divided by the voxel side length l, T represents all voxels divided by the voxel side length l, and p V represents the point closest to the voxel center in the current voxel, f represents the curvature characteristic function, and λ represents the regularization term parameter; represents the square of the two-norm; The 3D point cloud data is meshed according to the maximum voxel side length to obtain a plurality of the three-dimensional voxels; wherein, when meshing is performed according to the maximum voxel side length, the curvature feature loss of all the three-dimensional voxels is minimized.

2. The method according to claim 1, characterized in that The initial generation model is trained according to the second training sample set to obtain the trained artificial intelligence generated content model, including: The following processing is performed on each second training sample in the second training sample set: the target point cloud in the second training sample is converted into a tensor format and feature extracted to obtain a feature map; the feature map is upsampled, convolved, and normalized to obtain a second generated image; the target point cloud and the real 2D image are used as a first data pair, and the target point cloud and the second generated image are used as a second data pair; The initial generation model is trained according to the first data pair set and the second data pair set until the loss function is minimized, and the model with the minimum loss function is used as the trained artificial intelligence generated content model.

3. The method according to claim 1, characterized in that Detecting the first generated image based on a preset defect detection model to obtain a first training sample includes: Detecting the first generated image based on a preset defect detection model to obtain a detection failure image that fails the detection; Obtaining a false detection image after manual review of the detection failed image; Repeating the steps of obtaining the 3D posture point cloud of the three-dimensional model of the target workpiece at different posture points, and obtaining the false detection image after manual review of the failed detection image, until the number of repetitions reaches a preset number; All the false detection images are used as the first training samples.

4. The method according to claim 3, characterized in that Training the defect detection model based on the first training sample to obtain an optimized detection model for the target workpiece includes: The defect detection model is trained based on all the false detection images until all the false detection images can be recognized by the defect detection model, and the trained defect detection model is used as the optimized detection model for the target workpiece.

5. A workpiece detection device, characterized in that: The device comprises: An acquisition module is used to obtain 3D posture point clouds of the three-dimensional model of the target workpiece at different posture points; An image generation module, configured to input the 3D pose point cloud into a pre-trained AI-generated content model to obtain a first generated image; wherein the AI-generated content model is a model for generating 2D images based on 3D point clouds; a simulation detection module, configured to detect the first generated image based on a preset defect detection model to obtain a first training sample; a defect detection model training module, configured to train the defect detection model based on the first training sample to obtain an optimized detection model for the target workpiece; a defect detection module, configured to perform defect detection on the target workpiece according to the optimized detection model; The device also includes a model training module, which is used to obtain the artificial intelligence generated content model; The training process of the artificial intelligence-generated content model includes: sequentially collecting target data of the first workpiece when it is at each of the points to be collected; wherein, when the first workpiece is at any target point among the points to be collected, collecting 3D point cloud data of the first workpiece using a 3D camera, collecting a real 2D image of the first workpiece using a 2D camera, and obtaining a pose transformation matrix between the 3D point cloud data and the real 2D image; and using the 3D point cloud data, the real 2D image, and the pose transformation matrix as the target data at the target point; Performing the following processing on each group of the target data to obtain a second training sample set: performing point cloud voxel filtering on the 3D point cloud data to obtain a downsampled point cloud; calculating the downsampled point cloud according to the pose transformation matrix to obtain a target point cloud unified to the perspective of the real 2D image, and using the target point cloud and the real 2D image as second training samples; wherein the second training sample set includes all the second training samples; Training the initial generation model according to the second training sample set to obtain the trained artificial intelligence generated content model; The step of performing point cloud voxel filtering on the 3D point cloud data to obtain a downsampled point cloud comprises: Meshing the 3D point cloud data to obtain a plurality of three-dimensional voxels; For each target 3D voxel in the 3D voxels, performing a point cloud voxel filtering process; wherein the point cloud voxel filtering process includes: determining the voxel centroid of the target 3D voxel; and using the point in the target 3D voxel that is closest to the voxel centroid as a representative point of the target 3D voxel; Taking all the representative points as the downsampled point cloud; The 3D point cloud data is meshed to obtain a plurality of three-dimensional voxels, including: According to the formula Determine the maximum voxel side length; where lmax represents the maximum voxel side length, V represents all points of the current voxel divided by the voxel side length l, T represents all voxels divided by the voxel side length l, and p V represents the point closest to the voxel center in the current voxel, f represents the curvature characteristic function, and λ represents the regularization term parameter; represents the square of the two-norm; The 3D point cloud data is meshed according to the maximum voxel side length to obtain a plurality of the three-dimensional voxels; wherein, when meshing is performed according to the maximum voxel side length, the curvature feature loss of all the three-dimensional voxels is minimized.

6. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; The processor is configured to implement the workpiece detection method according to any one of claims 1 to 4 when executing a program stored in the memory.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the workpiece detection method according to any one of claims 1 to 4 is implemented.

Images