Three-dimensional defect detection method, device and electronic equipment

Abstract

The application discloses a three-dimensional defect detection method and device and electronic equipment, comprising: obtaining a point cloud image of an object to be detected; obtaining three-dimensional point cloud features of the point cloud image based on a preprocessing module of a three-dimensional defect detection model; processing the three-dimensional point cloud features based on a detection module of the three-dimensional defect detection model to obtain a three-dimensional defect detection result; wherein the three-dimensional defect detection model takes a three-dimensional defect detection task as a downstream task of the preprocessing module, and adjusts original parameters of the preprocessing module and the detection module to obtain a model corresponding to the downstream task.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically to a three-dimensional defect detection method, apparatus, and electronic device. Background Technology

[0002] In 3D (3D) industrial visual inspection, 3D data acquisition is becoming increasingly easy, and a large amount of 3D industrial data can be obtained. If a deep learning network is to be generated from 3D industrial data, the 3D industrial data needs to be labeled so that the labeled data can be used as training samples for the deep learning network. However, labeling 3D data is time-consuming, costly, and prone to errors, which can easily result in a small amount of effective labeled data. This leads to a small amount of training sample data for learning through labels, resulting in lower accuracy of the final deep learning network. Summary of the Invention

[0003] In view of the above, this application provides the following technical solution:

[0004] A three-dimensional defect detection method, comprising:

[0005] Obtain the point cloud image of the object to be detected;

[0006] The preprocessing module based on the three-dimensional defect detection model obtains the three-dimensional point cloud features of the point cloud image.

[0007] The detection module based on the three-dimensional defect detection model processes the three-dimensional point cloud features to obtain the three-dimensional defect detection results.

[0008] The three-dimensional defect detection model is obtained by adjusting the original parameters of the preprocessing module and the detection module using the target point cloud image corresponding to the downstream task, with the three-dimensional defect detection task as the downstream task.

[0009] Optionally, it also includes:

[0010] A preprocessing model is obtained by training based on a first image set, which is an unlabeled point cloud image set. The preprocessing model is used to reconstruct point cloud data. The preprocessing model includes at least an encoder, a decoder, and a task module. The encoder is used to encode and obtain three-dimensional point cloud features.

[0011] The initial encoding parameters of the encoder in the preprocessed model are applied to the network model structure corresponding to the 3D defect detection task, and the initial encoding parameters and the initial detection parameters of the detection module are adjusted using the second image set to obtain the 3D defect detection model.

[0012] The three-dimensional defect detection model includes a preprocessing module and a detection module. The preprocessing module is determined based on the adjustment of the initial encoding parameters of the encoder. The detection module is used to process the three-dimensional point cloud features output by the preprocessing module to obtain the three-dimensional defect detection result. The second image set is a labeled target point cloud image dataset corresponding to the three-dimensional defect detection task.

[0013] Optionally, the step of training the preprocessed model based on the first image set includes:

[0014] The original point cloud of each point cloud image in the first image set is divided into point cloud blocks;

[0015] A specific point cloud block in the point cloud block is masked to obtain a masked point cloud block;

[0016] The encoder encodes the original point cloud features and the point cloud features of unmasked point cloud blocks in each point cloud image to obtain the encoded features.

[0017] The encoded features are input into the decoder for decoding to obtain the decoded features;

[0018] Based on the task module, the decoded features are reconstructed into masked point cloud blocks to obtain the reconstructed masked point cloud blocks;

[0019] A preprocessing model is trained based on the deviation data between the reconstructed masked point cloud block and the original point cloud block before it was masked.

[0020] Optionally, the preprocessing model is trained based on the deviation between the reconstructed masked point cloud data and the original point cloud data before the masked point cloud data was unmasked, including:

[0021] Obtain the loss function;

[0022] The initial model parameters of the preprocessing model are adjusted based on the reconstructed masked point cloud blocks until the loss function converges, thus completing the training of the preprocessing model.

[0023] The loss function is determined based on the data relationship between the original point cloud data of the masked point cloud blocks and the point cloud data of the reconstructed masked point cloud blocks.

[0024] Optionally, the step of masking a specific point cloud block within the point cloud block to obtain a masked point cloud block includes:

[0025] Based on the image features of the object to be detected, key regions are identified;

[0026] The point cloud blocks belonging to the key region are masked to obtain masked point cloud blocks.

[0027] Optionally, the step of dividing the original point cloud in each point cloud image of the first image set to obtain point cloud blocks includes:

[0028] Sample the cloud image of each point in the first image set to obtain multiple pixel center points;

[0029] Multiple point cloud blocks are formed by scanning multiple neighboring pixels around the center point of each pixel.

[0030] Optionally, the second image set includes multiple target point cloud images, and each target point cloud image is labeled with a corresponding target 3D defect detection result. The step of adjusting the initial encoding parameters and the initial detection parameters of the detection module using the second image set to obtain a 3D defect detection model includes:

[0031] The initial detection model is used to detect each target point cloud image in the second image set to obtain the initial three-dimensional defect detection result corresponding to each target point cloud image. The initial detection model is a model constructed using a network model structure corresponding to the three-dimensional defect detection task, based on an encoder with initial encoding parameters and a detection module with initial detection parameters.

[0032] Based on the deviation data between the initial 3D defect detection result and the target 3D defect detection result, the initial encoding parameters in the initial detection model and the initial detection parameters of the detection module are adjusted to obtain the 3D defect detection model.

[0033] A three-dimensional defect detection device, comprising:

[0034] The first acquisition unit is used to acquire the point cloud image of the object to be detected;

[0035] The second acquisition unit is used to obtain the three-dimensional point cloud features of the point cloud image based on the preprocessing module of the three-dimensional defect detection model.

[0036] The processing unit is used to process the three-dimensional point cloud features based on the detection module of the three-dimensional defect detection model to obtain the three-dimensional defect detection result;

[0037] The three-dimensional defect detection model is obtained by adjusting the original parameters of the preprocessing module and the detection module using the target point cloud image corresponding to the downstream task, with the three-dimensional defect detection task as the downstream task.

[0038] A storage medium having a computer program stored thereon, which, when executed by a processor, implements the three-dimensional defect detection method as described in any one of the above descriptions.

[0039] An electronic device, comprising:

[0040] Memory, used to store applications and the data generated by the running of the applications;

[0041] A processor for executing the application to implement the three-dimensional defect detection method as described in any one of the above descriptions. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0043] Figure 1 A flowchart illustrating a three-dimensional defect detection method provided in an embodiment of this application;

[0044] Figure 2 A flowchart of a three-dimensional self-supervised training process is provided for embodiments of this application;

[0045] Figure 3 A 3D adhesive path segmentation effect diagram provided for an embodiment of this application;

[0046] Figure 4 This is a schematic diagram of the structure of a three-dimensional defect detection device provided in an embodiment of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0049] In industrial visual inspection, deep learning-based 3D defect detection methods often lack the massive amounts of labeled 3D data needed to train models, resulting in low accuracy in 3D defect detection. This is because only a large amount of labeled data can make the model learning process more complete during training. However, in the field of 3D defect detection, due to the need to annotate defect information in all three dimensions, or the scarcity of 3D data containing defects, the trained models tend to have poor accuracy, failing to meet practical application requirements. Furthermore, this necessitates significant manual annotation costs, reducing processing efficiency. Therefore, this application provides a 3D defect detection method that improves the accuracy and efficiency of 3D defect detection.

[0050] See Figure 1 This is a flowchart illustrating a three-dimensional defect detection method provided in an embodiment of this application. The method may include the following steps:

[0051] S101. Obtain the point cloud image of the object to be detected.

[0052] The object to be inspected can be any industrial component or instrument in an industrial product that requires defect detection. The point cloud image of the object to be inspected refers to a 3D point cloud image, which can be an image acquired by a 3D image acquisition device or a depth camera.

[0053] S102. The preprocessing module based on the three-dimensional defect detection model obtains the three-dimensional point cloud features of the point cloud image.

[0054] S103. The detection module based on the three-dimensional defect detection model processes the three-dimensional point cloud features to obtain the three-dimensional defect detection results.

[0055] The 3D defect detection model is a downstream task of the preprocessing module, where the 3D defect detection task is used as a downstream task. The model is obtained by adjusting the original parameters of the preprocessing and detection modules using the target point cloud image corresponding to the downstream task. In this embodiment of the self-supervised learning system based on point cloud images, a new downstream task-like preprocessing model is introduced to mine implicit label information from the point cloud image itself to supervise the training of the 3D defect detection model, thereby greatly reducing the 3D defect detection model's dependence on manually labeled data. The preprocessing module in the 3D defect detection model is determined based on parameter adjustments made to the encoder in the preprocessing model, and the detection module is determined based on the 3D defect detection task. For example, if the 3D defect detection task is to identify defective parts, the corresponding detection module outputs the position information of the detection box corresponding to the defective parts; or, if the 3D defect detection task is to determine the defect category, the detection module outputs the probability information of each defect category.

[0056] In one embodiment of this application, the method further includes: training based on a first image set to obtain a preprocessed model; applying the initial encoding parameters of the encoder in the obtained preprocessed model to the network model structure corresponding to the three-dimensional defect detection task; and adjusting the initial encoding parameters and the initial detection parameters of the detection module using a second image set to obtain a three-dimensional defect detection model.

[0057] The preprocessing model is a model that processes point cloud images; for example, it can be a point cloud reconstruction model. This preprocessing model can be a self-supervised learning model, meaning that for unlabeled data, a preprocessing task is designed to mine the data's inherent representational features as supervisory information, thereby improving the model's feature extraction capabilities. Specifically, when training with the first image set (an unlabeled set of point cloud images), the preprocessing model is used for point cloud data reconstruction. This preprocessing model includes at least an encoder, a decoder, and a task module, where the encoder is used to encode and obtain 3D point cloud features. The 3D defect detection model includes a preprocessing module and a detection module. The preprocessing module is determined after adjusting the initial encoding parameters of the encoder, and the detection module processes the 3D point cloud features output by the preprocessing module to obtain the 3D defect detection results. The second image set is a labeled target point cloud image dataset corresponding to the 3D defect detection task, and the number of images in the second image set is relatively small. Therefore, in practical 3D defect detection applications, the encoder model parameters in the pre-trained pre-processed model can be loaded, so that downstream 3D defect detection tasks only need to fine-tune the model parameters with less labeled data, thereby reducing the cost of manually annotating 3D data and improving the accuracy of 3D industrial defect detection.

[0058] This application provides a general model pre-training method based on self-supervised learning, namely, generating the preprocessing model in this application embodiment. In this method, the model is pre-trained effectively with zero manual annotation cost. Point cloud reconstruction can be selected as a proxy task for the preprocessing model. Specifically, the process may include the following steps:

[0059] S201. Divide the original point cloud in each point cloud image in the first image set to obtain point cloud blocks.

[0060] S202. Mask specific point cloud blocks in the point cloud block to obtain masked point cloud blocks.

[0061] S203. Based on the encoder, the original point cloud features and the point cloud features of the unmasked point cloud blocks in each point cloud image are encoded to obtain the encoded features.

[0062] S204. Input the encoded features into the decoder for decoding to obtain the decoded features.

[0063] S205. Based on the task module, the masked point cloud blocks are reconstructed from the decoded features to obtain the reconstructed masked point cloud blocks.

[0064] S206. Based on the deviation data between the reconstructed masked point cloud block and the original point cloud block before it was masked, a preprocessed model is trained.

[0065] In the embodiments of this application, the specific structure of the encoder, decoder and task module of the preprocessing model can be determined based on the model structure of the preprocessing module. Correspondingly, it may also include other modules or network layer structures. The embodiments of this application mainly describe the execution functions of each module and do not limit the specific structure of each module or other naming methods.

[0066] First, the original point cloud in the point cloud image is divided into irregular point cloud blocks. In one implementation, dividing the original point cloud in each point cloud image of the first image set to obtain point cloud blocks includes: sampling each point cloud image in the first image set to obtain multiple pixel center points; and scanning multiple neighboring pixels around each pixel center point to form multiple point cloud blocks. Specifically, the Farthest Point Sampling (FPS) algorithm can be used to sample N center points from the input point cloud image, denoted as P; then, the K-Nearest Neighborhood algorithm is used to search for K neighborhoods to obtain N point cloud blocks, denoted as F.

[0067] Then, specific point cloud blocks within the point cloud block are masked to obtain masked point cloud blocks. This can be done by randomly selecting some point cloud blocks for processing, or by masking point cloud blocks in specific regions, such as masking adhesive traces and border areas in a circuit board, allowing for greater focus on these regions during learning. Specifically, in one implementation, masking specific point cloud blocks within the point cloud block to obtain masked point cloud blocks includes: determining key regions based on the image features of the object to be detected; and masking point cloud blocks belonging to the key regions within the point cloud block to obtain masked point cloud blocks. The image features of the object to be detected refer to the image features of the 3D point cloud image of the object. Image features can include brightness features, pixel features, etc., which can be used to determine key regions that require focused learning. Masking the point cloud blocks in these key regions allows for adjustment of model parameters during pre-training based on the reconstruction of masked point cloud blocks in these regions, resulting in a pre-trained model that can accurately acquire the features of these regions in subsequent applications.

[0068] The encoder encodes the original point cloud features and the point cloud features of unmasked point cloud blocks in each point cloud image to obtain encoded features. These encoded features are then input into the decoder for decoding to obtain decoded features. The task module reconstructs masked point cloud blocks based on the decoded features, resulting in reconstructed masked point cloud blocks. A preprocessing model is trained based on the deviation data between the reconstructed masked point cloud block data and the original point cloud data before the masked point cloud blocks were uncovered. The training process based on the deviation data involves processing the results obtained from the preprocessing model against the actual data of the unmasked point cloud blocks. If the deviation is within an acceptable range, the current preprocessing model is used as the trained preprocessing model; otherwise, the model parameters need to be adjusted until the desired result is achieved. Correspondingly, in one implementation, training a preprocessing model based on the deviation between the reconstructed masked point cloud data and the original point cloud data before the masked point cloud data was masked includes: obtaining a loss function; adjusting the initial model parameters of the preprocessing model based on the reconstructed masked point cloud data until the loss function converges, thus completing the training of the preprocessing model; wherein, the loss function is determined based on the data relationship between the original point cloud data of the masked point cloud data and the point cloud data of the reconstructed masked point cloud data. For example, the chamfer distance between the original point cloud data and the reconstructed point cloud data of the masked point cloud data can be used as the loss function.

[0069] After obtaining the preprocessed model trained on the point cloud reconstruction proxy task, the model parameters of the corresponding modules in the preprocessed model are adjusted based on the target point cloud image and applied to the subsequent 3D defect detection model, thereby obtaining a 3D defect detection model capable of performing 3D defect detection. In one embodiment, the relevant model parameters are adjusted using multiple target point cloud images in a second image set. Each target point cloud image is labeled with a corresponding target 3D defect detection result, which can be considered as the true 3D defect detection result of the target point cloud image. For example, the defect location in the target point cloud image can be marked by a bounding box. Specifically, the encoding parameters and detection parameters of the detection module are adjusted using the second image set to obtain a 3D defect detection model. This includes: using an initial detection model to detect each target point cloud image in the second image set, obtaining an initial 3D defect detection result corresponding to each target point cloud image. The initial detection model is a model constructed using a network model structure corresponding to the 3D defect detection task, based on an encoder with initial encoding parameters and a detection module with initial detection parameters. Based on the deviation data between the initial 3D defect detection result and the target 3D defect detection result, the initial encoding parameters in the initial detection model and the initial detection parameters of the detection module are adjusted to obtain the 3D defect detection model. Therefore, in this embodiment, the model parameters of the relevant parts of a pre-trained preprocessed model can be loaded, allowing downstream tasks to fine-tune the model parameters with less labeled data, thereby reducing the cost of manually annotating 3D data and improving the accuracy of 3D industrial defect detection.

[0070] The following describes the 3D defect detection method in this application embodiment using specific application scenarios. This method can be applied to scenarios such as 3D screw defect detection, 3D adhesive defect detection on the edge of the casing, or 3D defect detection and assembly inspection of the internal components of electronic devices. See also Figure 2 ,exist Figure 2 The preprocessing model is a point cloud network that has been extensively pre-trained using a point cloud reconstruction proxy task, and is fully pre-trained solely on a large amount of unlabeled 3D data. Figure 2The encoder is represented by Encode, the decoder by Decode, and the task module by Prediction Head. After obtaining the 3D point cloud image, the Farthest Point Sampling (FPS) algorithm is first used to sample N center points of the input point cloud image, denoted as P; then, the K-Nearest Neighborhood algorithm is used to search for K neighborhoods to obtain N point cloud blocks, denoted as F, where F is represented as F: N*K*3, where N represents the number of point cloud blocks, K represents the number of pixels in each point cloud block, and 3 represents a three-dimensional pixel, that is, each pixel includes three-dimensional coordinate information (x, y, z).

[0071] For key locations (e.g., regions of interest in a detected part), a high-ratio random masking process is applied to the point cloud center point N and point cloud blocks. The masked point cloud blocks are used as the Ground Truth for the point cloud reconstruction task, denoted as Fm. That is, the point cloud data before the unmasked point cloud blocks are used as the target value for the point cloud reconstruction task. The unmasked point cloud blocks are then processed through a pre-designed lightweight point cloud ensemble encoding network to obtain point cloud features Fv; the original unmasked point cloud is denoted as Pv. This process, which performs ratio masking on point cloud blocks at key locations, allows the network to focus more on learning features of key regions. Figure 2 The number of unmasked point cloud patches is M, and the number of masked point cloud patches is (NM). The original point cloud Pv and point cloud features Fv of the unmasked point cloud patches are encoded using a point cloud preprocessing model to be trained. Then, the features obtained by the encoder are decoded using a lightweight PointNet to obtain Fvˊ. A simple MLP layer is used as the prediction head to reconstruct the missing regions of the point cloud. The goal of reconstruction is to recover each masked point cloud patch, and the reconstructed point cloud patch is denoted as Fm'. During training, the L2 chamfer distance between Fm and Fmˊ is used as the loss function. In subsequent applications, the Decode and Prediction Heads can be removed, and new Prediction Heads can be designed for different downstream tasks. The model is trained with the help of the point cloud reconstruction proxy task, enabling the model to learn high-quality, robust high-level latent features. In practical applications, loading the pre-trained model weights allows downstream tasks to fine-tune large model parameters with less labeled data, thereby reducing the cost of manually annotating 3D data and improving the accuracy of 3D industrial defect detection.

[0072] It should be noted that, in Figure 2 The network structure and technical features used in the network shown are merely illustrative examples. For instance, in... Figure 2The 2D Convolution, Batch normalization, and ReLU activation layers can all be replaced by other structures or network layers, and this application does not impose any restrictions on this.

[0073] In this application scenario, a lightweight network is used to segment and extract features from the 3D point cloud. Targeting the characteristics of industrial data, a high ratio of random masking is applied to key locations, allowing pre-training to focus more on critical regions. Then, the feature vector of each point cloud block is extracted and reconstructed into a masked point cloud using an encoder, decoder, and prediction head. Through pre-training on a large amount of unlabeled data, the model can learn high-quality, robust high-level latent 3D features. By reusing the pre-trained weights and fine-tuning with limited labeled data, the accuracy of 3D defect detection can be effectively improved. For example, see... Figure 3 In the detection of 3D defects in adhesive traces at the edges of electronic device housings, the pre-trained PointMLP point cloud segmentation network of this application is used. Through training on a large amount of unlabeled data, and by combining a strategy of using high-proportion masks for adhesive traces and border areas to focus on learning key regions, PointMLP training is more thorough, resulting in better and more accurate segmentation results.

[0074] Another embodiment of this application provides a three-dimensional defect detection device, see [link to relevant documentation]. Figure 4 The device includes:

[0075] The first acquisition unit 401 is used to acquire the point cloud image of the object to be detected;

[0076] The second acquisition unit 402 is used to obtain the three-dimensional point cloud features of the point cloud image based on the preprocessing module of the three-dimensional defect detection model.

[0077] The processing unit 403 is used to process the three-dimensional point cloud features based on the detection module of the three-dimensional defect detection model to obtain the three-dimensional defect detection result;

[0078] The three-dimensional defect detection model is obtained by adjusting the original parameters of the preprocessing module and the detection module using the target point cloud image corresponding to the downstream task, with the three-dimensional defect detection task as the downstream task.

[0079] Optionally, the apparatus further includes: a model generation unit, the model generation unit comprising:

[0080] A preprocessing model generation subunit is used to train a preprocessing model based on a first image set, wherein the first image set is an unlabeled point cloud image set. The preprocessing model is used to reconstruct point cloud data. The preprocessing model includes at least an encoder, a decoder, and a task module. The encoder is used to encode and obtain three-dimensional point cloud features.

[0081] The model parameter adjustment subunit is used to apply the initial encoding parameters of the encoder in the preprocessed model to the network model structure corresponding to the three-dimensional defect detection task, and to adjust the initial encoding parameters and the initial detection parameters of the detection module using the second image set to obtain the three-dimensional defect detection model.

[0082] The three-dimensional defect detection model includes a preprocessing module and a detection module. The preprocessing module is determined based on the adjustment of the initial encoding parameters of the encoder. The detection module is used to process the three-dimensional point cloud features output by the preprocessing module to obtain the three-dimensional defect detection result. The second image set is a labeled target point cloud image dataset corresponding to the three-dimensional defect detection task.

[0083] Optionally, the preprocessing model generation subunit includes:

[0084] The point cloud division subunit is used to divide the original point cloud in each point cloud image in the first image set into point cloud blocks;

[0085] The masking processing subunit is used to perform masking processing on specific point cloud blocks in the point cloud block to obtain masked point cloud blocks;

[0086] The encoding subunit is used to encode the original point cloud features and the point cloud features of the unmasked point cloud blocks in each point cloud image based on the encoder, so as to obtain the encoded features;

[0087] A decoding subunit is used to input the encoded features into a decoder for decoding to obtain decoded features;

[0088] The reconstruction subunit is used to reconstruct the masked point cloud blocks based on the decoding features of the task module, so as to obtain the reconstructed masked point cloud blocks.

[0089] The training subunit is used to train a preprocessed model based on the deviation data between the point cloud data of the reconstructed masked point cloud block and the original point cloud data before the masked point cloud block was unmasked.

[0090] Optionally, the training subunit is specifically used for:

[0091] Obtain the loss function;

[0092] The initial model parameters of the preprocessing model are adjusted based on the reconstructed masked point cloud blocks until the loss function converges, thus completing the training of the preprocessing model.

[0093] The loss function is determined based on the data relationship between the original point cloud data of the masked point cloud blocks and the point cloud data of the reconstructed masked point cloud blocks.

[0094] Optionally, the masking processing subunit is specifically used for:

[0095] Based on the image features of the object to be detected, key regions are identified;

[0096] The point cloud blocks belonging to the key region are masked to obtain masked point cloud blocks.

[0097] Optionally, the point cloud partitioning subunit is specifically used for:

[0098] Sample the cloud image of each point in the first image set to obtain multiple pixel center points;

[0099] Multiple point cloud blocks are formed by scanning multiple neighboring pixels around the center point of each pixel.

[0100] Optionally, the second image set includes multiple target point cloud images, and each target point cloud image is labeled with a corresponding target 3D defect detection result, wherein the model parameter adjustment subunit is specifically used for:

[0101] The initial detection model is used to detect each target point cloud image in the second image set to obtain the initial three-dimensional defect detection result corresponding to each target point cloud image. The initial detection model is a model constructed based on an encoder with initial encoding parameters and a detection module with initial detection parameters, using a network model structure corresponding to the three-dimensional defect detection task.

[0102] Based on the deviation data between the initial 3D defect detection result and the target 3D defect detection result, the initial encoding parameters in the initial detection model and the initial detection parameters of the detection module are adjusted to obtain the 3D defect detection model.

[0103] It should be noted that the specific implementation of each unit and subunit in this embodiment can be referred to the corresponding content above, and will not be described in detail here.

[0104] In another embodiment of this application, a storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the three-dimensional defect detection method as described in any of the preceding claims.

[0105] In another embodiment of this application, an electronic device is also provided, which may include:

[0106] Memory, used to store applications and the data generated by the running of the applications;

[0107] A processor for executing the application to implement the three-dimensional defect method as described in any of the above.

[0108] It should be noted that the specific implementation of the processor in this embodiment can be referred to the corresponding content above, and will not be described in detail here.

[0109] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0110] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0111] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0112] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional defect detection method, comprising: Obtain the point cloud image of the object to be detected; The preprocessing module based on the three-dimensional defect detection model obtains the three-dimensional point cloud features of the point cloud image. The detection module based on the three-dimensional defect detection model processes the three-dimensional point cloud features to obtain the three-dimensional defect detection results. The three-dimensional defect detection model is obtained by adjusting the original parameters of the preprocessing module and the detection module using the target point cloud image corresponding to the downstream task, with the three-dimensional defect detection task as the downstream task. The preprocessing module is used to reconstruct point cloud data. The preprocessing module includes at least an encoder, a decoder, and a task module. The encoder is used to encode and obtain three-dimensional point cloud features. The initial encoding parameters of the encoder in the preprocessing module are applied to the network model structure corresponding to the 3D defect detection task, and the initial encoding parameters and the initial detection parameters of the detection module are adjusted using the second image set to obtain the 3D defect detection model. The three-dimensional defect detection model includes a preprocessing module and a detection module. The preprocessing module is determined based on the initial encoding parameters of the encoder after adjustment. The detection module is used to process the three-dimensional point cloud features output by the preprocessing module to obtain the three-dimensional defect detection result. The second image set is a labeled target point cloud image dataset corresponding to the three-dimensional defect detection task. This also includes: A preprocessed model is obtained by training based on the first image set, which is an unlabeled point cloud image set, specifically including: The original point cloud in each point cloud image of the first image set is divided to obtain point cloud blocks; A high-ratio random masking process is applied to the key region point cloud blocks in the point cloud blocks to obtain masked point cloud blocks; The encoder encodes the original point cloud features and the point cloud features of unmasked point cloud blocks in each point cloud image to obtain the encoded features. The encoded features are input into the decoder for decoding to obtain the decoded features; Based on the task module, the decoded features are reconstructed into masked point cloud blocks to obtain the reconstructed masked point cloud blocks; Based on the deviation data between the reconstructed masked point cloud block and the original point cloud block before it was masked, a preprocessing model is trained so that the preprocessing model can focus more on learning the features of key regions.

2. The method according to claim 1, wherein training a preprocessing model based on the deviation between the reconstructed masked point cloud data and the original point cloud data before the masked point cloud data is unmasked, comprises: Obtain the loss function; The initial model parameters of the preprocessing model are adjusted based on the reconstructed masked point cloud blocks until the loss function converges, thus completing the training of the preprocessing model. The loss function is determined based on the data relationship between the original point cloud data of the masked point cloud blocks and the point cloud data of the reconstructed masked point cloud blocks.

3. The method according to claim 1, wherein the step of masking a specific point cloud block in the point cloud block to obtain a masked point cloud block includes: Based on the image features of the object to be detected, key regions are identified; The point cloud blocks belonging to the key region are masked to obtain masked point cloud blocks.

4. The method according to claim 1, wherein dividing the original point cloud in each point cloud image of the first image set to obtain point cloud blocks includes: Sample the cloud image of each point in the first image set to obtain multiple pixel center points; Multiple point cloud blocks are formed by scanning multiple neighboring pixels around the center point of each pixel.

5. The method according to claim 1, wherein the second image set includes multiple target point cloud images, and each target point cloud image is labeled with a corresponding target three-dimensional defect detection result, wherein, The step of adjusting the encoding parameters and the detection parameters of the detection module using a second image set to obtain a three-dimensional defect detection model includes: The initial detection model is used to detect each target point cloud image in the second image set to obtain the initial three-dimensional defect detection result corresponding to each target point cloud image. The initial detection model is a model constructed based on an encoder with initial encoding parameters and a detection module with initial detection parameters, using a network model structure corresponding to the three-dimensional defect detection task. Based on the deviation data between the initial 3D defect detection result and the target 3D defect detection result, the initial encoding parameters in the initial detection model and the initial detection parameters of the detection module are adjusted to obtain the 3D defect detection model.

6. A three-dimensional defect detection device, comprising: The first acquisition unit is used to acquire the point cloud image of the object to be detected; The second acquisition unit is used to obtain the three-dimensional point cloud features of the point cloud image based on the preprocessing module of the three-dimensional defect detection model. The processing unit is used to process the three-dimensional point cloud features based on the detection module of the three-dimensional defect detection model to obtain the three-dimensional defect detection result; The three-dimensional defect detection model is obtained by adjusting the original parameters of the preprocessing module and the detection module using the target point cloud image corresponding to the downstream task, with the three-dimensional defect detection task as the downstream task. The preprocessing module is used to reconstruct point cloud data. The preprocessing module includes at least an encoder, a decoder, and a task module. The encoder is used to encode and obtain three-dimensional point cloud features. The initial encoding parameters of the encoder in the preprocessing module are applied to the network model structure corresponding to the 3D defect detection task, and the initial encoding parameters and the initial detection parameters of the detection module are adjusted using the second image set to obtain the 3D defect detection model. The three-dimensional defect detection model includes a preprocessing module and a detection module. The preprocessing module is determined based on the initial encoding parameters of the encoder after adjustment. The detection module is used to process the three-dimensional point cloud features output by the preprocessing module to obtain the three-dimensional defect detection result. The second image set is a labeled target point cloud image dataset corresponding to the three-dimensional defect detection task. The three-dimensional defect detection device is also used for: A preprocessed model is obtained by training based on the first image set, which is an unlabeled point cloud image set, specifically including: The original point cloud in each point cloud image of the first image set is divided to obtain point cloud blocks; A high-ratio random masking process is applied to the key region point cloud blocks in the point cloud blocks to obtain masked point cloud blocks; The encoder encodes the original point cloud features and the point cloud features of unmasked point cloud blocks in each point cloud image to obtain the encoded features. The encoded features are input into the decoder for decoding to obtain the decoded features; Based on the task module, the decoded features are reconstructed into masked point cloud blocks to obtain the reconstructed masked point cloud blocks; Based on the deviation data between the reconstructed masked point cloud block and the original point cloud block before it was masked, a preprocessing model is trained so that the preprocessing model can focus more on learning the features of key regions.

7. A storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the three-dimensional defect detection method as described in any one of claims 1-5.

8. An electronic device, comprising: Memory, used to store applications and the data generated by the running of the applications; A processor for executing the application to implement the three-dimensional defect detection method as described in any one of claims 1-5.

Images