Three-dimensional model flaw detection method and device, computer equipment and storage medium

Abstract

The invention relates to a three-dimensional model flaw detection method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining three-dimensional model sample data, carrying out the data preprocessing and data division of the three-dimensional model sample data, and generating a training data set, a verification data set and a test data set; initializing the YOLO model to obtain an initial flaw detection model; converting data in the training data set into two-dimensional images of multiple angles to obtain a target two-dimensional image, and performing flaw detection on the target two-dimensional image through the initial flaw detection model to obtain a flaw detection result; calculating model index parameters and model loss, and adjusting the model parameters; performing iterative training on the adjusted initial flaw detection model through the target two-dimensional image to obtain a target flaw detection model; and detecting the to-be-detected three-dimensional model data through the target flaw detection model to obtain a target flaw detection result. According to the method, the three-dimensional model flaw detection efficiency and accuracy are improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent detection technology, and in particular to a three-dimensional model defect detection method, device, computer equipment and storage medium. Background Art

[0002] Building 3D models is of great significance whether for research or for publicity and display. The traditional modeling method is to use modeling software to carve step by step, which is time-consuming and laborious. With the development of the times, the 3D reconstruction technology based on light field has great technical advantages over manual modeling; however, it requires a large number of pictures taken around, and requires that the shooting angle difference between the two pictures is very small. The mainstream 3D reconstruction methods are divided into the following two categories. The first category is 3D reconstruction based on depth cameras. This type of method uses depth cameras (such as Microsoft's Kinect, Asus's XTion, Intel's RealSense, etc.) for scanning and reconstruction. These cameras are inexpensive and easy to operate, suitable for researchers and engineers to develop. Representative technologies include structured light, TOF (time-of-flight) and binocular cameras, which are widely used in mobile devices such as mobile phones. The second category is image-based 3D reconstruction. This type of method uses multi-view images as input and restores the 3D information of the environment through matching points, geometric principles, etc. Common algorithms include SfM (Structure from Motion), REMODE and SVO.

[0003] The above-mentioned 3D reconstruction methods require special depth cameras or a large number of multi-angle images, but it is usually difficult to achieve 3D reconstruction of some specific scenes, such as rare animals in the wild. Based on this background, even if reconstruction is possible, there is a high probability that there will be defects in some parts of the 3D model when the 3D model is established through 3D reconstruction; once a problem occurs, a large part of the repair cost lies in finding the location of the defect and manually fine-tuning it or remodeling this part. Therefore, how to quickly and accurately detect defects in the 3D model has become a problem that needs to be solved at present. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a three-dimensional model defect detection method, device, computer equipment and storage medium to improve the efficiency and accuracy of three-dimensional model defect detection.

[0005] In order to solve the above technical problems, the present application provides a three-dimensional model defect detection method, including: Acquire three-dimensional model sample data, and perform data preprocessing and data division on the three-dimensional model sample data to generate a training data set, a verification data set, and a test data set; The YOLO model is used as a defect detection model, and the network structure and model parameters of the defect detection model are initialized to obtain an initial defect detection model; Converting the data in the training data set into two-dimensional images at multiple angles to obtain a target two-dimensional image, and performing defect detection on the target two-dimensional image using the initial defect detection model to obtain a defect detection result; Calculating model index parameters and model losses according to the defect detection results, and adjusting model parameters of the initial defect detection model according to the model index parameters and the model losses; Iteratively training the adjusted initial defect detection model again using the target two-dimensional image to obtain a target defect detection model; The three-dimensional model data to be detected is acquired, and the three-dimensional model data to be detected is detected by using the target defect detection model to obtain a target defect detection result.

[0006] In order to solve the above technical problems, the present application provides a three-dimensional model defect detection device, including: A sample data acquisition unit, used to acquire three-dimensional model sample data, and perform data preprocessing and data division on the three-dimensional model sample data to generate a training data set, a verification data set, and a test data set; A detection model initialization unit, used to use the YOLO model as a defect detection model, and initialize the network structure and model parameters of the defect detection model to obtain an initial defect detection model; a two-dimensional image conversion unit, configured to convert the data in the training data set into two-dimensional images at multiple angles to obtain a target two-dimensional image, and perform defect detection on the target two-dimensional image using the initial defect detection model to obtain a defect detection result; A model parameter adjustment unit, configured to calculate a model index parameter and a model loss according to the defect detection result, and adjust the model parameters of the initial defect detection model according to the model index parameter and the model loss; A detection model training unit, used for iteratively training the adjusted initial defect detection model again through the target two-dimensional image to obtain a target defect detection model; The target defect detection result unit is used to obtain the three-dimensional model data to be detected, detect the three-dimensional model data to be detected by using the target defect detection model, and obtain the target defect detection result.

[0007] In order to solve the above technical problems, a technical solution adopted by the present invention is: to provide a computer device, including one or more processors; a memory for storing one or more programs, so that the one or more processors can implement any one of the above three-dimensional model defect detection methods.

[0008] In order to solve the above technical problems, a technical solution adopted by the present invention is: a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, any one of the above-mentioned three-dimensional model defect detection methods is implemented.

[0009] The embodiment of the present invention provides a method, device, computer equipment and storage medium for three-dimensional model defect detection. The method includes: obtaining three-dimensional model sample data, and performing data preprocessing and data division on the three-dimensional model sample data to generate a training data set, a verification data set and a test data set; using the YOLO model as a defect detection model, and initializing the network structure and model parameters of the defect detection model to obtain an initial defect detection model; converting the data in the training data set into two-dimensional images of multiple angles to obtain a target two-dimensional image, and performing defect detection on the target two-dimensional image through the initial defect detection model to obtain a defect detection result; calculating model index parameters and model loss according to the defect detection result, and adjusting the model parameters of the initial defect detection model according to the model index parameters and the model loss; iteratively training the adjusted initial defect detection model again through the target two-dimensional image to obtain a target defect detection model; obtaining the three-dimensional model data to be detected, and detecting the three-dimensional model data to be detected through the target defect detection model to obtain a target defect detection result. The embodiment of the present invention obtains three-dimensional model sample data, converts the three-dimensional model sample data into a two-dimensional image to train the YOLO model, generates a target defect detection model, and performs defect detection on the three-dimensional model data to be detected through the target defect detection model, thereby realizing automated three-dimensional model defect detection, avoiding manual defect detection, and improving the efficiency of three-dimensional model defect detection. At the same time, through the training of the YOLO model, defects from different perspectives can be accurately identified, which is conducive to improving the accuracy of three-dimensional model defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 It is a flowchart of the implementation process of the three-dimensional model defect detection method provided in the embodiment of the present application; Figure 2 is a flowchart for implementing the first sub-process in the three-dimensional model defect detection method provided in an embodiment of the present application; Figure 3 is a flowchart for implementing the second sub-process in the three-dimensional model defect detection method provided in an embodiment of the present application; Figure 4 It is a schematic diagram of converting three-dimensional model sample data provided by an embodiment of the present application into a two-dimensional image; Figure 5 is a flowchart for implementing the third sub-process in the three-dimensional model defect detection method provided in an embodiment of the present application; Figure 6 is a schematic diagram of three-dimensional model defect detection provided by an embodiment of the present application; Figure 7 is a flowchart for implementing the fourth sub-process in the three-dimensional model defect detection method provided in an embodiment of the present application; Figure 8 is a flowchart for implementing the fifth sub-process in the three-dimensional model defect detection method provided in an embodiment of the present application; Fig. 9 is a flowchart for implementing the sixth sub-process in the three-dimensional model defect detection method provided in an embodiment of the present application; Fig.10 is a schematic diagram of an input detection image provided in an embodiment of the present application; Fig.11 is a schematic diagram of a three-dimensional model defect detection process provided in an embodiment of the present application; Fig.12 It is a schematic diagram of the detection results provided in the embodiment of the present application; Fig.13 is a schematic diagram of a three-dimensional model defect detection device provided in an embodiment of the present application; Fig.14 It is a schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0012] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0013] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0014] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0015] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0016] It should be noted that the three-dimensional model defect detection method provided in the embodiment of the present application is generally executed by a server, and accordingly, the three-dimensional model defect detection device is generally configured in the server.

[0017] See also Figure 1 , Figure 1 A specific implementation of a three-dimensional model defect detection method is shown.

[0018] It should be noted that if there are substantially the same results, the method of the present invention is not limited to Figure 1 The process sequence shown is limited to the following steps: S1: Acquire three-dimensional model sample data, and perform data preprocessing and data division on the three-dimensional model sample data to generate a training data set, a verification data set, and a test data set.

[0019] Specifically, three-dimensional model sample data containing various types of defects are collected. These models can come from different fields, such as architecture, sculpture, natural landscape, etc., to ensure the diversity and extensiveness of the data set. The collected three-dimensional model sample data should cover a variety of defect types, including but not limited to irregular shapes, uneven surfaces, missing parts, redundant structures, etc. Then the three-dimensional model sample data is manually annotated to clearly indicate the defective areas in the model. Defect annotation includes information such as the type, location, and size of the annotated defects. During the annotation process, professional three-dimensional modeling and inspection personnel are required to participate to ensure the accuracy and consistency of the annotation. After the three-dimensional model sample data is annotated, the three-dimensional model sample data is preprocessed to improve the generalization ability and robustness of the subsequent model, and the preprocessed three-dimensional model sample data is divided into data to obtain a training data set, a verification data set, and a test data set.

[0020] See also Figure 2 , Figure 2 A specific implementation of step S1 is shown, which is described in detail as follows: S11: Acquire the three-dimensional model sample data, and store the annotation information of the defective area in the three-dimensional model sample data according to a preset format.

[0021] S12: performing data enhancement and data cleaning on the three-dimensional model sample data to obtain pre-processed sample data, wherein the data enhancement includes data rotation, scaling, cropping and color adjustment.

[0022] S13: Dividing the preprocessed sample data according to a preset division ratio to obtain the training data set, the verification data set and the test data set.

[0023] Specifically, after the three-dimensional model sample data is annotated, the annotation information of the defective area in the three-dimensional model sample data is stored in a preset format, wherein the preset format may be a JSON format or an XML format, so as to facilitate subsequent data processing and model training. In order to improve the generalization ability and robustness of the model, the three-dimensional model sample data is subjected to data enhancement processing. Data enhancement includes operations such as rotation, scaling, cropping, and color adjustment. Among them, rotation refers to randomly rotating the model to simulate the observation effect at different angles. Scaling refers to scaling the model to different degrees to simulate observation at different distances. Cropping refers to cropping images from different parts of the model to increase the number of local samples of the model. Color adjustment refers to changing the color and brightness of the model to simulate the observation effect under different lighting conditions. It should be noted that the consistency of the annotation data needs to be ensured so that there is no contradiction between different annotations of the same model. Then, the data enhancement of the three-dimensional model sample data is cleaned to remove unclear or low-quality three-dimensional model sample data to ensure the quality of the data set.

[0024] Finally, the preprocessed sample data is divided according to the preset division ratio to obtain the training data set, the verification data set and the test data set. Among them, the training data set is used for model training, the verification data set is used for model tuning, and the test data set is used for model performance evaluation. Ensure that the data set is evenly distributed to avoid a certain type of defect or model from accounting for too large a proportion in the data set. It should be noted that the preset division ratio is set according to the actual situation and is not limited here.

[0025] S2: Using the YOLO model as a defect detection model, and initializing the network structure and model parameters of the defect detection model to obtain an initial defect detection model.

[0026] Specifically, YOLO (You Only Look Once) is a popular real-time target detection system that treats the target detection task as a regression problem, directly mapping from image pixels to bounding box coordinates and category probabilities. In the embodiments of the present application, different YOLO versions can be selected as defect detection models, for example, YOLOv8 or YOLOv10 can be selected. In the embodiments of the present application, transfer learning technology is used, and a model pre-trained on a large data set (such as ImageNet) is used as a starting point, which can speed up the training process and improve the performance of the model on a small data set. The pre-trained model has learned rich feature representations, which are also useful for new data sets.

[0027] Since the embodiment of the present application is to detect the three-dimensional model, the YOLO network structure is adjusted according to the characteristics of the three-dimensional model. For example, the feature extraction layer is added or modified to better adapt to the complexity and diversity of the three-dimensional model. By adjusting the depth and width of the YOLO network, the complexity and computational efficiency of the model are balanced. At the same time, the present application needs to initialize the model parameters of the defect detection model, and the model parameters include learning rate, batch size, number of iterations, activation parameters and optimizer, etc. At the same time, the embodiment of the present application can define a suitable loss function to train the model to accurately predict the location and category of the defect. The loss function usually includes positioning loss (such as IoU loss) and classification loss (such as cross entropy loss). Positioning loss is used to measure the difference between the predicted bounding box and the true bounding box, and classification loss is used to measure the difference between the predicted category and the true category. The present application selects a suitable optimizer, such as Adam or SGD, to optimize the weight of the model. Set a suitable learning rate and learning rate decay strategy to ensure that the model can converge quickly while avoiding overfitting.

[0028] S3: Convert the data in the training data set into two-dimensional images at multiple angles to obtain a target two-dimensional image, and perform defect detection on the target two-dimensional image using the initial defect detection model to obtain a defect detection result.

[0029] Specifically, the multi-view projection technology of the 3D model is used to generate 2D images from different angles to ensure that the model can fully identify and understand defects in the 3D space. The generated 2D images are optimized, including contrast adjustment and noise removal, to improve the detection performance of the model. The converted target 2D image is used as the input of the initial defect detection model, and defect detection is performed on it to obtain the defect detection result.

[0030] See also Figure 3 and Figure 4 , Figure 3 A specific implementation of step S3 is shown. Figure 4This is a schematic diagram of converting the three-dimensional model sample data provided by the embodiment of the present application into a two-dimensional image, which is described in detail as follows: S31: using a three-dimensional rendering technology to convert each data in the training data set into a two-dimensional image at multiple angles to obtain an initial two-dimensional image.

[0031] Specifically, use 3D rendering technology to generate 2D images of the model from multiple angles for each data in the training dataset. These angles should be able to fully cover all parts of the model to ensure that no possible defective areas are missed. During the rendering process, factors such as light, shadows, and textures need to be considered to generate realistic images that better simulate real-world viewing conditions. Determine the appropriate number and distribution of viewing angles. Evenly distributed viewing angles can provide better coverage, but computational costs and actual needs also need to be considered. The best viewing angle can be automatically determined using an algorithm, or commonly used viewing angles such as front, side, top view, etc. can be selected based on experience.

[0032] S32: performing image enhancement and image unification processing on the initial two-dimensional image to obtain a basic two-dimensional image.

[0033] Specifically, the generated 2D image is enhanced to improve the detection performance of the model. Image enhancement includes contrast adjustment, noise removal, sharpening, etc. Image processing techniques, such as histogram equalization, are used to enhance the contrast of the image to make defects easier to detect. Different lighting conditions are simulated to generate images under different lighting conditions. This helps the model learn to detect defects under various lighting conditions. Before the target 2D image is input into the initial defect detection model, preprocessing operations, such as normalization, resizing, etc., are performed to unify the image.

[0034] S33: marking the defect area of ​​the basic two-dimensional image to obtain the target two-dimensional image.

[0035] Specifically, after converting the 3D image model sample data into a 2D image, the defect area in the 2D image is not marked, so it is necessary to mark the defect area of ​​the basic 2D image, including the location, type and other information of the defect, to obtain the target 2D image.

[0036] S34: performing defect detection on the target two-dimensional image using the initial defect detection model to obtain the defect detection result.

[0037] See also Figure 5 and Figure 6 , Figure 5 A specific implementation of step S34 is shown. Figure 6 This is a schematic diagram of three-dimensional model defect detection provided by an embodiment of the present application, which is described in detail as follows: S341: Locating and identifying the defect area in the target two-dimensional image through the initial defect detection model to generate an initial defect detection result, wherein the initial defect detection result includes the bounding box coordinates of the defect position, the defect type and the confidence level of the defect.

[0038] S342: Fusing and correcting the initial defect detection results of the same defect area to obtain the defect detection result.

[0039] Specifically, when inputting the target two-dimensional image into the initial defect detection model, it is necessary to ensure the order and consistency of the image sequence so that the model can correctly understand and analyze it. The initial defect detection model locates and identifies the defect areas of the input target two-dimensional image in turn, and generates an initial defect detection result corresponding to each target two-dimensional image. The initial defect detection result includes the bounding box coordinates of the defect location, the defect type and the confidence of the defect. Defect categories include missing, redundant, deformed, cracked, etc. At the same time, in order for the model to be able to identify and classify defects in a targeted manner, a unique identifier is defined for each defect category before inputting it into the initial defect detection model to facilitate model output and subsequent processing.

[0040] Then the model integrates the detection results of each view image to obtain the overall defect information of the 3D model. The initial defect detection results of the same defect area in different view angles are fused and corrected to obtain a more accurate defect location and category, thereby obtaining the defect detection results of the same 3D model detection sample data.

[0041] S4: Calculating model index parameters and model losses according to the defect detection results, and adjusting model parameters of the initial defect detection model according to the model index parameters and the model losses.

[0042] Specifically, the model index parameters and model loss are calculated according to the defect detection results, the detection performance of the initial defect detection model is evaluated by the model index parameters, and the performance of the model on different types of defects and three-dimensional models is analyzed to determine the strengths and weaknesses of the model. According to the results of the performance evaluation, an optimization strategy is formulated, such as adjusting model parameters, increasing training data, and improving data enhancement methods. The model loss can be calculated by a preset loss function, and the model parameters of the initial defect detection model can be adjusted to improve the ability of the model to detect defects.

[0043] See also Figure 7 , Figure 7 A specific implementation of step S4 is shown, which is described in detail as follows: S41: Calculate the model index parameters according to the defect detection result, wherein the model index parameters include accuracy, recall rate, F1 score, average precision and mAP.

[0044] S42: performing loss calculations for defect area positioning and defect classification based on the defect detection result and the annotation information in the verification data set using a preset loss function to obtain positioning loss and classification loss.

[0045] S43: Calculate the model loss according to the positioning loss and the classification loss.

[0046] S44: adjusting the model parameters of the initial defect detection model according to the model index parameters and the model loss.

[0047] Specifically, the model indicator parameters include precision, recall, F1 score, average precision (AP) and mAP (mean Average Precision). Precision refers to the proportion of samples predicted by the model to be positive that are actually positive, and recall refers to the proportion of all actual positive samples predicted by the model to be positive. These two indicators will help us balance the recall and precision of the model. The recall rate is the proportion of samples predicted by the model to be positive in actual positive samples; the F1 score is the harmonic mean of the precision and recall rate; the average precision is the area under the precision and recall curves at different detection thresholds; and mAP is the mean of the average precision of all categories. Furthermore, the embodiment of the present application can construct a confusion matrix, which is a tool for visualizing model performance that can display the number of true positives, false positives, true negatives, and false negatives predicted by the model. The confusion matrix will help understand the performance of the model on different defect categories.

[0048] In the embodiment of the present application, the IoU loss function and the cross entropy loss are used to calculate the loss of defect area positioning and defect classification based on the defect detection results and the corresponding annotation information in the verification data set, and the positioning loss and the classification loss are obtained. Then, the positioning loss and the classification loss can be matched with the preset weights, and then the model loss is calculated. Then, the model parameters of the initial defect detection model are adjusted according to the model indicator parameters and the model loss.

[0049] S5: iteratively training the adjusted initial defect detection model again using the target two-dimensional image to obtain a target defect detection model.

[0050] Specifically, the adjusted initial defect detection model is iteratively trained again through the target two-dimensional image. The purpose of iterative training is to improve the detection accuracy and robustness of the model. Verification and optimization are regarded as an iterative process, and the evaluation, analysis and fine-tuning steps are repeated until the model performance reaches a satisfactory level. Once the model performance reaches a satisfactory level, the model is ready to be deployed in actual applications. Ensure that the deployment environment of the model is consistent with the training environment to avoid performance changes caused by environmental differences. Whether a satisfactory level is obtained can be determined based on whether the model loss reaches a preset threshold.

[0051] See also Figure 8 , Figure 8 A specific implementation of step S5 is shown, which is described in detail as follows: S51: iteratively training the adjusted initial defect detection model through the target two-dimensional image, and generating a new model loss in each iterative training.

[0052] S52: When the new model loss reaches a preset threshold or the number of iterations reaches a preset number, the iterative training is stopped to obtain a trained defect detection model.

[0053] S53: Testing the trained defect detection model using the test data set to obtain the target defect detection model.

[0054] Specifically, the adjusted initial defect detection model is iteratively trained through the target two-dimensional image, and a new model loss is generated in each iterative training; then it is determined whether the model loss at this time reaches a preset threshold or whether the number of iterations reaches a preset number of times. If not, the model parameters of the defect detection model are readjusted, and the model training is performed again until the new model loss reaches a preset threshold or the number of iterations reaches a preset number of times, then the iterative training is stopped to obtain the trained defect detection model. Finally, in order to evaluate and test the ability of the defect detection model, the trained defect detection model is tested through a test data set to obtain a target defect detection model.

[0055] In a specific embodiment, when performing model training, the learning rate is set to 0.001, the batch size is 8, the number of training rounds is 50,000, the input image size is 640, and the weight decay is 0.0005.

[0056] Furthermore, after training and verification generate a target defect detection model, the target defect detection model is deployed to actual applications. When deploying the model, ensure that the deployment environment of the model is consistent with the training environment to avoid performance changes caused by environmental differences.

[0057] S6: Acquire the three-dimensional model data to be detected, and detect the three-dimensional model data to be detected by using the target defect detection model to obtain a target defect detection result.

[0058] Specifically, the above steps have trained and deployed the target defect detection model, and in actual application scenarios or reasoning scenarios, the 3D model data to be detected is obtained, and the 3D model data to be detected is detected by the target defect detection model to obtain the target defect detection result. The target defect detection result includes the defect area and defect type in the 3D model.

[0059] See also Fig. 9 , Fig. 9 A specific implementation of step S6 is shown, which is described in detail as follows: S61: Acquire the three-dimensional model data to be detected, and convert the three-dimensional model data to be detected into two-dimensional images at multiple angles to obtain two-dimensional images to be detected.

[0060] S62: performing image enhancement and image cropping on the two-dimensional image to be detected to obtain a target two-dimensional image to be detected.

[0061] S63: Detecting the target two-dimensional image to be detected by using the target defect detection model to obtain the target defect detection result.

[0062] Specifically, in order to detect defects in 3D model data, the 3D model data to be detected is obtained, and the 3D model data to be detected is converted into a multi-angle 2D image to obtain a 2D image to be detected. Then, in order to improve the accuracy of defect detection, the 2D image to be detected is enhanced and cropped to obtain a target 2D image to be detected. Finally, the target 2D image to be detected is detected by the target defect detection model to obtain a target defect detection result.

[0063] In an embodiment of the present application, three-dimensional model sample data is obtained, and data preprocessing and data division are performed on the three-dimensional model sample data to generate a training data set, a verification data set and a test data set; a YOLO model is used as a defect detection model, and the network structure and model parameters of the defect detection model are initialized to obtain an initial defect detection model; data in the training data set are converted into two-dimensional images at multiple angles to obtain a target two-dimensional image, and defect detection is performed on the target two-dimensional image through the initial defect detection model to obtain a defect detection result; model index parameters and model loss are calculated according to the defect detection result, and model parameters of the initial defect detection model are adjusted according to the model index parameters and the model loss; the adjusted initial defect detection model is iteratively trained again through the target two-dimensional image to obtain a target defect detection model; three-dimensional model data to be detected is obtained, and the three-dimensional model data to be detected is detected by the target defect detection model to obtain a target defect detection result. The embodiment of the present invention obtains three-dimensional model sample data, converts the three-dimensional model sample data into a two-dimensional image to train the YOLO model, generates a target defect detection model, and performs defect detection on the three-dimensional model data to be detected through the target defect detection model, thereby realizing automated three-dimensional model defect detection, avoiding manual defect detection, and improving the efficiency of three-dimensional model defect detection. At the same time, through the training of the YOLO model, defects from different perspectives can be accurately identified, which is conducive to improving the accuracy of three-dimensional model defect detection.

[0064] like Figures 10 to 12 As shown, Fig.10 is a schematic diagram of an input detection image provided in an embodiment of the present application; Fig.11 is a schematic diagram of a three-dimensional model defect detection process provided in an embodiment of the present application; Fig.12 is a schematic diagram of the detection results provided in the embodiment of the present application. In a specific embodiment, Fig.10 The input detection image in the target defect detection model of this application is input to perform defect detection. During the detection process, it will generate Fig.11 The feature maps of different network layers in the final output Fig.12 The test results in Fig.12 In the figure, the confidence levels of Defect 1 and Defect 2 are 95% and 88% respectively, both exceeding the preset confidence threshold (e.g., 85%).

[0065] The embodiment of the present application uses the bounding box positioning function of YOLO to accurately mark the defect location in the model, providing clear guidance for subsequent manual repair. The model of the present application can adapt to different types of 3D models and defect types, and has good generalization ability. The embodiment of the present application realizes rapid detection, provides instant feedback for iterative repair of 3D models, and accelerates the model optimization process.

[0066] Please refer to Fig.13 , as a response to the above Figure 1 The present application provides an embodiment of a three-dimensional model defect detection device. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0067] like Fig.13 As shown, the three-dimensional model defect detection device of this embodiment includes: a sample data acquisition unit 71, a detection model initialization unit 72, a two-dimensional image conversion unit 73, a model parameter adjustment unit 74, a detection model training unit 75 and a target defect detection result unit 76, wherein: The sample data acquisition unit 71 is used to acquire the three-dimensional model sample data, and perform data preprocessing and data division on the three-dimensional model sample data to generate a training data set, a verification data set and a test data set; A detection model initialization unit 72 is used to use the YOLO model as a defect detection model, and initialize the network structure and model parameters of the defect detection model to obtain an initial defect detection model; A two-dimensional image conversion unit 73 is used to convert the data in the training data set into two-dimensional images at multiple angles to obtain a target two-dimensional image, and perform defect detection on the target two-dimensional image through the initial defect detection model to obtain a defect detection result; A model parameter adjustment unit 74, configured to calculate a model index parameter and a model loss according to the defect detection result, and adjust the model parameters of the initial defect detection model according to the model index parameter and the model loss; A detection model training unit 75, configured to iteratively train the adjusted initial defect detection model again through the target two-dimensional image to obtain a target defect detection model; The target defect detection result unit 76 is used to obtain the three-dimensional model data to be detected, and detect the three-dimensional model data to be detected by using the target defect detection model to obtain the target defect detection result.

[0068] Furthermore, the sample data acquisition unit 71 includes: A standard information storage unit, used to obtain the three-dimensional model sample data, and store the annotation information of the defect area in the three-dimensional model sample data according to a preset format; A data preprocessing unit, used to perform data enhancement and data cleaning on the three-dimensional model sample data to obtain preprocessed sample data, wherein data enhancement includes data rotation, scaling, cropping and color adjustment; The data partitioning unit is used to partition the preprocessed sample data according to a preset partitioning ratio to obtain the training data set, the verification data set and the test data set.

[0069] Further, the two-dimensional image conversion unit 73 includes: An initial two-dimensional image generating unit, configured to convert each data in the training data set into two-dimensional images of multiple angles by using a three-dimensional rendering technology to obtain an initial two-dimensional image; A basic two-dimensional image generating unit, used for performing image enhancement and image unification processing on the initial two-dimensional image to obtain a basic two-dimensional image; A target two-dimensional image generating unit, used for marking the defect area of ​​the basic two-dimensional image to obtain the target two-dimensional image; The defect detection result generating unit is used to perform defect detection on the target two-dimensional image by using the initial defect detection model to obtain the defect detection result.

[0070] Furthermore, the defect detection result generating unit includes: A defect detection unit, configured to locate and identify a defect area in the target two-dimensional image by using the initial defect detection model, and generate an initial defect detection result, wherein the initial defect detection result includes a bounding box coordinate of the defect position, a defect type, and a confidence level of the defect; The defect detection result fusion unit is used to fuse and correct the initial defect detection results of the same defect area to obtain the defect detection result.

[0071] Furthermore, the model parameter adjustment unit 74 includes: A model index parameter calculation unit, used to calculate the model index parameters according to the defect detection result, wherein the model index parameters include accuracy, recall rate, F1 score, average precision and mAP; A loss calculation unit, used to perform loss calculations for defect area positioning and defect classification based on the defect detection result and the annotation information in the verification data set through a preset loss function to obtain positioning loss and classification loss; A model loss calculation unit, used to calculate the model loss according to the positioning loss and the classification loss; A parameter adjustment unit is used to adjust the model parameters of the initial defect detection model according to the model indicator parameters and the model loss.

[0072] Furthermore, the detection model training unit 75 includes: an iterative training unit, configured to iteratively train the adjusted initial defect detection model using the target two-dimensional image, and generate a new model loss in each iterative training; A training stopping unit, used to stop iterative training when the new model loss reaches a preset threshold or the number of iterations reaches a preset number, so as to obtain a trained defect detection model; A test processing unit is used to perform test processing on the trained defect detection model through the test data set to obtain the target defect detection model.

[0073] Furthermore, the target defect detection result unit 76 includes: A to-be-detected three-dimensional model data acquisition unit, used to acquire the to-be-detected three-dimensional model data, and convert the to-be-detected three-dimensional model data into a two-dimensional image at multiple angles to obtain a to-be-detected two-dimensional image; A target to-be-detected two-dimensional image generating unit, used for performing image enhancement and image cropping on the two-dimensional image to be detected to obtain a target to-be-detected two-dimensional image; The target result generating unit is used to detect the target two-dimensional image to be detected by using the target defect detection model to obtain the target defect detection result.

[0074] To solve the above technical problems, the present application also provides a computer device. Fig.14 , Fig.14 This is a basic structural block diagram of the computer device in this embodiment.

[0075] The computer device 8 includes a memory 81, a processor 82, and a network interface 83 which are interconnected through a system bus. It should be noted that: Fig.14 Only a computer device 8 having three components, memory 81, processor 82, and network interface 83, is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0076] Computer devices can be computing devices such as desktop computers, notebooks, PDAs, and cloud servers. Computer devices can interact with users through keyboards, mice, remote controls, touch pads, or voice control devices.

[0077] The memory 81 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 81 can be an internal storage unit of the computer device 8, such as a hard disk or memory of the computer device 8. In other embodiments, the memory 81 can also be an external storage device of the computer device 8, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 8. Of course, the memory 81 can also include both the internal storage unit of the computer device 8 and its external storage device. In this embodiment, the memory 81 is generally used to store the operating system and various application software installed on the computer device 8, such as the program code of the three-dimensional model defect detection method, etc. In addition, the memory 81 can also be used to temporarily store various types of data that have been output or are to be output.

[0078] In some embodiments, the processor 82 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor 82 is generally used to control the overall operation of the computer device 8. In this embodiment, the processor 82 is used to run the program code stored in the memory 81 or process data, such as running the program code of the above-mentioned three-dimensional model defect detection method to implement various embodiments of the three-dimensional model defect detection method.

[0079] The network interface 83 may include a wireless network interface or a wired network interface, and the network interface 83 is generally used to establish a communication connection between the computer device 8 and other electronic devices.

[0080] The present application also provides another implementation, namely, providing a computer-readable storage medium, which stores a computer program. The computer program can be executed by at least one processor to enable the at least one processor to perform the steps of a three-dimensional model defect detection method as described above.

[0081] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0082] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is similarly within the scope of protection of the present application.

Claims

1. A three-dimensional model defect detection method, characterized in that: include: Acquire three-dimensional model sample data, and perform data preprocessing and data division on the three-dimensional model sample data to generate a training data set, a verification data set, and a test data set; The YOLO model is used as a defect detection model, and the network structure and model parameters of the defect detection model are initialized to obtain an initial defect detection model; Converting the data in the training data set into two-dimensional images at multiple angles to obtain a target two-dimensional image, and performing defect detection on the target two-dimensional image using the initial defect detection model to obtain a defect detection result; Calculating model index parameters and model losses according to the defect detection results, and adjusting model parameters of the initial defect detection model according to the model index parameters and the model losses; Iteratively training the adjusted initial defect detection model again using the target two-dimensional image to obtain a target defect detection model; The three-dimensional model data to be detected is acquired, and the three-dimensional model data to be detected is detected by using the target defect detection model to obtain a target defect detection result.

2. The three-dimensional model defect detection method according to claim 1, characterized in that: The acquiring of the three-dimensional model sample data, and performing data preprocessing and data division on the three-dimensional model sample data to generate a training data set, a verification data set, and a test data set includes: Acquire the three-dimensional model sample data, and store the annotation information of the defective area in the three-dimensional model sample data according to a preset format; Performing data enhancement and data cleaning on the three-dimensional model sample data to obtain preprocessed sample data, wherein the data enhancement includes data rotation, scaling, cropping and color adjustment; The preprocessed sample data is divided into data according to a preset division ratio to obtain the training data set, the verification data set and the test data set.

3. The three-dimensional model defect detection method according to claim 1, characterized in that: The step of converting the data in the training data set into two-dimensional images at multiple angles to obtain a target two-dimensional image, and performing defect detection on the target two-dimensional image using the initial defect detection model to obtain a defect detection result includes: Using three-dimensional rendering technology to convert each data in the training data set into two-dimensional images at multiple angles to obtain an initial two-dimensional image; Performing image enhancement and image unification processing on the initial two-dimensional image to obtain a basic two-dimensional image; Marking the defective area of ​​the basic two-dimensional image to obtain the target two-dimensional image; The target two-dimensional image is subjected to defect detection using the initial defect detection model to obtain the defect detection result.

4. The three-dimensional model defect detection method according to claim 3, characterized in that: The performing defect detection on the target two-dimensional image by using the initial defect detection model to obtain the defect detection result includes: Positioning and identifying the defect area in the target two-dimensional image by using the initial defect detection model to generate an initial defect detection result, wherein the initial defect detection result includes the bounding box coordinates of the defect position, the defect type and the confidence level of the defect; The initial defect detection results of the same defect area are fused and corrected to obtain the defect detection result.

5. The three-dimensional model defect detection method according to claim 1, characterized in that: The calculating model index parameters and model loss according to the defect detection result, and adjusting the model parameters of the initial defect detection model according to the model index parameters and the model loss, comprises: Calculating the model index parameters according to the defect detection result, wherein the model index parameters include accuracy, recall rate, F1 score, average precision and mAP; The defect area positioning and defect classification losses are calculated based on the defect detection result and the annotation information in the verification data set by using a preset loss function to obtain the positioning loss and the classification loss; Calculate the model loss according to the positioning loss and the classification loss; The model parameters of the initial defect detection model are adjusted according to the model indicator parameters and the model loss.

6. The three-dimensional model defect detection method according to claim 1, characterized in that: The iterative training of the adjusted initial defect detection model again through the target two-dimensional image to obtain the target defect detection model includes: Iteratively training the adjusted initial defect detection model using the target two-dimensional image, generating a new model loss in each iterative training; When the new model loss reaches a preset threshold or the number of iterations reaches a preset number, the iterative training is stopped to obtain a trained defect detection model; The trained defect detection model is tested using the test data set to obtain the target defect detection model.

7. The three-dimensional model defect detection method according to any one of claims 1 to 6, characterized in that: The acquiring of the to-be-detected three-dimensional model data and detecting the to-be-detected three-dimensional model data by using the target defect detection model to obtain the target defect detection result includes: Acquire the three-dimensional model data to be detected, and convert the three-dimensional model data to be detected into two-dimensional images at multiple angles to obtain the two-dimensional images to be detected; Performing image enhancement and image cropping on the two-dimensional image to be detected to obtain a target two-dimensional image to be detected; The target defect detection model is used to detect the target two-dimensional image to obtain the target defect detection result.

8. A three-dimensional model defect detection device, characterized in that: include: A sample data acquisition unit, used to acquire three-dimensional model sample data, and perform data preprocessing and data division on the three-dimensional model sample data to generate a training data set, a verification data set, and a test data set; A detection model initialization unit, used to use the YOLO model as a defect detection model, and initialize the network structure and model parameters of the defect detection model to obtain an initial defect detection model; a two-dimensional image conversion unit, configured to convert the data in the training data set into two-dimensional images at multiple angles to obtain a target two-dimensional image, and perform defect detection on the target two-dimensional image using the initial defect detection model to obtain a defect detection result; A model parameter adjustment unit, configured to calculate a model index parameter and a model loss according to the defect detection result, and adjust the model parameters of the initial defect detection model according to the model index parameter and the model loss; A detection model training unit, used for iteratively training the adjusted initial defect detection model again through the target two-dimensional image to obtain a target defect detection model; The target defect detection result unit is used to obtain the three-dimensional model data to be detected, detect the three-dimensional model data to be detected by using the target defect detection model, and obtain the target defect detection result.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the three-dimensional model defect detection method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the three-dimensional model defect detection method according to any one of claims 1 to 7 is implemented.

Images