A method, device and storage medium for generating surface deformation defects

Abstract

The present invention relates to the field of data generation technology, and in particular to a method, device, and storage medium for generating surface deformation defects. The surface deformation defect generation method comprises the following steps: calculating a support plane for an input three-dimensional point cloud data set and performing dimensionality reduction to obtain an original depth map data set; extracting a defect morphological feature set from the original depth map data set; performing a random transformation on the defect morphological feature set and fusing it with a normal depth map data set to obtain a generated depth map data set. Existing small sample learning and data generation methods are difficult to meet actual production needs and cannot solve the problem of data scarcity. The present invention designs a surface deformation defect generation method that can effectively generate a large amount of data containing surface deformation defects that is authentic, diverse, and reliable. The data generation effect of the present invention is better than that of existing methods, and the gain for post-defect detection is also better than that of existing data generation and data enhancement methods.

Description

Technical Field

[0001] The present invention relates to the technical field of data generation, and in particular to a method, device and storage medium for generating surface deformation defects. Background Art

[0002] Thanks to the development of deep learning, deep learning-based defect detection algorithms have gained widespread demand and application in various production and processing industries, including heavy industry, light industry, and agriculture. However, existing deep learning-based defect detection algorithms rely on large amounts of training data (e.g., thousands or tens of thousands of examples). However, in practical applications of defect detection research, data scarcity is often encountered, and only a small amount of training data (e.g., a few or dozens of examples) is available. To address this data scarcity issue, academia and industry have conducted extensive research. Related research can be divided into two categories: small-sample learning methods and data generation methods.

[0003] Small-shot learning reduces a model's reliance on large amounts of data by improving its generalization capabilities. Meta-learning methods, a mainstream approach in the field, aim to leverage a large amount of pre-training data with diverse categories, a sophisticated meta-learning module, and a mature meta-learning model training paradigm to impart superior generalization capabilities. This allows the model to master a new target with minimal training data—for example, five images in a 5-shot paradigm—in order to identify a new target. However, the characteristics of actual defects differ significantly from the targets identified in publicly available pre-training data. Furthermore, the best meta-learning-based small-shot target detection algorithms still fall far short of the accuracy requirements for practical implementation.

[0004] Data generation methods augment training data by generating large amounts of data. These methods often use generative adversarial networks, which can make the results less reliable. More importantly, addressing data scarcity through data generation requires the model to be able to generate large amounts of data in the target category with only a small amount of data. Methods such as DAGAN have attempted this, but are still far from achieving small-sample generation across different defect categories in real-world data. Summary of the Invention

[0005] The purpose of the present invention is to address the problems existing in the background technology and to propose a method, device and storage medium for generating surface deformation defects.

[0006] The technical solution of the present invention is a method for generating surface deformation defects, comprising the following specific steps:

[0007] S1. Calculate the support plane of the input 3D point cloud dataset and perform dimensionality reduction to obtain the original depth map dataset;

[0008] S2. Perform morphological feature purification on the original depth map dataset and the corresponding annotations to obtain a defect morphological feature set;

[0009] S3. Perform random transformation on the defect morphological feature set and fuse it with the normal depth map data set to obtain a generated depth map data set.

[0010] Preferably, S1 further includes the following specific steps:

[0011] S11, traversing the input 3D point cloud dataset; solving for the 3D point cloud a supporting plane with the minimum sum of distances to all points, and the supporting plane satisfies the constraint that all points are on the same side of the supporting plane;

[0012] S12, rotate the support plane to the XY plane, using the same rotation transformation for all points and the support plane;

[0013] S13. Directly extract the Z direction value of the rotated point cloud data, arrange and interpolate the values ​​according to the X and Y directions, and obtain the original depth map data set in the form of a two-dimensional matrix.

[0014] Preferably, S2 further includes the following specific steps:

[0015] S21, eliminating depth variations other than those caused by the defect itself in the defect annotation frame area of ​​the original depth map data by surface fitting, to obtain defect features after surface correction;

[0016] S22. By calculating the mean square error between the defect features after surface correction and the XY plane and comparing them with the repair threshold, the defect features with large background noise are eliminated to obtain a defect morphological feature set.

[0017] Preferably, S3 includes the following specific steps:

[0018] S31. randomly selecting a defect feature from the defect morphology feature set, and performing a random transformation on the defect feature;

[0019] S32. Randomly select a normal depth map data from the normal depth map data set, and fuse it with the defect feature after random transformation to obtain generated depth map data, and then loop to obtain the generated depth map data set.

[0020] Preferably, the 3D point cloud dataset input in S1 is a collection of 3D point clouds obtained by rotationally scanning industrial products or agricultural and sideline products produced within a period of time and containing deformation defects that are manually detected using a high-precision 3D laser sensor.

[0021] Preferably, the normal depth map data in S3 is a collection of three-dimensional point clouds obtained by rotating and scanning industrial products or agricultural and sideline products produced within a period of time and manually identified as not containing deformation defects using a high-precision three-dimensional laser sensor.

[0022] Preferably, the generated depth map data set obtained in S3 includes data of surface deformation defects, which is used to expand the surface deformation defect data.

[0023] The present invention provides a surface deformation defect generation device, comprising a processor and a memory;

[0024] The memory stores a program or instruction, which is loaded and executed by the processor to implement the above-mentioned surface deformation defect generation method.

[0025] The present invention provides a computer-readable storage medium, which stores a program or instruction. When the program or instruction is executed by a processor, any step of the above-mentioned surface deformation defect generation method is implemented.

[0026] Compared with the prior art, the present invention has the following beneficial technical effects:

[0027] 1. Based on the characteristics of surface deformation defects and utilizing the actual production prior knowledge and detection requirements of deformation defects, the defect generation algorithm designed in this invention generates data with reliability, authenticity, and diversity, which can effectively supplement the data quantity and defect patterns and greatly improve the effect of post-defect detection.

[0028] 2. The present invention purifies the target defects in the point cloud data set and extracts the defect morphological feature set, then performs multiple random transformations on the defect morphological feature set and fuses it with normal data, thereby generating a large amount of data. The present invention only requires a small amount of training data and has high reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0030] Figure 2 A comparison chart of real defect data and defect data generated by the present invention. DETAILED DESCRIPTION

[0031] Example 1

[0032] Referring to the process of the present invention (such as Figure 1 ), suppose there is an existing 3D point cloud dataset S origin3d , then this embodiment specifically includes the following steps:

[0033] 1. For the 3D point cloud dataset S origin3d All 3D point clouds in Calculate the support plane and perform dimensionality reduction to obtain the original depth map dataset. The specific steps are as follows:

[0034] The first step is to solve the distance 3D point cloud The supporting plane P with the smallest sum of distances between all points support And the supporting plane P support All points are on the supporting plane P support According to the principle of laser 3D scanning sensor, the support plane of the point cloud obtained by scanning is actually the polar coordinate expansion of the 3D scanning sensor trajectory surface with the workpiece centroid as the reference system. Let P support ={(x,y,z)|Ax+By+Cz+D=0}, that is, solve the Lagrangian optimization problem in the following formula.

[0035]

[0036] The second step is to support the plane P support Rotate to the XY plane and use the same rotation transformation for all points and the support plane. It is easy to know that the normal vector of the support plane is (A, B, C). According to the three-dimensional coordinate rotation formula, let θ x =arctan(B / C),θ y =arctan(A / C), then the specific rotation matrix M is expressed as the formula:

[0037]

[0038] Multiply all points by the rotation matrix M to obtain the point cloud data after perspective correction After the rotation, the Z direction information in the point cloud is completely preserved, and the X and Y direction information can be converted into the subscript of the two-dimensional matrix, and then the Z direction component can be extracted as the depth channel. Directly extract the Z direction value, arrange and interpolate according to the X and Y direction values, and obtain a depth map in the form of a two-dimensional matrix. Because the workpiece retains the Z direction information of the laser 3D scanning sensor to the maximum extent after rotation, the X and Y direction values ​​of the point cloud data are now eliminated through matrix arrangement, and the Z direction values ​​of the point cloud are arranged into a two-dimensional matrix according to the actual position and interval relationship, and the depth map can be obtained. First, determine all sampling points of the point cloud according to the preset size (w, h) of the depth map. Because the sampling density of the laser 3D scanning sensor in the X and Y directions is different, (w, h) needs to be determined according to the actual geometric relationship of the workpiece during sampling, rather than by uniformly dividing the point cloud. Then use the nearest neighbor interpolation method for all sampling points, calculate the Z direction value of each point as the depth, and obtain complete depth map data. Depth map data It is a two-dimensional matrix of size (w,h), shaped like a normal single-channel image, and can be viewed and edited with normal image software.

[0039] 2. For the original depth map data S originDepthThe morphological features of the corresponding annotations are purified to obtain the defect morphological feature set; the specific steps are as follows:

[0040] In the first step, surface deformation defects are manifested as depth changes in the 3D point cloud. The texture of the workpiece surface and its own geometric shape will also cause depth changes. In order to extract accurate defect morphological features, the original depth map data must be eliminated first. Defect annotation box in Depth changes in the area other than those caused by the defects themselves. Extend outward by 5 pixels and fill the edge of the depth map with null values. The set of all non-empty points in the extended area is called P extended , note the annotation box The set of all non-empty points in is P inside . Use the least squares method to fit the point set P through the quadratic surface extended Get the surface equation z = F(x, y), eliminate the depth change caused by the surface for the points inside and outside the frame, and get the point set P inside the frame after the surface correction fixedInside ={(x,y,zF(x,y))|(x,y,z)∈P inside}, and the point set P outside the frame after surface correction fixedExtended ={(x,y,zF(x,y))|(x,y,z)∈P extended}.

[0041] In the second step, for defects that appear in more complex or strong texture areas, the correction by quadratic surface mentioned above is difficult to achieve the expected effect, so such defects should be discarded. By calculating the surface-corrected point set P fixedExtended The mean square error of the XY plane and the repair threshold TH resh old mse Determine whether Defects within P morph ology If the point set P outside the box fixedExtended The mean square error of the XY plane is greater than the repair threshold Th reshold mse , then discard it. Otherwise, the corrected point set P in the box fixedInside Add morphological feature set P morph ology By traversing the above steps one and two for the original depth map data and all its defect annotations, the final morphological feature set P can be obtained. morph ology .

[0042] 3. Defect morphological feature set P morph ology After random transformation, it is fused with the normal depth map data to obtain the generated depth map data S synth DepthBecause the surface deformation defects appear as depth changes in the depth map, the morphological feature set P morph plogy By randomly transforming the defect features in the dataset and superimposing them on the depth map of the normal data, a large amount of defect data with new patterns can be obtained. morph plogy Randomly select a defect And perform random transformation. The specific random transformation methods are enumerated as follows: The X, Y, and Z axis coordinates are randomly scaled. The scaling ratio is determined by the maximum and minimum sizes of each axis of the defect in the purified defect set; Perform coordinate rotation parallel to the XY plane, where the rotation angle range is [0,2π); Add the bitwise combination with another random defect to fuse; Perform affine transformation parallel to the XY plane; Superimpose Gaussian noise.

[0043] The second step is to start from the original depth map set S originDepth Randomly select a depth map data Then the defect after random transformation In the selected depth map data The random positions are bitwise added to generate the depth map data Traverse the above process to generate the depth map data S synth Depth .

[0044] Based on 61 original 3D point cloud data sets obtained from 3D sensor scanning of defective products containing slight surface deformation defects detected by inspectors during a two-month production period of a cooperative company, the present invention generated 900 generated depth maps and merged them with the 61 original 3D point clouds as training data. Figure 2 The comparison results between the real data and the generated data are shown. It can be seen that the defect data generated by the present invention is very close to the real data. There is no obvious unfused texture in the edge area of ​​the defect. The generated defect data is diverse, authentic, and reliable. Defects appear in different texture areas, which effectively enriches the defect pattern in the training data.

[0045] Example 2

[0046] The present invention also provides a surface deformation defect generation device, comprising a processor and a memory; the memory stores a program or instruction, and the program or instruction is loaded and executed by the processor to implement the surface deformation defect generation method of embodiment 1.

[0047] Example 3

[0048] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions, which, when executed on a computer, cause the computer to execute the surface deformation defect generation method of Example 1.

[0049] Those skilled in the art will clearly understand that the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0050] The method, device, and storage medium for generating surface deformation defects provided by the present invention can be implemented in a variety of ways and means. The foregoing merely represents a preferred embodiment of the present invention. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications are also within the scope of protection of the present invention. Any components not specified in this embodiment may be implemented using existing technologies.

Claims

1. A method for generating surface deformation defects, characterized in that: The specific steps include: S1. Calculate the support plane of the input 3D point cloud dataset and perform dimensionality reduction to obtain the original depth map dataset; S11, traversing the input 3D point cloud dataset; solving for the 3D point cloud a supporting plane with the minimum sum of distances to all points, and the supporting plane satisfies the constraint that all points are on the same side of the supporting plane; S12, rotate the support plane to the XY plane, using the same rotation transformation for all points and the support plane; S13, directly extracting the Z direction value of the rotated point cloud data, arranging and interpolating the values ​​according to the X and Y directions, and obtaining an original depth map dataset in the form of a two-dimensional matrix; S2. Perform morphological feature purification on the original depth map dataset and the corresponding annotations to obtain a defect morphological feature set; S21, eliminating depth variations other than those caused by the defect itself in the defect annotation frame area of ​​the original depth map data by surface fitting, to obtain defect features after surface correction; S22, by calculating the mean square error of the defect features after surface correction and the XY plane and comparing them with the repair threshold, eliminating the defect features with large background noise to obtain a defect morphological feature set; S3, performing random transformation on the defect morphological feature set and fusing it with the normal depth map data set to obtain a generated depth map data set; S31. randomly selecting a defect feature from the defect morphology feature set, and performing a random transformation on the defect feature; S32. Randomly select a normal depth map data from the normal depth map data set, and fuse it with the defect feature after random transformation to obtain generated depth map data, and then loop to obtain the generated depth map data set.

2. A method for generating surface deformation defects according to claim 1, characterized in that: The 3D point cloud dataset input in S1 is a collection of 3D point clouds obtained by rotating and scanning industrial products or agricultural and sideline products that have been manually inspected and contain deformation defects produced over a period of time using a high-precision 3D laser sensor.

3. The method for generating surface deformation defects according to claim 1, wherein: The normal depth map data in S3 is a collection of three-dimensional point clouds obtained by rotating and scanning industrial products or agricultural and sideline products produced within a period of time and manually identified as not containing deformation defects using a high-precision three-dimensional laser sensor.

4. The method for generating surface deformation defects according to claim 1, wherein: The generated depth map data set obtained in S3 includes data of surface deformation defects, and is used to expand the surface deformation defect data.

5. A surface deformation defect generating device, characterized in that: including processor and memory; The memory stores a program or instruction, which is loaded and executed by the processor to implement the surface deformation defect generation method according to any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, any step of the surface deformation defect generation method according to any one of claims 1 to 4 is implemented.

Images