3D scanning processing methods, apparatus, and computer equipment based on prior data

Abstract

This invention provides a 3D scanning processing method, apparatus, and computer device based on prior data, relating to the field of 3D scanning technology. When performing 3D scanning on a scanned object, this invention acquires an initial reference model and 3D positioning data of the scanned object. The initial reference model is prior data representing the surface information of the scanned object. The initial reference model is aligned with the 3D positioning data to obtain a target reference model. Based on the target reference model, the scanned object is processed using 3D scanning to obtain the target object's surface information. By introducing prior data representing the surface information of the scanned object during 3D scanning, the invention improves scanning accuracy and efficiency, reduces the complexity of scanning data processing algorithms, and thus enhances the user experience.

Description

Technical Field

[0001] This invention relates to the field of 3D scanning technology, and in particular to a 3D scanning processing method, apparatus, and computer equipment based on prior data. Background Technology

[0002] Currently, the surface information of an object is typically obtained by performing a 3D scan. The specific processing flow of a 3D scan includes:

[0003] 1. Acquire multiple local 3D point clouds on the surface of an object using multi-view vision and structured light;

[0004] 2. Stitch and align multiple local 3D point clouds to obtain a 3D point cloud;

[0005] 3. Generate a 3D mesh from the 3D point cloud to obtain the surface information of the object.

[0006] However, the aforementioned 3D scanning methods have low scanning accuracy and high complexity in the scanning data processing algorithms, resulting in low scanning efficiency and a poor user experience. Summary of the Invention

[0007] The purpose of this invention is to provide a three-dimensional scanning processing method, apparatus, and computer equipment based on prior data, so as to improve scanning accuracy and efficiency, reduce the complexity of scanning data processing algorithms, and thus improve user experience.

[0008] In a first aspect, embodiments of the present invention provide a three-dimensional scanning processing method based on prior data, including:

[0009] Acquire the initial reference model and 3D positioning data of the object being scanned; wherein, the initial reference model is prior data used to represent the surface information of the object being scanned;

[0010] Align the initial reference model with the 3D positioning data to obtain the target reference model;

[0011] Based on the target reference model, the scanned object is subjected to three-dimensional scanning processing to obtain the surface information of the target object.

[0012] Furthermore, the three-dimensional positioning data is obtained by scanning the positioning object corresponding to the scanned object, the preset area of ​​the scanned object, or the outline of the scanned object; wherein, the positioning object and the scanned object have a preset relative positional relationship, and the positioning object has preset features.

[0013] Furthermore, the initial reference model is aligned with the 3D positioning data to obtain the target reference model, including:

[0014] Feature recognition is performed on the 3D positioning data to obtain target positioning features; among which, target positioning features include preset features corresponding to the positioning object, regional features corresponding to the preset area, or contour features of the scanned object;

[0015] Based on the target localization features, the initial reference model is transformed into the target coordinate system corresponding to the three-dimensional localization data to obtain the target reference model.

[0016] Furthermore, based on the target reference model, a 3D scanning process is performed on the scanned object to obtain the surface information of the target object, including:

[0017] Acquire real-time image data obtained from a 3D scan of the object being scanned;

[0018] Based on the target reference model, real-time data processing is performed on real-time image data to obtain real-time target data; the data processing includes 3D reconstruction and point cloud registration, as well as real-time meshing or point cloud fusion.

[0019] Based on the target reference model, non-real-time data processing is performed on all acquired real-time target data to obtain the surface information of the target object. The non-real-time data processing includes mesh optimization processing corresponding to real-time meshing, or non-real-time meshing and mesh optimization processing corresponding to point cloud fusion. The mesh optimization processing includes mesh accuracy optimization and / or mesh filtering and denoising processing.

[0020] Furthermore, real-time data processing includes 3D reconstruction, point cloud registration, and real-time meshing; based on the target reference model, real-time data processing is performed on the real-time image data to obtain the target's real-time data, including:

[0021] Based on the object surface position information and object surface normal information of the target reference model, three-dimensional reconstruction is performed on real-time image data to obtain the first point cloud data;

[0022] Based on the object surface geometry information of the target reference model, vertex weight optimization is performed on the first point cloud data to obtain the second point cloud data;

[0023] Point cloud registration is performed on the second point cloud data to obtain the third point cloud data;

[0024] Based on the nearest distance from the point to the object surface in the target reference model, noise identification and removal are performed on each point in the third point cloud data to obtain the fourth point cloud data.

[0025] The fourth point cloud data is processed into a grid in real time to obtain the initial grid data;

[0026] Based on the geometric information of the target reference model, the initial mesh data is optimized for the target region to obtain real-time target data; the optimization of the target region's mesh accuracy includes downsampling of mesh vertices in flat regions.

[0027] Furthermore, based on the geometric information of the target reference model, the initial mesh data is optimized for the target region to obtain real-time target data, including:

[0028] Based on the geometric information of the target reference model, target regions are identified from the initial grid data; the target regions include flat regions and feature regions.

[0029] The grid accuracy of the identified target area is optimized accordingly to obtain real-time target data; the grid accuracy optimization of the target area also includes upsampling of grid vertices or reduction of point spacing within the feature area.

[0030] Furthermore, non-real-time data processing includes mesh optimization processing corresponding to real-time meshing, which includes mesh accuracy optimization and mesh filtering and denoising processing; based on the target reference model, non-real-time data processing is performed on all acquired real-time target data to obtain the target object surface information, including:

[0031] Based on the geometric information of the target reference model, the grid vertices in the flat area are downsampled and the grid vertices in the feature area are upsampled or the point distance is reduced to obtain the initial optimized data.

[0032] Based on the geometric information of the target reference model, all initial optimization data are subjected to mesh filtering and denoising to obtain the surface information of the target object; in the mesh filtering and denoising process, neighborhood calculation is performed along the direction of minimum principal curvature of the point to be processed.

[0033] Secondly, embodiments of the present invention also provide a three-dimensional scanning processing apparatus based on prior data, comprising:

[0034] The acquisition module is used to acquire the initial reference model and 3D positioning data of the scanned object; wherein, the initial reference model is prior data used to represent the surface information of the scanned object;

[0035] The alignment module is used to align the initial reference model with the 3D positioning data to obtain the target reference model;

[0036] The processing module is used to perform 3D scanning processing on the scanned object based on the target reference model to obtain the surface information of the target object.

[0037] Thirdly, embodiments of the present invention also provide a computer device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the three-dimensional scanning processing method based on prior data of the first aspect.

[0038] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to perform the three-dimensional scanning processing method based on prior data of the first aspect.

[0039] The 3D scanning processing method, apparatus, and computer device based on prior data provided in this invention acquire an initial reference model and 3D positioning data of the scanned object during 3D scanning. The initial reference model is prior data representing the surface information of the scanned object. The initial reference model is aligned with the 3D positioning data to obtain a target reference model. Based on the target reference model, 3D scanning processing is performed on the scanned object to obtain the target object's surface information. By introducing prior data representing the object's surface information during 3D scanning, the scanning accuracy and efficiency can be improved, while reducing the complexity of the scanning data processing algorithm, thereby enhancing the user experience. Attached Figure Description

[0040] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 A flowchart illustrating a three-dimensional scanning processing method based on prior data provided in an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of a coding point on a positioning object provided in an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of a minimum principal curvature direction provided for an embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram of the structure of a three-dimensional scanning processing device based on prior data provided in an embodiment of the present invention;

[0045] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0046] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] Current 3D scanning technology has the following problems:

[0048] 1. The scanning efficiency is not high enough, and the computational resource consumption is large;

[0049] 2. The scanning accuracy is not high enough;

[0050] 3. It is greatly affected by noise, and noise points are prone to appear in the 3D point cloud;

[0051] 4. Due to the influence of point cloud noise, the mesh generation quality is unstable, and noise and impurities are prone to appear in areas such as detailed features, model corners, and thin walls;

[0052] 5. Due to noise, the accuracy of scanning holes, edges, and other locations on the object is not high enough.

[0053] Based on this, the present invention provides a three-dimensional scanning processing method, apparatus, and computer equipment based on prior data, which can at least solve one of the above problems.

[0054] To facilitate understanding of this embodiment, a detailed description of a three-dimensional scanning processing method based on prior data disclosed in this embodiment of the invention will be provided first.

[0055] This invention provides a three-dimensional scanning processing method based on prior data, which can be executed by a computer device with data processing capabilities. See also... Figure 1 The diagram shows a flowchart of a three-dimensional scanning processing method based on prior data. The method mainly includes the following steps S110 to S130:

[0056] Step S110: Obtain the initial reference model and three-dimensional positioning data of the scanned object; wherein, the initial reference model is prior data used to represent the surface information of the scanned object.

[0057] An initial reference model of the object to be scanned can be imported before or during the scanning process. The object to be scanned can be any object requiring 3D scanning and possessing prior data. This prior data can be, but is not limited to, CAD (Computer-Aided Design) data, point cloud data, or mesh data. For example, in industrial inspection scenarios, to check the quality of components processed by machine tools, a 3D model of the component can be obtained by scanning it, allowing for quality inspection based on the 3D model. Therefore, the object to be scanned can be a component processed by machine tools, and its prior data can be the CAD data or other processing data used in processing the component. Similarly, in 3D printing quality inspection scenarios, a 3D model of the object to be printed can be obtained by scanning it, allowing for 3D printing quality inspection based on the 3D model. Therefore, the object to be scanned can be an object obtained through 3D printing, and its prior data can be the CAD data used in 3D printing. The object to be scanned can also be any object requiring 3D scanning, and its prior data can be point cloud data or mesh data obtained through measurements using appropriate precision equipment. In addition, grid data can also be obtained by converting CAD data.

[0058] To align the initial reference model with the scanned object's scan data, it is necessary to acquire the object's 3D positioning data. Optionally, the 3D positioning data can be obtained by scanning a positioning object corresponding to the object, a preset area of ​​the object, or the outline of the object; wherein, the positioning object and the object have a preset relative positional relationship, and the positioning object has preset features.

[0059] In one possible implementation, the aforementioned 3D positioning data can be positioning object image data obtained by scanning the positioning object corresponding to the scanned object. The 3D scanner can include a single camera or at least two cameras. Based on this, the positioning object image data can be at least two grayscale images obtained by a single camera capturing the positioning object, or at least one grayscale image obtained by each of at least two cameras capturing the positioning object (i.e., each camera captures at least one grayscale image of the positioning object). The positioning object can be an object with distinct features relative to the scanned object, that is, the preset features of the positioning object are significantly different from the features of the scanned object. The preset features on the positioning object can be set according to actual needs and are not limited here. The positioning object can be an object inherent on the mounting platform where the scanned object is located, or an object artificially placed at a specific location on the scanned object; the positioning object can be a single object or a combination of multiple sub-objects. For example, the positioning object is an object with a sticker such as... Figure 2The cube shown contains a coding point, which can be attached to one or more faces of the cube. This coding point is a preset feature on the cube. It should be noted that the coding point can be set according to actual needs and is not limited to... Figure 2 The shape shown.

[0060] In another possible implementation, the aforementioned 3D positioning data can be image data of a preset region of the scanned object obtained by pre-scanning a specific area (i.e., a preset region). This positioning image data can be at least two grayscale images obtained by a single camera capturing the preset region of the scanned object, or at least one grayscale image obtained by at least two cameras each capturing the preset region of the scanned object (i.e., each camera captures at least one grayscale image of the preset region of the scanned object). The preset region of the scanned object can be a region with a specific pattern, which can uniquely locate the scanned object. In a specific implementation, the user can be prompted to pre-scan a specific region of the scanned object to obtain the image data of the preset region of the scanned object.

[0061] In another possible implementation, the aforementioned 3D positioning data can be the contour data of the scanned object obtained by scanning the approximate contour of the scanned object using a fast scanning method. The fast scanning method has a higher scanning speed than the normal scanning method used for subsequent 3D scanning of the scanned object, but its scanning accuracy is lower than that of the normal scanning method.

[0062] Step S120: Align the initial reference model with the 3D positioning data to obtain the target reference model.

[0063] The aforementioned alignment refers to transforming the initial reference model to a specific target coordinate system. This target coordinate system can be the coordinate system corresponding to the 3D positioning data. During subsequent point cloud registration, all relevant data can be transformed to this 3D positioning data to facilitate subsequent data processing. Considering that the position of a 3D scanner may shift when scanning an object, with each scanning position corresponding to a coordinate system, the 3D positioning data may involve one coordinate system (e.g., image data of the object being scanned taken by at least two cameras at one location, or image data of a preset area of ​​the scanned object), or it may involve multiple coordinate systems (e.g., image data of the object being scanned taken by a single camera or at least two cameras at at least two locations, image data of a preset area of ​​the scanned object, or contour data of the scanned object). When multiple coordinate systems are involved, one coordinate system can be selected as the target coordinate system (e.g., the coordinate system corresponding to the first scanning position, or other coordinate systems can be selected), and the 3D positioning data can be pre-aligned. Then, the initial reference model is aligned with the pre-aligned 3D positioning data to obtain the target reference model.

[0064] In some possible embodiments, step S120 may include: performing feature recognition on the three-dimensional positioning data to obtain target positioning features; wherein, the target positioning features include preset features corresponding to the positioning object, region features corresponding to a preset region, or contour features of the scanned object; based on the target positioning features, transforming the initial reference model to the target coordinate system corresponding to the three-dimensional positioning data to obtain the target reference model. The aforementioned three-dimensional positioning data may be pre-aligned three-dimensional positioning data.

[0065] Optionally, based on the target positioning features, a coordinate transformation matrix can be first determined between the coordinate system of the initial reference model and the target coordinate system corresponding to the 3D positioning data. Then, based on this coordinate transformation matrix, the initial reference model is transformed into the target coordinate system to obtain the target reference model. Specifically, if the 3D positioning data is the aforementioned image data of the positioning object, the position information of the preset features corresponding to the positioning object in the coordinate system of the initial reference model can be determined based on the relative positional relationship between the positioning object and the scanned object. Then, based on the position information of the preset features in the coordinate system of the initial reference model and the position information of the preset features in the image data of the positioning object, a coordinate transformation matrix between the coordinate system of the initial reference model and the target coordinate system corresponding to the image data of the positioning object is obtained. Finally, based on this coordinate transformation matrix, the initial reference model is transformed into the target coordinate system to obtain the target reference model. If the 3D positioning data is the image data of a preset area of ​​the scanned object or the contour data of the scanned object, the target positioning features in the 3D positioning data (i.e., the regional features corresponding to the preset area or the contour features of the scanned object) can be first stitched (i.e. matched) with the initial reference model to obtain the position information of the target positioning features in the initial reference model. Then, based on the position information of the target positioning features in the initial reference model and the position information of the target positioning features in the 3D positioning data, the coordinate transformation matrix between the coordinate system of the initial reference model and the target coordinate system corresponding to the 3D positioning data is obtained. Based on this coordinate transformation matrix, the initial reference model is transformed into the target coordinate system to obtain the target reference model.

[0066] Step S130: Based on the target reference model, perform three-dimensional scanning processing on the scanned object to obtain the surface information of the target object.

[0067] After obtaining the target reference model through the alignment operation in step S120, a 3D scan of the object can begin. The information in the target reference model is used to optimize the various algorithms in the scanning process. If an initial reference model is imported during the scanning process, the scanned data prior to importing the initial reference model can be processed first, or the scanned data can be discarded, and the 3D scan of the object can be performed again.

[0068] In some possible embodiments, during the scanning process, the real-time image data obtained from the 3D scanning of the scanned object can be processed in real time using the target reference model. Alternatively, at the end of the scanning process, all the real-time target data obtained after the real-time data processing can be processed non-real-time using the target reference model to obtain the surface information of the target object. Based on this, step S130 may include the following steps S131 to S133:

[0069] Step S131: Obtain real-time image data obtained by performing a 3D scan on the object being scanned.

[0070] Real-time image data can include image data captured at one location of the scanned object. If the 3D scanner uses a single camera, the real-time image data can be at least two grayscale images captured at least twice by the 3D scanner at the current location of the scanned object (the location of the 3D scanner is different at different capture times); if the 3D scanner uses at least two cameras, the real-time image data can be at least two grayscale images captured once by the 3D scanner at the current location of the scanned object.

[0071] Step S132: Based on the target reference model, perform real-time data processing on the real-time image data to obtain the target real-time data; wherein, the data processing includes 3D reconstruction and point cloud registration, and also includes real-time meshing or point cloud fusion.

[0072] Step S132 above is a real-time processing process, which requires high processing speed. Real-time meshing or point cloud fusion can be selected according to actual needs.

[0073] Step S133: Based on the target reference model, perform non-real-time data processing on all acquired real-time target data to obtain the surface information of the target object; wherein, the non-real-time data processing includes mesh optimization processing corresponding to real-time meshing, or non-real-time meshing and mesh optimization processing corresponding to point cloud fusion, and the mesh optimization processing includes mesh accuracy optimization and / or mesh filtering and denoising processing.

[0074] Step S133 described above is not a real-time processing procedure and has relatively low requirements for processing speed. It is mainly used to optimize the surface features and edge corner areas of the model, thereby improving the overall effect of the 3D scan. The optimization results of all the real-time target data corresponding to the scanned object are the target object surface information of the scanned object.

[0075] In this embodiment, the optimization process of introducing prior data improves scanning accuracy, reduces the complexity of all algorithms in the scanning data processing pipeline, optimizes efficiency, and enhances user experience.

[0076] The 3D scanning processing method based on prior data provided in this invention acquires an initial reference model and 3D positioning data of the scanned object during 3D scanning. The initial reference model is prior data representing the surface information of the scanned object. The initial reference model is aligned with the 3D positioning data to obtain a target reference model. Based on the target reference model, the scanned object is subjected to 3D scanning processing to obtain the target object's surface information. By introducing prior data representing the object's surface information during 3D scanning, the method improves scanning accuracy and efficiency, reduces the complexity of the scanning data processing algorithm, and thus enhances the user experience.

[0077] This invention provides two 3D scanning processing methods. Method 1 mainly includes steps such as 3D reconstruction, point cloud registration, real-time meshing, and mesh optimization. Method 2 mainly includes steps such as 3D reconstruction, point cloud registration, point cloud fusion, non-real-time meshing, and mesh optimization. Generally, considering the high performance requirements of real-time meshing, when the performance of the computer equipment performing the above-mentioned 3D scanning processing method based on prior data is poor, the processing flow of Method 2 (corresponding to non-real-time meshing) can be selected; conversely, when the performance of the computer equipment is superior, the processing flow of Method 1 (corresponding to real-time meshing) can be selected. It should be noted that these two processing flows are optional and can be selected according to user needs. Even if the performance of the computer equipment is superior, the processing flow of Method 2 can still be selected.

[0078] To facilitate understanding, the two methods described above will be explained in detail below.

[0079] Method 1:

[0080] Real-time image data can first be obtained by 3D reconstruction to obtain initial point cloud data of a single frame, then by point cloud registration to obtain point cloud data in the target coordinate system, then by real-time meshing to obtain mesh data, and finally by mesh optimization to obtain surface information of the target object.

[0081] In the first method, step S132 can be implemented through the following process: Based on the object surface position information and object surface normal information of the target reference model, perform 3D reconstruction on the real-time image data to obtain first point cloud data; based on the object surface geometric information of the target reference model, perform vertex weight optimization on the first point cloud data to obtain second point cloud data; perform point cloud registration on the second point cloud data to obtain third point cloud data; based on the nearest distance from the point to the object surface in the target reference model, perform noise identification and noise removal on each point in the third point cloud data to obtain fourth point cloud data; perform real-time meshing processing on the fourth point cloud data to obtain initial mesh data; based on the geometric information of the target reference model, optimize the mesh accuracy of the initial mesh data for the target region to obtain real-time target data; wherein, the mesh accuracy optimization for the target region includes downsampling of mesh vertices in flat regions.

[0082] In the first method, step S133 above can be implemented through the following process: based on the geometric information of the target reference model, downsampling of grid vertices in flat areas and upsampling or point spacing reduction of grid vertices in feature areas are performed on all real-time target data to obtain initial optimized data; based on the geometric information of the target reference model, grid filtering and denoising processing is performed on all initial optimized data to obtain target object surface information; wherein, in the process of grid filtering and denoising processing, neighborhood calculation is performed along the direction of minimum principal curvature of the point to be processed.

[0083] The following is a detailed description of the processing flow for Method 1.

[0084] The accuracy and efficiency of laser line reconstruction 3D point algorithms can be improved by utilizing a target reference model. Specifically, by using the object surface position information and surface normal information of the target reference model for real-time 3D reconstruction of image data, mismatches during the 3D reconstruction process can be reduced, matching efficiency improved, and the scanning frame rate optimized. The vertex weight optimization process described above is an online matching process for 3D reconstruction. When optimizing the vertex weights of the first point cloud data using the object surface geometric information of the target reference model, the vertex weights in areas affecting vertex accuracy can be reduced based on the physical characteristics of the laser line (laser line scanning cameras in 3D scanners scan by emitting laser lines). This reduces the vertex weights in areas such as convex corners (to reduce the impact of energy overflow), concave corners (to reduce the impact of specular reflection), and edges (to reduce the impact of missing lines), thereby optimizing the accuracy of positions such as holes and edges on the scanned object and improving overall accuracy. Vertex weights are used to characterize the quality and accuracy of the corresponding points. For example:

[0085] When calculating the laser point normal vector (the input data required for meshing), traditional methods generally require finding nearby points, determining the laser line direction based on the nearby points, and calculating the normal vector using the directions of multiple laser lines. In this embodiment, however, the corresponding surface in the target reference model can be found directly based on the laser point coordinates, and the known normal vector of that surface in the target reference model can be used.

[0086] By utilizing the nearest distance from a point to the surface of an object in the target reference model, noise generated by various errors can be removed, optimizing the efficiency of noise removal. Specifically, during noise identification and removal, each point in the third-party point cloud data can first be converted into the target reference model, and the nearest distance from each point to the surface of the object in the target reference model can be calculated. It is then determined whether this nearest distance reaches a preset distance threshold. If it does, the point is identified as noise and removed; otherwise, it is determined not to be noise and retained. The distance threshold can be set according to actual needs and is not limited here.

[0087] When optimizing the mesh accuracy of a target region, the number of mesh vertices in flat regions can be reduced by downsampling, thus improving mesh generation efficiency. When the computer equipment executing this method meets performance requirements, the accuracy of feature regions can be improved by using smaller pixel pitches (to enhance mesh refinement) or by appropriately increasing the number of vertices in feature regions, thereby optimizing the 3D scanning effect of feature regions and achieving optimization of mesh generation efficiency and quality. Based on this, the aforementioned real-time target data can be obtained through the following process: Based on the geometric information of the target reference model, target region identification is performed on the initial mesh data; where the target region includes flat regions and feature regions; the identified target regions are then subjected to corresponding mesh accuracy optimization to obtain real-time target data; where the mesh accuracy optimization of the target region also includes upsampling of mesh vertices or reduction of pixel pitches within the feature regions. Flat regions refer to areas in an image with relatively uniform visual characteristics. These regions may lack obvious color, texture, or shape variations and appear relatively uniform. Flat regions do not contain much visual detail or information and have a relatively small role in image analysis and understanding; therefore, the number of mesh vertices can be appropriately reduced. Feature regions refer to areas in an image that possess specific visual attributes. These regions may stand out due to their color, texture, shape, or other visual characteristics. Feature regions can include, but are not limited to, areas with edges, textures, and color variations. Improving the accuracy of feature regions can enhance the quality of 3D scanning.

[0088] Optionally, when downsampling the grid vertices within the flat region of all target real-time data, the downsampling factor can be greater than the downsampling factor when downsampling the grid vertices within the flat region of the initial grid data. That is, the reduction in the number of grid vertices within the flat region during real-time processing (corresponding to step S132) is greater than the reduction in the number of grid vertices within the flat region during non-real-time processing (corresponding to step S133). For example, the number of vertices in the flat region can be reduced by 30% during real-time processing, while the number of vertices in the flat region can be reduced by 80% during non-real-time processing. This ensures the real-time performance requirements of the real-time processing while improving its processing efficiency, and further enhances the grid generation effect, thus optimizing both the grid generation effect and efficiency.

[0089] In the mesh filtering denoising process, neighborhood calculation can be performed along the direction of minimum principal curvature of the feature, that is, only considering the direction of the least surface change, which can enhance the edge corner effect. The direction of minimum principal curvature refers to the direction where, at a point on the surface, there are infinitely many orthogonal curvatures, among which there exists a curve whose curvature is maximized, and the curvature perpendicular to the surface with the maximum curvature is minimized by a minimum value Kmin. The direction containing this minimum value Kmin is the direction of minimum principal curvature. For example, ... Figure 3 As shown, the curvature of point P on the broken line in the figure along the direction of the arrow is 0. This arrow direction is the direction of minimum principal curvature of point P. When calculating the neighborhood of point P, only points along the direction of minimum principal curvature are considered. That is, the neighborhood calculation only considers the neighboring points of point P along the broken line direction shown by the arrow. This achieves the best corner feature preservation effect. In this way, by using mesh filtering denoising based on the geometric information of the target reference model, the influence of point cloud noise can be reduced, the stability of mesh generation quality can be improved, and to a certain extent, the noise and specks that are prone to appear in detailed features, model corners, thin walls, and other areas can be avoided.

[0090] It should be noted that the geometric information of the target reference model provides its basic framework, while the surface geometry of the target reference model further refines and improves this framework. Together, they determine the visual appearance and physical properties of the target reference model. The surface geometry of the target reference model can include information such as the normal direction of each face, surface roughness, and texture coordinates; the geometric information of the target reference model can include information such as the vertex coordinates, edge lengths, and face shapes of all faces.

[0091] Method 2:

[0092] Real-time image data can first be obtained by 3D reconstruction to obtain initial point cloud data of a single frame, then by point cloud registration to obtain point cloud data in the target coordinate system, then by point cloud fusion to merge point cloud data from multiple perspectives, then by non-real-time meshing to obtain mesh data, and finally by mesh optimization to obtain surface information of the target object.

[0093] In Method 2, step S132 can be implemented as follows: Based on the object surface position information and object surface normal information of the target reference model, perform 3D reconstruction on the real-time image data to obtain the first point cloud data; based on the object surface geometric information of the target reference model, perform vertex weight optimization on the first point cloud data to obtain the second point cloud data; perform point cloud registration on the second point cloud data to obtain the third point cloud data; based on the nearest distance from the point to the object surface in the target reference model, perform noise identification and noise removal on each point in the third point cloud data to obtain the fourth point cloud data; based on one or more of the object surface position information, object surface normal information, and object surface geometric information in the target reference model, perform point cloud fusion on the fourth point cloud data to obtain the target real-time data.

[0094] In Method 2, step S133 can be implemented as follows: perform non-real-time meshing on all real-time target data to obtain initial mesh data; based on the geometric information of the target reference model, downsample the mesh vertices in flat areas and upsample or reduce the point spacing of mesh vertices in feature areas to obtain initial optimized data; based on the geometric information of the target reference model, perform mesh filtering and denoising on the initial optimized data to obtain the surface information of the target object; wherein, during the mesh filtering and denoising process, neighborhood calculation is performed along the direction of minimum principal curvature of the point to be processed.

[0095] For the parts not described in detail in Method 2 above, please refer to the corresponding content in Method 1, which will not be repeated here.

[0096] To facilitate understanding, the implementation process of the above-mentioned three-dimensional scanning processing method based on prior data will be introduced below, taking Method 1 as an example.

[0097] 1. Before or during scanning, import the initial reference model (CAD data, point cloud data, or mesh data) of the object being scanned.

[0098] 2. Pre-align the initial reference model with the scanned data to obtain the target reference model.

[0099] Alignment methods include, but are not limited to, one of the following:

[0100] 2.1. Alignment is achieved using the positioning objects on the mounting platform of the object being scanned;

[0101] 2.2. Alignment is achieved using artificially added aids;

[0102] 2.3. Prompt the user to pre-scan a specific area of ​​the object being scanned for stitching;

[0103] 2.4. Users can quickly scan the approximate outline of the object to be scanned using a faster scanning method, and then use this outline to stitch the images together.

[0104] 3. After alignment, scanning begins. Information from the target reference model is used to optimize various algorithms during the scanning process, including:

[0105] 3.1. Improved Accuracy and Efficiency of Laser Line Reconstruction Algorithm for 3D Points

[0106] Based on the object surface position information and object surface normal information of the target reference model, false matching is reduced and matching efficiency is improved.

[0107] Based on the object surface geometry information of the target reference model, during the online matching process, to address the issue of vertex accuracy, the vertex weights in areas such as convex corners (energy overflow), concave corners (specular reflection), and edges (line missing) are reduced according to the physical characteristics of the laser line, thereby improving the overall accuracy.

[0108] 3.2. Scan Noise Removal

[0109] Based on the shortest distance from the generated point to the surface of the object in the target reference model, noise points caused by various errors are removed.

[0110] 3.3. Optimization of Mesh Generation Effect and Efficiency

[0111] Based on the geometric information of the target reference model, the number of mesh vertices in flat areas is reduced, thereby improving mesh generation efficiency.

[0112] 3.4. Optimization of model surface features and edge corner regions

[0113] Based on the geometric information of the target reference model, smaller point spacing can be used for the feature regions, or the number of vertices in the feature regions can be appropriately increased to optimize the feature region effect.

[0114] During the mesh filtering denoising process, the neighborhood calculation is performed along the direction of the minimum principal curvature of the feature to enhance the edge corner effect.

[0115] Compared to 3D scanning schemes without prior data, the embodiments of the present invention can improve scanning efficiency, improve scanning accuracy, reduce scanning noise points, and improve the data quality of details in the 3D model of the scanned object.

[0116] Corresponding to the above-described 3D scanning processing method based on prior data, this embodiment of the invention also provides a 3D scanning processing apparatus based on prior data. See also... Figure 4 The diagram shows a structural schematic of a 3D scanning processing device based on prior data. The device includes:

[0117] The acquisition module 401 is used to acquire the initial reference model and three-dimensional positioning data of the scanned object; wherein, the initial reference model is prior data used to represent the surface information of the scanned object;

[0118] Alignment module 402 is used to align the initial reference model with the 3D positioning data to obtain the target reference model;

[0119] The processing module 403 is used to perform three-dimensional scanning processing on the scanned object based on the target reference model to obtain the surface information of the target object.

[0120] The 3D scanning processing apparatus based on prior data provided in this embodiment of the invention acquires an initial reference model and 3D positioning data of the scanned object through an acquisition module 401 when performing 3D scanning on the object. The initial reference model is prior data representing the surface information of the scanned object. An alignment module 402 aligns the initial reference model with the 3D positioning data to obtain a target reference model. A processing module 403 then performs 3D scanning processing on the scanned object based on the target reference model to obtain the target object's surface information. By introducing prior data representing the object's surface information during 3D scanning, the apparatus improves scanning accuracy and efficiency, reduces the complexity of the scanning data processing algorithm, and thus enhances the user experience.

[0121] Furthermore, the aforementioned three-dimensional positioning data is obtained by scanning the positioning object corresponding to the scanned object, the preset area of ​​the scanned object, or the outline of the scanned object; wherein, the positioning object and the scanned object have a preset relative positional relationship, and the positioning object has preset features.

[0122] Furthermore, the alignment module 402 is specifically used for: performing feature recognition on the three-dimensional positioning data to obtain target positioning features; wherein, the target positioning features include preset features corresponding to the positioning object, regional features corresponding to the preset area, or contour features of the scanned object; based on the target positioning features, converting the initial reference model to the target coordinate system corresponding to the three-dimensional positioning data to obtain the target reference model.

[0123] Further, the aforementioned processing module 403 is specifically used for: acquiring real-time image data obtained by performing a three-dimensional scan of the scanned object; performing real-time data processing on the real-time image data based on the target reference model to obtain target real-time data; wherein, the data processing includes three-dimensional reconstruction and point cloud registration, and also includes real-time meshing or point cloud fusion; and performing non-real-time data processing on all acquired target real-time data based on the target reference model to obtain target object surface information; wherein, the non-real-time data processing includes mesh optimization processing corresponding to real-time meshing, or non-real-time meshing and mesh optimization processing corresponding to point cloud fusion, and the mesh optimization processing includes mesh accuracy optimization and / or mesh filtering and denoising processing.

[0124] Furthermore, real-time data processing includes 3D reconstruction, point cloud registration, and real-time meshing; the aforementioned processing module 403 is also used for: performing 3D reconstruction on the real-time image data based on the object surface position information and object surface normal information of the target reference model to obtain first point cloud data; optimizing the vertex weights of the first point cloud data based on the object surface geometric information of the target reference model to obtain second point cloud data; performing point cloud registration on the second point cloud data to obtain third point cloud data; performing noise identification and noise removal on each point in the third point cloud data based on the nearest distance from the point to the object surface in the target reference model to obtain fourth point cloud data; performing real-time meshing processing on the fourth point cloud data to obtain initial mesh data; and optimizing the mesh accuracy of the initial mesh data for the target region based on the geometric information of the target reference model to obtain target real-time data; wherein, the mesh accuracy optimization for the target region includes downsampling of mesh vertices in flat regions.

[0125] Furthermore, the aforementioned processing module 403 is also used to: identify target regions in the initial mesh data based on the geometric information of the target reference model; wherein the target regions include flat regions and feature regions; optimize the mesh accuracy of the identified target regions accordingly to obtain real-time target data; wherein the mesh accuracy optimization of the target regions also includes upsampling of mesh vertices or reduction of point spacing within the feature regions.

[0126] Furthermore, the non-real-time data processing includes mesh optimization processing corresponding to real-time meshing, which includes mesh accuracy optimization and mesh filtering and denoising processing. The aforementioned processing module 403 is also used to: based on the geometric information of the target reference model, downsample the mesh vertices in flat areas and upsample or reduce the point spacing of mesh vertices in feature areas to obtain initial optimized data; based on the geometric information of the target reference model, perform mesh filtering and denoising processing on all initial optimized data to obtain the surface information of the target object; wherein, during the mesh filtering and denoising process, neighborhood calculation is performed along the direction of minimum principal curvature of the point to be processed.

[0127] The three-dimensional scanning processing device based on prior data provided in this embodiment has the same implementation principle and technical effect as the aforementioned three-dimensional scanning processing method based on prior data embodiment. For the sake of brevity, any parts not mentioned in the embodiment of the three-dimensional scanning processing device based on prior data can be referred to the corresponding content in the aforementioned three-dimensional scanning processing method based on prior data embodiment.

[0128] like Figure 5 As shown, an embodiment of the present invention provides a computer device 500, including: a processor 501, a memory 502 and a bus. The memory 502 stores a computer program that can run on the processor 501. When the computer device 500 is running, the processor 501 and the memory 502 communicate through the bus, and the processor 501 executes the computer program to implement the above-mentioned three-dimensional scanning processing method based on prior data.

[0129] Specifically, the memory 502 and processor 501 mentioned above can be general-purpose memory and processor, without any specific limitations here.

[0130] This invention also provides a computer-readable storage medium storing a computer program. When a processor runs this computer program, it executes the three-dimensional scanning processing method based on prior data described in the preceding method embodiments. The computer-readable storage medium includes various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), RAM, magnetic disk, or optical disk.

[0131] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0132] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limitations; therefore, other examples of exemplary embodiments may have different values.

[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0134] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0135] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0136] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A three-dimensional scanning processing method based on prior data, characterized in that, include: Acquire an initial reference model and three-dimensional positioning data of the object being scanned; wherein the initial reference model is prior data used to represent the surface information of the object being scanned; Align the initial reference model with the three-dimensional positioning data to obtain the target reference model; Based on the target reference model, a three-dimensional scanning process is performed on the scanned object to obtain the surface information of the target object, including: acquiring real-time image data obtained by three-dimensional scanning of the scanned object; performing real-time data processing on the real-time image data based on the target reference model to obtain real-time target data; wherein, the data processing includes three-dimensional reconstruction and point cloud registration, and also includes real-time meshing or point cloud fusion; and performing non-real-time data processing on all the acquired real-time target data based on the target reference model to obtain the surface information of the target object; wherein, the non-real-time data processing includes mesh optimization processing corresponding to the real-time meshing, or non-real-time meshing and mesh optimization processing corresponding to the point cloud fusion, wherein the mesh optimization processing includes mesh accuracy optimization and / or mesh filtering and denoising processing.

2. The method according to claim 1, characterized in that, The three-dimensional positioning data is obtained by scanning the positioning object corresponding to the scanned object, the preset area of ​​the scanned object, or the outline of the scanned object; wherein, the positioning object and the scanned object have a preset relative positional relationship, and the positioning object has preset features.

3. The method according to claim 2, characterized in that, Aligning the initial reference model with the three-dimensional positioning data to obtain the target reference model includes: The three-dimensional positioning data is subjected to feature recognition to obtain target positioning features; wherein, the target positioning features include preset features corresponding to the positioning object, region features corresponding to the preset region, or contour features of the scanned object; Based on the target positioning features, the initial reference model is transformed into the target coordinate system corresponding to the three-dimensional positioning data to obtain the target reference model.

4. The method according to claim 1, characterized in that, The real-time data processing includes 3D reconstruction, point cloud registration, and real-time meshing; the real-time data processing of the real-time image data based on the target reference model to obtain target real-time data includes: Based on the object surface position information and object surface normal information of the target reference model, the real-time image data is reconstructed in three dimensions to obtain the first point cloud data; Based on the object surface geometry information of the target reference model, vertex weight optimization is performed on the first point cloud data to obtain the second point cloud data; Point cloud registration is performed on the second point cloud data to obtain the third point cloud data; Based on the nearest distance from a point to the surface of an object in the target reference model, noise identification and removal are performed on each point in the third point cloud data to obtain the fourth point cloud data. The fourth point cloud data is processed into a real-time grid to obtain initial grid data; Based on the geometric information of the target reference model, the initial mesh data is optimized for the target region to obtain real-time target data; wherein, the optimization of the target region's mesh accuracy includes downsampling of mesh vertices within flat regions.

5. The method according to claim 4, characterized in that, The process of optimizing the mesh accuracy of the initial mesh data for the target region based on the geometric information of the target reference model to obtain real-time target data includes: Based on the geometric information of the target reference model, target regions are identified in the initial mesh data; wherein, the target regions include the flat regions and feature regions; The identified target region is subjected to corresponding grid precision optimization to obtain real-time target data; wherein, the grid precision optimization of the target region also includes upsampling of grid vertices or reduction of point spacing within the feature region.

6. The method according to claim 1, characterized in that, The non-real-time data processing includes mesh optimization processing corresponding to the real-time meshing, and the mesh optimization processing includes mesh accuracy optimization and mesh filtering and noise reduction processing. The step of performing non-real-time data processing on all the acquired real-time target data based on the target reference model to obtain the target object surface information includes: Based on the geometric information of the target reference model, the grid vertices in the flat area are downsampled and the grid vertices in the feature area are upsampled or the point distance is reduced to obtain the initial optimized data. Based on the geometric information of the target reference model, all the initial optimization data are subjected to mesh filtering and denoising processing to obtain the surface information of the target object; wherein, during the mesh filtering and denoising process, neighborhood calculation is performed along the direction of minimum principal curvature of the point to be processed.

7. A three-dimensional scanning processing device based on prior data, characterized in that, include: An acquisition module is used to acquire an initial reference model and three-dimensional positioning data of the scanned object; wherein, the initial reference model is prior data used to represent the surface information of the scanned object; An alignment module is used to align the initial reference model with the three-dimensional positioning data to obtain a target reference model; The processing module is used to perform three-dimensional scanning processing on the scanned object based on the target reference model to obtain the surface information of the target object; The processing module is specifically used for: acquiring real-time image data obtained by performing a 3D scan of the scanned object; performing real-time data processing on the real-time image data based on the target reference model to obtain target real-time data; wherein, the data processing includes 3D reconstruction and point cloud registration, and also includes real-time meshing or point cloud fusion; and performing non-real-time data processing on all the acquired target real-time data based on the target reference model to obtain target object surface information; wherein, the non-real-time data processing includes mesh optimization processing corresponding to the real-time meshing, or non-real-time meshing and mesh optimization processing corresponding to the point cloud fusion, and the mesh optimization processing includes mesh accuracy optimization and / or mesh filtering and denoising processing.

8. A computer device, characterized in that, It includes a memory and a processor; the memory stores a computer program that can run on the processor, and when the processor executes the computer program, it implements the three-dimensional scanning processing method based on prior data as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program is executed by the processor to perform the three-dimensional scanning processing method based on prior data as described in any one of claims 1-6.

Images