Three-dimensional scanning processing method and apparatus based on prior data, and computer device

Abstract

The present invention relates to the technical field of three-dimensional scanning, and provides a three-dimensional scanning processing method and apparatus based on prior data, and a computer device. The method of the present invention comprises: during three-dimensional scanning of an object to be scanned, acquiring an initial reference model and three-dimensional positioning data of the object to be scanned, wherein the initial reference model is prior data used for representing object surface information of the object; aligning the initial reference model with the three-dimensional positioning data to obtain a target reference model; and, on the basis of the target reference model, performing three-dimensional scanning processing on the object to obtain target object surface information. When three-dimensional scanning processing is performed on an object to be scanned, prior data representing object surface information of the object is introduced. In this way, by utilizing the prior data, scanning accuracy and scanning efficiency can be improved, and the complexity of scanning data processing algorithms can be reduced, thereby enhancing user experience.

Description

Three-dimensional scanning processing method and device based on prior data and computer equipment Cross-reference to related applications

[0000] This application claims priority to the Chinese patent application No. 202411276895.1, filed on September 12, 2024, with the China Patent Office, the title of which is “Three-dimensional scanning processing method and device based on prior data and computer equipment”, the entire content of which is incorporated herein by reference. The Chinese patent application No. 202411276895.1, filed on September 12, 2024, with the China Patent Office, the title of which is “Three-dimensional scanning processing method and device based on prior data and computer equipment”, the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional scanning, in particular to a three-dimensional scanning processing method and device based on prior data and computer equipment. BACKGROUND

[0002] Currently, the surface information of an object is gradually obtained by three-dimensional scanning of the object, and the specific processing flow of three-dimensional scanning includes:

[0003] 1. Obtain multiple local three-dimensional point clouds of the surface of the object through multi-view vision and structured light;

[0004] 2. Align and splice the multiple local three-dimensional point clouds to obtain a three-dimensional point cloud;

[0005] 3. Generate a three-dimensional mesh through the three-dimensional point cloud to obtain the surface information of the object.

[0006] However, the above-mentioned three-dimensional scanning processing method has low scanning accuracy, high complexity of scanning data processing algorithm, and low scanning efficiency, resulting in poor user experience. SUMMARY

[0007] The present application aims to provide a three-dimensional scanning processing method and device based on prior data and computer equipment to improve scanning accuracy and scanning efficiency, reduce the complexity of scanning data processing algorithm, and thus improve user experience.

[0008] In a first aspect, the present application provides a three-dimensional scanning processing method based on prior data, comprising: obtaining an initial reference model and three-dimensional positioning data of a scanned object; wherein the initial reference model is prior data used to represent the surface information of the scanned object; aligning the initial reference model with the three-dimensional positioning data to obtain a target reference model; and performing three-dimensional scanning processing on the scanned object based on the target reference model to obtain target surface information of the object.

[0009] Further, the three-dimensional positioning data is obtained by scanning a positioning object corresponding to the scanned object, a preset region of the scanned object, or a contour of the scanned object; wherein the positioning object and the scanned object have a preset relative position relationship, and the positioning object has a preset feature.

[0010] Further, the initial reference model is aligned with the three-dimensional positioning data to obtain a target reference model, including: performing feature recognition on the three-dimensional positioning data to obtain target positioning features; wherein the target positioning features include the preset features corresponding to the positioning object, the region features corresponding to the preset region, or the contour features of the scanned object; and converting the initial reference model to a target coordinate system corresponding to the three-dimensional positioning data based on the target positioning features to obtain the target reference model.

[0011] Further, based on the target reference model, the scanned object is subjected to three-dimensional scanning processing to obtain target object surface information, including: obtaining real-time image data obtained by three-dimensional scanning of the scanned object; performing real-time data processing on the real-time image data based on the target reference model to obtain target real-time data; wherein the data processing includes three-dimensional reconstruction and point cloud registration, and further includes real-time meshing or point cloud fusion; and performing non-real-time data processing on all the obtained target real-time data based on the target reference model to obtain the target object surface information; wherein the non-real-time data processing includes mesh optimization processing corresponding to the real-time meshing, or non-real-time meshing and mesh optimization processing corresponding to the point cloud fusion, and the mesh optimization processing includes mesh precision optimization and / or mesh filtering and denoising processing.

[0012] Further, the real-time data processing includes three-dimensional reconstruction, point cloud registration, and real-time meshing; the real-time data processing on the real-time image data based on the target reference model to obtain the target real-time data includes: performing three-dimensional reconstruction on the real-time image data based on object surface position information and object surface normal information of the target reference model to obtain first point cloud data; performing vertex weight optimization on the first point cloud data based on object surface geometry information of the target reference model to obtain second point cloud data; performing point cloud registration on the second point cloud data to obtain third point cloud data; performing outlier identification and outlier removal on each point in the third point cloud data based on the nearest distance from the point to the object surface in the target reference model to obtain fourth point cloud data; performing real-time meshing processing on the fourth point cloud data to obtain initial mesh data; and performing mesh precision optimization of a target region on the initial mesh data based on the geometry information of the target reference model to obtain the target real-time data; wherein the mesh precision optimization of the target region includes down-sampling of mesh vertices in a flat region.

[0013] Further, based on the geometry information of the target reference model, the initial mesh data is subjected to mesh accuracy optimization in the target region to obtain target real-time data, including: based on the geometry information of the target reference model, the initial mesh data is subjected to target region identification; wherein the target region includes a flat region and a feature region; the identified target region is subjected to corresponding mesh accuracy optimization to obtain target real-time data; wherein the mesh accuracy optimization of the target region further includes upsampling of mesh vertices or point distance reduction in the feature region.

[0014] Further, the non-real-time data processing includes mesh optimization processing corresponding to real-time meshing, and the mesh optimization processing includes mesh accuracy optimization and mesh filtering denoising processing; based on the target reference model, all the obtained target real-time data is subjected to non-real-time data processing to obtain target object surface information, including: based on the geometry information of the target reference model, the mesh vertices in the flat region of all the target real-time data are subjected to downsampling, and the mesh vertices in the feature region are subjected to upsampling or point distance reduction to obtain initial optimization data; based on the geometry information of the target reference model, all the initial optimization data is subjected to mesh filtering denoising processing to obtain target object surface information; wherein in the mesh filtering denoising processing, neighborhood calculation is performed along the minimum principal curvature direction of the to-be-processed point.

[0015] In a second aspect, an embodiment of the present application further provides a three-dimensional scanning processing device based on prior data, including:

[0016] The acquisition module is configured to acquire an initial reference model and three-dimensional positioning data of a scanned object; wherein the initial reference model is prior data used to represent object surface information of the scanned object; the alignment module is configured to align the initial reference model with the three-dimensional positioning data to obtain a target reference model; and the processing module is configured to perform three-dimensional scanning processing on the scanned object based on the target reference model to obtain target object surface information.

[0017] In a third aspect, an embodiment of the present application further provides a computer device, including a memory and a processor, and the memory stores a computer program capable of running on the processor; when the processor executes the computer program, the first aspect of the three-dimensional scanning processing method based on prior data is implemented.

[0018] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program; when the processor runs the computer program, the first aspect of the three-dimensional scanning processing method based on prior data is executed.

[0019] The three-dimensional scanning processing method, device and computer device based on prior data provided by the embodiment of the present application can obtain an initial reference model and three-dimensional positioning data of a scanned object when performing three-dimensional scanning on the scanned object, wherein the initial reference model is prior data used to represent object surface information of the scanned object; the initial reference model is aligned with the three-dimensional positioning data to obtain a target reference model; and the scanned object is processed based on the target reference model to obtain target object surface information. When performing three-dimensional scanning on the scanned object, the prior data used to represent object surface information of the scanned object is introduced, so that the scanning accuracy and scanning efficiency can be improved by using the prior data, the complexity of the scanning data processing algorithm is reduced, and the user experience is improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the specific embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Fig. 1 is a flow diagram of a three-dimensional scanning processing method based on prior data provided by an embodiment of the present application;

[0022] Fig. 2 is a schematic diagram of an encoding point on a positioning object provided by an embodiment of the present application;

[0023] Fig. 3 is a schematic diagram of a minimum principal curvature direction provided by an embodiment of the present application;

[0024] Fig. 4 is a structural schematic diagram of a three-dimensional scanning processing device based on prior data provided by an embodiment of the present application;

[0025] Fig. 5 is a structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0026] The technical solutions of the present application will be described in detail below with reference to the embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0027] The existing three-dimensional scanning technology has the following problems:

[0028] 1. The scanning efficiency is not high, and the computing resource consumption is large;

[0029] 2. The scanning accuracy is not high;

[0030] 3. The three-dimensional point cloud is greatly affected by noise, and noise points are prone to appear in the three-dimensional point cloud;

[0031] 4. The quality of the generated mesh is unstable due to the influence of point cloud noise, and noise points and noise are prone to appear in areas such as detailed features, model corners, and thin walls;

[0032] 5. The accuracy of the positions such as holes and edges on the scanned object is not high enough due to the influence of noise points.

[0033] Therefore, the embodiment of the present application provides a three-dimensional scanning processing method and device based on prior data and a computer device, which can at least solve one of the above problems.

[0034] In order to facilitate the understanding of the present embodiment, first, a three-dimensional scanning processing method based on prior data disclosed by the present embodiment is introduced in detail.

[0035] The embodiment of the present application provides a three-dimensional scanning processing method based on prior data, which can be executed by a computer device with data processing capability. Referring to a flowchart of a three-dimensional scanning processing method based on prior data shown in FIG. 1, the method mainly includes the following steps S110 to S130:

[0036] Step S110, obtaining an initial reference model and three-dimensional positioning data of a scanned object; wherein the initial reference model is prior data used to represent the object surface information of the scanned object.

[0037] The initial reference model of the scanned object can be imported before scanning the scanned object or during the process of scanning the scanned object. The scanned object can be an object with three-dimensional scanning requirements and prior data, which can be, but is not limited to, CAD (Computer Aided Design) data, point cloud data, mesh data, etc. For example, in an industrial detection scenario, it is necessary to detect whether a device processed by a machine tool or the like is qualified, and the three-dimensional model of the device can be obtained by scanning the device, so that quality detection is performed based on the three-dimensional model. Based on this, the scanned object can be a device processed by a machine tool or the like, and the prior data can be CAD data or other processing data used when processing the device. For another example, in a three-dimensional printing quality detection scenario, a three-dimensional model of a printed object can be obtained by scanning the printed object, so that three-dimensional printing quality detection is performed based on the three-dimensional model. Based on this, the scanned object can be an object obtained by three-dimensional printing, and the prior data can be CAD data or the like used when three-dimensional printing. The scanned object can also be any object with three-dimensional scanning requirements, and the prior data can be point cloud data or mesh data measured by a corresponding precision device. In addition, the mesh data can also be obtained by converting CAD data.

[0038] In order to realize the alignment of the initial reference model and the scanning data of the scanned object, three-dimensional positioning data of the scanned object needs to be obtained. Optionally, the three-dimensional positioning data can be obtained by scanning a positioning object corresponding to the scanned object, a preset region of the scanned object, or a contour of the scanned object, etc. The positioning object has a preset relative positional relationship with the scanned object, and has a preset feature.

[0039] In a possible implementation, the three-dimensional positioning data can be positioning object image data obtained by scanning a positioning object corresponding to the scanned object; the three-dimensional scanner can include a single camera or at least two cameras, based on which the positioning object image data can be at least two gray-scale images obtained by photographing the positioning object by the single camera, or at least one gray-scale image obtained by photographing the positioning object by each of the at least two cameras (that is, each camera photographs at least one gray-scale image of the positioning object). The positioning object can be an object with obvious features relative to the scanned object, that is, the preset features of the positioning object are obviously different from the features of the scanned object. The preset features of the positioning object can be set according to actual needs, which are not limited herein. The positioning object can be an object inherent to the mounting table on which the scanned object is located, or an object manually placed at a specific position of the scanned object; the positioning object can be an independent object or a combination of multiple sub-objects. For example, the positioning object is a cube with code points as shown in FIG. 2, and the code points can be attached to one face or multiple faces of the cube, and the code points are the preset features of the cube. It should be noted that the code points can be set according to actual needs, and are not limited to the shapes shown in FIG. 2.

[0040] In another possible implementation, the three-dimensional positioning data can be preset region image data of the scanned object obtained by pre-scanning a specific region (that is, a preset region) of the scanned object. The positioning object image data can be at least two gray-scale images obtained by photographing the preset region of the scanned object by a single camera, or at least one gray-scale image obtained by photographing the preset region of the scanned object by each of the at least two cameras (that is, each camera photographs at least one gray-scale image of the preset region of the scanned object). The preset region of the scanned object can be a region with a specific pattern, and the specific pattern can uniquely locate the scanned object. When implemented, the user can be prompted to pre-scan the specific region of the scanned object to obtain the preset region image data of the scanned object.

[0041] In yet another possible implementation, the three-dimensional positioning data can be contour data of the scanned object obtained by scanning the general contour of the scanned object in a fast scanning manner. The scanning speed of the fast scanning manner is greater than the scanning speed of a normal scanning manner in subsequent three-dimensional scanning of the scanned object, and the scanning accuracy of the fast scanning manner is lower than the scanning accuracy of the normal scanning manner.

[0042] In step S120, the initial reference model is aligned with the three-dimensional positioning data to obtain a target reference model.

[0043] The alignment above refers to converting the initial reference model to a specific target coordinate system, which can be the coordinate system corresponding to the three-dimensional positioning data. During subsequent point cloud registration, the corresponding data can be converted to the three-dimensional positioning data to facilitate subsequent data processing. Considering that the three-dimensional scanner will move when scanning the object, one scanning position corresponds to one coordinate system, therefore the three-dimensional positioning data can involve one coordinate system (such as the three-dimensional positioning data being the positioning object image data or the preset region image data of the scanned object obtained by at least two cameras at one position), or can involve multiple coordinate systems (such as the three-dimensional positioning data being the positioning object image data, the preset region image data of the scanned object or the contour data of the scanned object obtained by a single camera or at least two cameras at at least two positions). When involving multiple coordinate systems, one of the coordinate systems can be selected as the target coordinate system (such as the coordinate system corresponding to the first scanning position, of course, other coordinate systems can also be selected), the three-dimensional positioning data is pre-aligned, and then the initial reference model is aligned with the pre-aligned three-dimensional positioning data to obtain the target reference model.

[0044] In some possible embodiments, the step S120 can include: performing feature recognition on the three-dimensional positioning data to obtain a target positioning feature; wherein the target positioning feature includes a preset feature corresponding to the positioning object, a region feature corresponding to the preset region, or a contour feature of the scanned object; and converting the initial reference model to a target coordinate system corresponding to the three-dimensional positioning data based on the target positioning feature to obtain the target reference model. The three-dimensional positioning data can be the pre-aligned three-dimensional positioning data.

[0045] Optionally, a coordinate conversion matrix between the coordinate system of the initial reference model and the target coordinate system corresponding to the three-dimensional positioning data can be determined based on the target positioning feature, and then the initial reference model can be converted to the target coordinate system based on the coordinate conversion matrix to obtain the target reference model. In a specific implementation, if the three-dimensional positioning data is the positioning object image data, the position information of the preset feature corresponding to the positioning object in the coordinate system of the initial reference model can be determined based on the relative positional relationship between the positioning object and the scanned object, and then the coordinate conversion matrix between the coordinate system of the initial reference model and the target coordinate system corresponding to the positioning object image data can be obtained based on the position information of the preset feature in the coordinate system of the initial reference model and the position information of the preset feature in the positioning object image data, and the initial reference model can be converted to the target coordinate system based on the coordinate conversion matrix to obtain the target reference model. If the three-dimensional positioning data is the preset region image data of the scanned object or the contour data of the scanned object, the target positioning feature (i.e., the region feature corresponding to the preset region or the contour feature of the scanned object) in the three-dimensional positioning data can be spliced (i.e., matched) with the initial reference model to obtain the position information of the target positioning feature in the initial reference model, and then the coordinate conversion matrix between the coordinate system of the initial reference model and the target coordinate system corresponding to the three-dimensional positioning data can be obtained based on the position information of the target positioning feature in the initial reference model and the position information of the target positioning feature in the three-dimensional positioning data, and the initial reference model can be converted to the target coordinate system based on the coordinate conversion matrix to obtain the target reference model.

[0046] In step S130, the scanned object is processed based on the target reference model to obtain the target object surface information.

[0047] After the target reference model is obtained through the alignment operation in step S120, the scanned object can be scanned, and the information in the target reference model can be used to optimize each algorithm in the scanning process. For the case where the initial reference model is imported in the scanning process, the scanned data before the initial reference model is imported can be processed preferentially, or the scanned data can be discarded, and the three-dimensional scanning of the scanned object can be performed again.

[0048] In some possible embodiments, real-time data processing can be performed on real-time image data obtained by three-dimensional scanning of the scanned object using the target reference model in the scanning process, and non-real-time data processing can be performed on all target real-time data obtained after real-time data processing using the target reference model when the scanning is completed, so as to obtain the target object surface information. Based on this, step S130 can include steps S131 to S133 as follows:

[0049] In step S131, real-time image data obtained by three-dimensional scanning of the scanned object is acquired.

[0050] The real-time image data can include image data taken at one position of the scanned object. If the three-dimensional scanner uses a single camera, the real-time image data can be at least two gray-scale images taken at the current position of the scanned object by the three-dimensional scanner at least twice (the positions of the three-dimensional scanner are different at different times of taking pictures); if the three-dimensional scanner uses at least two cameras, the real-time image data can be at least two gray-scale images taken at the current position of the scanned object by the three-dimensional scanner once.

[0051] In step S132, real-time data processing is performed on the real-time image data based on the target reference model to obtain target real-time data; wherein the data processing includes three-dimensional reconstruction and point cloud registration, and also includes real-time meshing or point cloud fusion.

[0052] The above step S132 is a real-time processing process, and the requirement for processing speed is high. Real-time meshing or point cloud fusion can be selected according to actual needs.

[0053] In step S133, non-real-time data processing is performed on all the target real-time data obtained based on the target reference model to obtain target object surface information; wherein the non-real-time data processing includes mesh optimization processing corresponding to real-time meshing, or non-real-time meshing and mesh optimization processing corresponding to point cloud fusion, and the mesh optimization processing includes mesh precision optimization and / or mesh filtering and denoising processing.

[0054] The above step S133 is not a real-time processing process, and the requirement for processing speed is low. The optimization processing is mainly used for optimizing the model surface features and edge corner regions, and improving the overall effect of three-dimensional scanning through the optimization processing. The optimization processing result of all the target real-time data of the scanned object is the target object surface information of the scanned object.

[0055] In the embodiment, the optimization process of introducing the prior data improves the scanning precision, reduces the complexity of all algorithms in the scanning data processing pipeline, optimizes the efficiency, and improves the user experience.

[0056] The three-dimensional scanning processing method based on prior data provided in the embodiment of the application includes the following steps: when a scanned object is scanned, an initial reference model and three-dimensional positioning data of the scanned object are obtained; wherein the initial reference model is prior data used to represent object surface information of the scanned object; the initial reference model is aligned with the three-dimensional positioning data to obtain a target reference model; and based on the target reference model, the scanned object is processed to obtain target object surface information. In the three-dimensional scanning processing of the scanned object, the prior data used to represent the object surface information of the scanned object is introduced, so that the scanning precision and scanning efficiency are improved by using the prior data, the complexity of the scanning data processing algorithm is reduced, and the user experience is improved.

[0057] The embodiments of the present application provide two three-dimensional scanning processing modes. The processing flow of mode one mainly includes steps of three-dimensional reconstruction, point cloud registration, real-time meshing and mesh optimization processing. The processing flow of mode two mainly includes steps of three-dimensional reconstruction, point cloud registration, point cloud fusion, non-real-time meshing and mesh optimization processing. Generally, considering that real-time meshing has higher requirement on device performance, when the performance of a computer device for executing the three-dimensional scanning processing method based on prior data is poor, the processing flow of mode two corresponding to non-real-time meshing can be selected. Conversely, when the performance of the computer device is better, the processing flow of mode one corresponding to real-time meshing can be selected. It should be noted that the processing flows of the two modes are optional flows, which can be selected according to user demand. Even if the performance of the computer device is better, the processing flow of mode two can also be selected.

[0058] For the convenience of understanding, the two modes are described in detail below.

[0059] Mode one:

[0060] The real-time image data can be firstly subjected to three-dimensional reconstruction to obtain initial point cloud data of a single frame, then subjected to point cloud registration to obtain point cloud data in a target coordinate system, then subjected to real-time meshing to obtain mesh data, and finally subjected to mesh optimization processing to obtain surface information of the target object.

[0061] In mode two, the step S132 can be implemented by the following process: based on the object surface position information and the object surface normal information of the target reference model, three-dimensional reconstruction is performed on the real-time image data to obtain first point cloud data; based on the object surface geometry information of the target reference model, vertex weight optimization is performed on the first point cloud data to obtain second point cloud data; point cloud registration is performed on the second point cloud data to obtain third point cloud data; based on the nearest distance from a point to the object surface in the target reference model, each point in the third point cloud data is subjected to outlier identification and outlier removal to obtain fourth point cloud data; real-time meshing is performed on the fourth point cloud data to obtain initial mesh data; based on the geometry information of the target reference model, mesh precision optimization of a target region is performed on the initial mesh data to obtain target real-time data; wherein the mesh precision optimization of the target region includes downsampling of mesh vertices in a flat region.

[0062] In the manner, the step S133 can be realized by the following process: based on the geometric information of the target reference model, downsampling the grid vertices in the flat region and upsampling the grid vertices or point distance reduction in the feature region of all target real-time data to obtain initial optimization data; based on the geometric information of the target reference model, performing grid filtering denoising processing on all initial optimization data to obtain the target object surface information; wherein, in the grid filtering denoising processing, the neighborhood calculation is performed along the minimum principal curvature direction of the to-be-processed point.

[0063] The processing flow of the above-mentioned manner one will be introduced in detail below.

[0064] The target reference model can be used to improve the accuracy and efficiency of the laser line reconstruction three-dimensional point algorithm. Specifically, using the object surface position information and object surface normal information of the target reference model for real-time image data three-dimensional reconstruction can reduce the mismatch in the three-dimensional reconstruction process, improve the matching efficiency, and optimize the scanning frame rate. The vertex weight optimization process is an online matching process for three-dimensional reconstruction. When the object surface geometric information of the target reference model is used to optimize the vertex weight of the first point cloud data, the vertex weight in the convex corner (to reduce the influence of energy overflow at this point), the concave corner (to reduce the influence of mirror reflection at this point), the edge (to reduce the influence of line loss at this point), and other regions can be reduced according to the physical characteristics of the laser line (the laser line scanning camera scans by emitting a laser line) to optimize the accuracy of the hole, edge and other positions on the scanned object, thereby improving the overall accuracy. The vertex weight is used to represent the quality and accuracy of the corresponding point. For example:

[0065] In calculating the laser point normal vector (input data required in grid processing), the traditional method generally needs to find the surrounding neighboring points, determine the laser line direction according to the neighboring points, and calculate the normal vector using the directions of multiple laser lines. The embodiment can directly find the corresponding surface in the target reference model according to the laser point coordinates, and use the known normal vector of the surface in the target reference model.

[0066] The nearest distance of the point to the object surface in the target reference model can be used to remove the noise points generated by various errors and optimize the noise point removal efficiency. Specifically, in the process of identifying and removing noise points, each point in the third point cloud data can be first converted into the target reference model, and the nearest distance of each point to the object surface in the target reference model is calculated to determine whether the nearest distance reaches a preset distance threshold. If yes, the point is determined as a noise point and removed; if not, the point is determined as not a noise point and retained. The distance threshold can be set according to actual needs, which is not limited here.

[0067] When the grid precision of the target area is optimized, the number of grid vertices of the flat area can be reduced by downsampling the grid vertices of the flat area, and the grid generation efficiency can be improved. When the performance requirements of the computer device executing the method are met, the precision of the feature area can be improved by using a smaller point distance to improve the fineness of the grid or by appropriately increasing the number of vertices in the feature area, thereby optimizing the three-dimensional scanning effect of the feature area, so as to realize the optimization of the grid generation efficiency and quality. Based on this, the above target real-time data can be achieved through the following process: based on the geometric information of the target reference model, the initial grid data is subjected to target area identification; wherein the target area includes a flat area and a feature area; the identified target area is subjected to corresponding grid precision optimization to obtain target real-time data; wherein the grid precision optimization of the target area further includes upsampling or point distance reduction of the grid vertices in the feature area. The flat area refers to an area in the image with relatively consistent visual characteristics. These areas may lack obvious color, texture or shape changes and appear relatively uniform. The flat area does not contain much visual detail or information, and has relatively small effect on image analysis and understanding, so the number of grid vertices can be appropriately reduced. The feature area refers to an area in the image with specific visual properties. These areas may appear different because of their color, texture, shape or other visual characteristics. The feature area can include but is not limited to edge, texture and color change areas. By improving the precision of the feature area, the three-dimensional scanning effect can be improved.

[0068] Optionally, when the grid vertices in the flat area of all target real-time data are downsampled, the downsampling multiple can be greater than the downsampling multiple when the grid vertices in the flat area of the initial grid data are downsampled, that is, the number of reduced grid vertices in the flat area in the real-time processing process (corresponding to step S132) is greater than the number of reduced grid vertices in the flat area in the non-real-time processing process (corresponding to step S133), for example, the flat area can reduce 30% of the number of vertices in the real-time processing process, and the flat area can reduce 80% of the number of vertices in the non-real-time processing process. In this way, the real-time requirements of the real-time processing process can be guaranteed, and the processing efficiency of the real-time processing process can be improved, and the grid generation effect can be further improved, that is, the optimization of the grid generation effect and efficiency is realized.

[0069] In the mesh filtering denoising process, the neighborhood calculation can be along the minimum principal curvature direction of the feature, that is, only the direction with the smallest change of the curved surface is considered, so that the edge corner effect can be enhanced. Among them, the minimum principal curvature direction refers to: there are infinite orthogonal curvatures on a certain point on the curved surface, and there is a curve such that the curvature of the curve is maximum, and the curvature perpendicular to the maximum curvature surface is minimum Kmin, and the direction of the minimum Kmin curvature is the minimum principal curvature direction. For example, as shown in FIG. 3, the curvature of point P on the broken line in the figure is 0 in the arrow direction, and the arrow direction is the minimum principal curvature direction of point P. When calculating the neighborhood of point P, only the points in the minimum principal curvature direction are taken into account for calculation, that is, only the points adjacent to point P in the direction of the arrow shown in the broken line are considered in the neighborhood calculation, so that the best corner feature reservation effect can be achieved. In this way, through the mesh filtering denoising processing based on the geometric information of the target reference model, the influence of the point cloud noise can be reduced, the stability of the mesh generation quality can be improved, and the noise and noise in the detail feature, model corner, thin wall and other areas can be avoided to a certain extent.

[0070] It should be noted that the above geometric information of the target reference model provides a basic framework for the target reference model, and the object surface geometric information of the target reference model is a further refinement and improvement of this basic framework, and the two together determine the visual performance and physical characteristics of the target reference model. The object surface geometric information of the target reference model can include the normal direction of each face, the roughness and texture coordinates of the surface, and the like; the geometric information of the target reference model can include the vertex coordinates of all faces, the length of the edge, and the shape of the face, and the like.

[0071] Method two:

[0072] The real-time image data can be first reconstructed by three-dimensional reconstruction to obtain initial point cloud data of a single frame, then registered by point cloud to obtain point cloud data in a target coordinate system, then fused by point cloud to merge point cloud data under multiple perspectives, and then processed by non-real-time meshing to obtain mesh data, and finally processed by mesh optimization to obtain target object surface information.

[0073] In the second way, the step S132 can be implemented by the following process: based on the object surface position information and the object surface normal information of the target reference model, performing three-dimensional reconstruction on the real-time image data to obtain first point cloud data; based on the object surface geometry information of the target reference model, performing vertex weight optimization on the first point cloud data to obtain second point cloud data; performing point cloud registration on the second point cloud data to obtain third point cloud data; based on the nearest distance from a point to the object surface in the target reference model, identifying and removing outliers in the third point cloud data to obtain fourth point cloud data; based on one or more of the object surface position information, the object surface normal information and the object surface geometry information in the target reference model, performing point cloud fusion on the fourth point cloud data to obtain target real-time data.

[0074] In the second way, the step S133 can be implemented by the following process: performing non-real-time meshing processing on all target real-time data to obtain initial mesh data; based on the geometry information of the target reference model, performing down-sampling of mesh vertices in flat regions and up-sampling or point distance reduction of mesh vertices in feature regions on the initial mesh data to obtain initial optimization data; based on the geometry information of the target reference model, performing mesh filtering and denoising processing on the initial optimization data to obtain target object surface information; wherein, in the mesh filtering and denoising processing, neighborhood calculation is performed along the direction of the minimum principal curvature of the point to be processed.

[0075] The parts not described in detail in the second way can refer to the corresponding contents in the first way, which will not be described here.

[0076] For ease of understanding, the implementation process of the above three-dimensional scanning processing method based on prior data will be introduced below by taking the first way as an example.

[0077] 1. Before or during scanning, import the initial reference model (CAD data, point cloud data or mesh data) of the scanned object.

[0078] 2. Pre-align the initial reference model with the scanning data to obtain a target reference model.

[0079] The alignment method includes but is not limited to one of the following:

[0080] 2.1. Aligning by using the positioning object on the mounting table of the scanned object;

[0081] 2.2. Aligning by using an artificially added auxiliary object;

[0082] 2.3. Prompting the user to pre-scan a specific area of the scanned object for splicing;

[0083] 2.4. The user uses a faster scanning method to quickly scan the approximate outline of the scanned object, and uses this outline for splicing.

[0084] 3. After alignment, start scanning, and use the information in the target reference model to optimize various algorithms during the scanning process, including:

[0085] 3.1. Precision and efficiency improvement of laser line reconstruction three-dimensional point algorithm

[0086] Based on the object surface position information and object surface normal information of the target reference model, reduce mismatching and improve matching efficiency.

[0087] Based on the object surface geometry information of the target reference model, in the online matching process, aiming at the problem of affecting the vertex accuracy, according to the physical characteristics of the laser line, the vertex weight in the convex corner (energy overflow), concave corner (mirror reflection), edge (line missing) and other regions is reduced, and the overall accuracy is improved.

[0088] 3.2. Scanning of miscellaneous points

[0089] Based on the nearest distance of the generated points to the object surface in the target reference model, remove miscellaneous points generated due to various errors.

[0090] 3.3. Optimization of grid generation effect and efficiency

[0091] Based on the geometry information of the target reference model, reduce the number of grid vertices in the flat area, and improve the grid generation efficiency.

[0092] 3.4. Optimization of model surface features and edge corner regions

[0093] Based on the geometry information of the target reference model, use smaller point distance for feature regions, or appropriately increase the number of vertices for feature regions, to optimize the effect of feature regions.

[0094] In the grid filtering and denoising process, the neighborhood calculation is along the minimum principal curvature direction of the feature, and the edge corner effect is strengthened.

[0095] Compared with the three-dimensional scanning scheme without prior data, the embodiment of the application can improve the scanning efficiency, improve the scanning precision, reduce the scanning noise points, and improve the data quality of the three-dimensional model details of the scanned object.

[0096] Corresponding to the above-mentioned three-dimensional scanning processing method based on prior data, the embodiment of the application also provides a three-dimensional scanning processing device based on prior data. Referring to the structure diagram of a three-dimensional scanning processing device based on prior data shown in FIG. 4, the device comprises:

[0097] The acquisition module 401 is configured to acquire an initial reference model of the scanned object and three-dimensional positioning data; wherein the initial reference model is prior data used to represent object surface information of the scanned object.

[0098] The alignment module 402 is configured to align the initial reference model with the three-dimensional positioning data to obtain a target reference model.

[0099] The processing module 403 is configured to perform three-dimensional scanning processing on the scanned object based on the target reference model to obtain target object surface information.

[0100] The three-dimensional scanning processing device based on prior data provided by the embodiment of the application, when performing three-dimensional scanning on the scanned object, the acquisition module 401 is configured to acquire an initial reference model of the scanned object and three-dimensional positioning data; wherein the initial reference model is prior data used to represent object surface information of the scanned object; the alignment module 402 is configured to align the initial reference model with the three-dimensional positioning data to obtain a target reference model; and the processing module 403 is configured to perform three-dimensional scanning processing on the scanned object based on the target reference model to obtain target object surface information. When performing three-dimensional scanning processing on the scanned object, the prior data used to represent object surface information of the scanned object is introduced, so that the scanning accuracy and scanning efficiency can be improved by using the prior data, the complexity of the scanning data processing algorithm is reduced, and the user experience is improved.

[0101] Further, the three-dimensional positioning data is obtained by scanning a positioning object corresponding to the scanned object, a preset region of the scanned object, or a contour of the scanned object; wherein the positioning object has a preset relative position relationship with the scanned object, and the positioning object has a preset feature.

[0102] Further, the alignment module 402 is specifically configured to: perform feature recognition on the three-dimensional positioning data to obtain a target positioning feature; wherein the target positioning feature includes a preset feature corresponding to the positioning object, a region feature corresponding to the preset region, or a contour feature of the scanned object; and convert the initial reference model to a target coordinate system corresponding to the three-dimensional positioning data based on the target positioning feature to obtain the target reference model.

[0103] Further, the processing module 403 is specifically configured to: acquire real-time image data obtained by three-dimensional scanning of the scanned object; perform real-time data processing on the real-time image data based on the target reference model to obtain target real-time data; wherein the data processing includes three-dimensional reconstruction and point cloud registration, and further includes real-time meshing or point cloud fusion; perform non-real-time data processing on all the acquired target real-time data based on the target reference model to obtain the target object surface information; wherein the non-real-time data processing includes mesh optimization processing corresponding to the real-time meshing, or non-real-time meshing and mesh optimization processing corresponding to the point cloud fusion, and the mesh optimization processing includes mesh precision optimization and / or mesh filtering denoising processing.

[0104] Further, the real-time data processing includes three-dimensional reconstruction, point cloud registration and real-time meshing; the processing module 403 is further configured to: perform three-dimensional reconstruction on the real-time image data based on the object surface position information and the object surface normal information of the target reference model to obtain first point cloud data; perform vertex weight optimization on the first point cloud data based on the object surface geometric information of the target reference model to obtain second point cloud data; perform point cloud registration on the second point cloud data to obtain third point cloud data; perform outlier identification and outlier removal on each point in the third point cloud data based on the nearest distance from the point to the object surface in the target reference model to obtain fourth point cloud data; perform real-time meshing processing on the fourth point cloud data to obtain initial mesh data; perform mesh precision optimization on the initial mesh data in a target region based on the geometric information of the target reference model to obtain the target real-time data; wherein the mesh precision optimization in the target region includes down-sampling of mesh vertices in a flat region.

[0105] Further, the processing module 403 is further configured to: perform target region identification on the initial mesh data based on the geometric information of the target reference model; wherein the target region includes a flat region and a feature region; perform corresponding mesh precision optimization on the identified target region to obtain the target real-time data; wherein the mesh precision optimization in the target region further includes up-sampling or point distance reduction of mesh vertices in the feature region.

[0106] Further, the non-real-time data processing includes mesh optimization processing corresponding to the real-time meshing, and the mesh optimization processing includes mesh precision optimization and mesh filtering denoising processing; the processing module 403 is further configured to: perform down-sampling of mesh vertices in a flat region and up-sampling or point distance reduction of mesh vertices in a feature region on all the target real-time data based on the geometric information of the target reference model to obtain initial optimization data; perform mesh filtering denoising processing on all the initial optimization data based on the geometric information of the target reference model to obtain the target object surface information; wherein, in the mesh filtering denoising processing, neighborhood calculation is performed along the direction of the minimum principal curvature of the point to be processed.

[0107] The three-dimensional scanning processing device based on prior data provided by the embodiment has the same implementation principle and technical effects as the three-dimensional scanning processing method based on prior data, and for brief description, the part of the three-dimensional scanning processing device based on prior data not mentioned in the embodiment can refer to the corresponding content in the three-dimensional scanning processing method based on prior data.

[0108] As shown in FIG. 5, the computer device 500 provided by the embodiment of the application includes a processor 501, a memory 502 and a bus. The memory 502 stores a computer program capable of running on the processor 501. When the computer device 500 runs, the processor 501 and the memory 502 communicate through the bus. The processor 501 executes the computer program to implement the three-dimensional scanning processing method based on prior data.

[0109] Specifically, the memory 502 and the processor 501 can be general memory and processor, which are not specifically limited here.

[0110] The embodiment of the application further provides a computer readable storage medium, which stores a computer program. When the computer program is run by a processor, the three-dimensional scanning processing method based on prior data in the foregoing method embodiment is executed. The computer readable storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk or an optical disk and various storage program codes.

[0111] The term "and / or" in the present document is only used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" in the present document means any one of the plurality or any combination of at least two of the plurality, for example, including at least one of A, B and C can mean including any one or more elements selected from the set consisting of A, B and C.

[0112] In all examples shown and described herein, any specific value should be interpreted as merely exemplary and not as a limitation, and thus other examples of the example embodiments can have different values.

[0113] The flow diagrams and the block diagrams in the drawings are presented to illustrate the architecture, functionality, and operation of possible implementations of apparatuses, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0114] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. The described apparatus embodiments are merely schematic. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, or a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, or the among different units, can be indirect couplings or communication connections through some interfaces, and can be in electrical, mechanical, or other forms.

[0115] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. In actual implementation, some or all of the units can be selected according to the actual needs to achieve the purposes of the embodiments.

[0116] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically as separated, or two or more units can be integrated in one unit.

[0117] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

[0118] Industrial applicability: the three-dimensional scanning processing method, device and computer equipment based on prior data provided by the embodiments of the present disclosure can obtain an initial reference model and three-dimensional positioning data of a scanned object when performing three-dimensional scanning on the scanned object; the initial reference model is prior data used to represent object surface information of the scanned object; the initial reference model is aligned with the three-dimensional positioning data to obtain a target reference model; and the scanned object is processed based on the target reference model to obtain target object surface information. When performing three-dimensional scanning processing on the scanned object, the prior data used to represent the object surface information of the scanned object is introduced, so that the scanning accuracy and scanning efficiency can be improved by using the prior data, the complexity of the scanning data processing algorithm is reduced, and the user experience is improved.

Claims

1. A three-dimensional scanning processing method based on prior data, comprising: obtaining an initial reference model of a scanned object and three-dimensional positioning data; wherein the initial reference model is prior data used to represent object surface information of the scanned object; aligning the initial reference model with the three-dimensional positioning data to obtain a target reference model; based on the target reference model, performing three-dimensional scanning processing on the scanned object to obtain target object surface information.

2. The method of claim 1, wherein, The three-dimensional positioning data is obtained by scanning a positioning object corresponding to the scanned object, a preset region of the scanned object, or a contour of the scanned object; wherein the positioning object and the scanned object have a preset relative positional relationship, and the positioning object has a preset feature.

3. The method of claim 2, wherein, The aligning the initial reference model with the three-dimensional positioning data to obtain a target reference model comprises: performing feature recognition on the three-dimensional positioning data to obtain target positioning features; wherein the target positioning features include preset features corresponding to the positioning object, region features corresponding to the preset region, or contour features of the scanned object; based on the target positioning features, converting the initial reference model to a target coordinate system corresponding to the three-dimensional positioning data to obtain a target reference model.

4. The method of any one of claims 1-3, wherein, The based on the target reference model, performing three-dimensional scanning processing on the scanned object to obtain target object surface information comprises: obtaining real-time image data obtained by three-dimensional scanning of the scanned object; based on the target reference model, performing real-time data processing on the real-time image data to obtain target real-time data; wherein the data processing includes three-dimensional reconstruction and point cloud registration, and also includes real-time meshing or point cloud fusion; based on the target reference model, performing non-real-time data processing on all the target real-time data obtained to obtain target object surface information; wherein the non-real-time data processing includes mesh optimization processing corresponding to the real-time meshing, or non-real-time meshing and mesh optimization processing corresponding to the point cloud fusion, and the mesh optimization processing includes mesh precision optimization and / or mesh filtering denoising processing.

5. The method of claim 4, wherein, The real-time data processing includes three-dimensional reconstruction, point cloud registration, and real-time meshing; the based on the target reference model, performing real-time data processing on the real-time image data to obtain target real-time data comprises: based on object surface position information and object surface normal information of the target reference model, performing three-dimensional reconstruction on the real-time image data to obtain first point cloud data; based on object surface geometry information of the target reference model, performing vertex weight optimization on the first point cloud data to obtain second point cloud data; performing point cloud registration on the second point cloud data to obtain third point cloud data; based on the nearest distance from a point to an object surface in the target reference model, performing outlier identification and outlier removal on each point in the third point cloud data to obtain fourth point cloud data; performing real-time meshing processing on the fourth point cloud data to obtain initial mesh data; The initial mesh data is subjected to mesh precision optimization of a target region based on geometric information of the target reference model, to obtain target real-time data; wherein the mesh precision optimization of the target region includes down-sampling of mesh vertices in a flat region.

6. The method of claim 5, wherein, The initial mesh data is subjected to mesh precision optimization of a target region based on geometric information of the target reference model, to obtain target real-time data, including: The initial mesh data is subjected to target region identification based on geometric information of the target reference model; wherein the target region includes the flat region and a feature region; The identified target region is subjected to corresponding mesh precision optimization, to obtain target real-time data; wherein the mesh precision optimization of the target region further includes up-sampling or point distance reduction of mesh vertices in the feature region.

7. The method of any one of claims 4-6, wherein, The non-real-time data processing includes mesh optimization processing corresponding to the real-time meshing, and the mesh optimization processing includes mesh precision optimization and mesh filtering denoising processing; The target object surface information is obtained by subjecting all the target real-time data to non-real-time data processing based on the target reference model, including: The initial optimization data is obtained by subjecting all the target real-time data to down-sampling of mesh vertices in a flat region and up-sampling or point distance reduction of mesh vertices in a feature region based on geometric information of the target reference model; The target object surface information is obtained by subjecting all the initial optimization data to mesh filtering denoising processing based on geometric information of the target reference model; wherein, in the mesh filtering denoising processing, neighborhood calculation is performed along the direction of the minimum principal curvature of the point to be processed. 8.A three-dimensional scanning processing apparatus based on prior data, comprising: an acquisition module configured to acquire an initial reference model of a scanned object and three-dimensional positioning data; wherein the initial reference model is prior data used to represent object surface information of the scanned object; an alignment module configured to align the initial reference model with the three-dimensional positioning data, to obtain a target reference model; a processing module configured to perform three-dimensional scanning processing on the scanned object based on the target reference model, to obtain target object surface information. 9.A computer device, comprising a memory and a processor; the memory stores a computer program capable of running on the processor; when the processor executes the computer program, the three-dimensional scanning processing method based on prior data according to any one of claims 1-7 is implemented.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is run by the processor to execute the three-dimensional scanning processing method based on prior data according to any one of claims 1-7.

Images