Three-dimensional scanning processing method and device based on prior data and computer equipment

Abstract

The invention provides a three-dimensional scanning processing method and device based on prior data and computer equipment, and relates to the technical field of three-dimensional scanning, and the method comprises the steps: obtaining an initial reference model and three-dimensional positioning data of a scanned object when the scanned object is subjected to three-dimensional scanning; wherein the initial reference model is prior data used for representing object surface information of a scanned object; aligning the initial reference model with the three-dimensional positioning data to obtain a target reference model; and performing three-dimensional scanning processing on the scanned object based on the target reference model to obtain surface information of the target object. When the scanned object is subjected to three-dimensional scanning processing, the prior data representing the object surface information of the scanned object is introduced, so that the scanning precision and the scanning efficiency can be improved by utilizing the prior data, the complexity of a scanning data processing algorithm is reduced, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional scanning technology, and in particular to a three-dimensional scanning processing method, device and computer equipment based on prior data. Background Art

[0002] At present, the surface information of an object is gradually obtained by performing three-dimensional scanning on the object. The specific processing flow of three-dimensional scanning includes:

[0003] 1. Obtain multiple local 3D point clouds on the surface of an object through multi-eye vision and structured light;

[0004] 2. Align multiple local 3D point clouds to obtain a 3D point cloud;

[0005] 3. Generate a 3D mesh through the 3D point cloud to obtain the surface information of the object.

[0006] However, the above three-dimensional scanning processing method has low scanning accuracy, high complexity of the scanning data processing algorithm, and low scanning efficiency, resulting in a poor user experience. Summary of the invention

[0007] The purpose of the present invention is to provide a three-dimensional scanning processing method, device and computer equipment based on prior data to improve scanning accuracy and scanning efficiency, reduce the complexity of the scanning data processing algorithm, and thus improve the user experience.

[0008] In a first aspect, an embodiment of the present invention provides a three-dimensional scanning processing method based on prior data, comprising:

[0009] Acquire an initial reference model and three-dimensional positioning data of the scanned object; wherein the initial reference model is prior data for representing the surface information of the scanned object;

[0010] Aligning the initial reference model with the three-dimensional positioning data to obtain a target reference model;

[0011] Based on the target reference model, the scanned object is subjected to three-dimensional scanning processing to obtain the surface information of the target object.

[0012] Furthermore, the three-dimensional positioning data is obtained by scanning a positioning object corresponding to the scanned object, a preset area of ​​the scanned object, or a contour of the scanned object; wherein the positioning object has a preset relative position relationship with the scanned object, and the positioning object has preset characteristics.

[0013] Furthermore, the initial reference model is aligned with the three-dimensional positioning data to obtain a target reference model, including:

[0014] Performing feature recognition on the three-dimensional positioning data to obtain target positioning features; wherein the target positioning features include preset features corresponding to the positioning object, area features corresponding to the preset area, or contour features of the scanned object;

[0015] Based on the target positioning features, the initial reference model is converted to the target coordinate system corresponding to the three-dimensional positioning data to obtain the target reference model.

[0016] Furthermore, based on the target reference model, the scanned object is subjected to three-dimensional scanning processing to obtain the surface information of the target object, including:

[0017] Acquire real-time image data obtained by three-dimensional scanning of the scanned object;

[0018] Based on the target reference model, real-time image data is processed in real time to obtain target real-time data; wherein the data processing includes three-dimensional reconstruction and point cloud registration, and also includes real-time gridding or point cloud fusion;

[0019] Based on the target reference model, non-real-time data processing is performed on all acquired real-time target data to obtain surface information of the target object; wherein the non-real-time data processing includes mesh optimization processing corresponding to real-time meshing, or non-real-time meshing and mesh optimization processing corresponding to point cloud fusion, and the mesh optimization processing includes mesh accuracy optimization and / or mesh filtering and denoising processing.

[0020] Furthermore, the real-time data processing includes three-dimensional reconstruction, point cloud registration and real-time meshing; based on the target reference model, the real-time image data is processed in real time to obtain the target real-time data, including:

[0021] Based on the object surface position information and the object surface normal information of the target reference model, the real-time image data is three-dimensionally reconstructed to obtain the first point cloud data;

[0022] Based on the object surface geometry information of the target reference model, the vertex weights of the first point cloud data are optimized to obtain the second point cloud data;

[0023] Performing point cloud registration on the second point cloud data to obtain third point cloud data;

[0024] Based on the shortest distance from the point to the surface of the object in the target reference model, performing clutter recognition and clutter removal on each point in the third point cloud data to obtain fourth point cloud data;

[0025] Performing real-time gridding processing on the fourth point cloud data to obtain initial grid data;

[0026] Based on the geometric information of the target reference model, the initial mesh data is optimized for mesh accuracy in the target area to obtain target real-time data; wherein, the mesh accuracy optimization in the target area includes downsampling of mesh vertices in the flat area.

[0027] Furthermore, based on the geometric information of the target reference model, the initial mesh data is optimized for mesh accuracy in the target area to obtain target real-time data, including:

[0028] Based on the geometric information of the target reference model, the target area is identified on the initial grid data; wherein the target area includes a flat area and a feature area;

[0029] The mesh accuracy of the identified target area is optimized accordingly to obtain the target real-time data; wherein the mesh accuracy optimization of the target area also includes upsampling of mesh vertices or reduction of point distance in the feature area.

[0030] Furthermore, the non-real-time data processing includes a mesh optimization process corresponding to the real-time meshing, and the mesh optimization process includes mesh accuracy optimization and mesh filtering and denoising process; based on the target reference model, non-real-time data processing is performed on all acquired target real-time data to obtain target object surface information, including:

[0031] Based on the geometric information of the target reference model, down-sample the mesh vertices in the flat area of ​​all target real-time data, and up-sample or reduce the point distance of the mesh vertices in the feature area to obtain initial optimized data;

[0032] Based on the geometric information of the target reference model, all the initial optimized data are subjected to grid filtering and denoising processing to obtain the surface information of the target object; wherein, in the process of grid filtering and denoising processing, neighborhood calculation is performed along the direction of the minimum principal curvature of the point to be processed.

[0033] In a second aspect, an embodiment of the present invention further provides a three-dimensional scanning processing device based on prior data, comprising:

[0034] An acquisition module, used to acquire an initial reference model and three-dimensional positioning data of the scanned object; wherein the initial reference model is a priori data used to represent the surface information of the scanned object;

[0035] An alignment module, used to align the initial reference model with the three-dimensional positioning data to obtain a target reference model;

[0036] The processing module is used to perform three-dimensional scanning processing on the scanned object based on the target reference model to obtain the surface information of the target object.

[0037] In a third aspect, an embodiment of the present invention further provides a computer device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the three-dimensional scanning processing method based on prior data of the first aspect is implemented.

[0038] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the three-dimensional scanning processing method based on prior data of the first aspect is executed.

[0039] The three-dimensional scanning processing method, device and computer equipment based on prior data provided by the embodiments of the present invention obtain the initial reference model and three-dimensional positioning data of the scanned object when performing three-dimensional scanning on the scanned object; wherein the initial reference model is the prior data used to represent the surface information of the scanned object; the initial reference model is aligned with the three-dimensional positioning data to obtain the target reference model; based on the target reference model, the scanned object is subjected to three-dimensional scanning processing to obtain the surface information of the target object. When performing three-dimensional scanning processing on the scanned object, prior data representing the surface information of the scanned object is introduced, so that the use of prior data can improve the scanning accuracy and scanning efficiency, reduce the complexity of the scanning data processing algorithm, and thus improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A schematic flow chart of a three-dimensional scanning processing method based on prior data provided by an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of coding points on a positioning object provided by an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of a minimum principal curvature direction provided by an embodiment of the present invention;

[0044] Figure 4 A schematic diagram of the structure of a three-dimensional scanning processing device based on prior data provided by an embodiment of the present invention;

[0045] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] The existing 3D scanning technology has the following problems:

[0048] 1. The scanning efficiency is not high enough and the computing resources are consumed a lot;

[0049] 2. The scanning accuracy is not high enough;

[0050] 3. Affected by noise, clutter is likely to appear in the 3D point cloud;

[0051] 4. Affected by point cloud noise, the quality of mesh generation is unstable, and spots and noise are prone to appear in detail features, model corners, thin walls and other areas;

[0052] 5. Due to the influence of noise, the accuracy of holes, edges and other positions on the scanned object is not high enough.

[0053] Based on this, the embodiments of the present invention provide a three-dimensional scanning processing method, device and computer equipment based on prior data, which can solve at least one of the above problems.

[0054] To facilitate understanding of this embodiment, a three-dimensional scanning processing method based on prior data disclosed in an embodiment of the present invention is first introduced in detail.

[0055] The embodiment of the present invention provides a three-dimensional scanning processing method based on prior data, which can be executed by a computer device with data processing capabilities. Figure 1 The flowchart of a three-dimensional scanning processing method based on prior data is shown, and the method mainly includes the following steps S110 to S130:

[0056] Step S110, acquiring an initial reference model and three-dimensional positioning data of the scanned object; wherein the initial reference model is a priori data for representing the surface information of the scanned object.

[0057] The initial reference model of the scanned object can be imported before scanning the scanned object or during scanning the scanned object. The scanned object can be an object with three-dimensional scanning requirements and prior data, and the prior data can be but not limited to CAD (Computer Aided Design) data, point cloud data or mesh data, etc. For example, in an industrial inspection scenario, it is necessary to detect whether the quality of a device processed by a machine such as a machine tool is qualified. The three-dimensional model of the device can be obtained by scanning the device, and the quality inspection can be performed based on the three-dimensional model. Based on this, the scanned object can be a device processed by a machine such as a machine tool, and its prior data can be the CAD data used when processing the device and other processing data required. For another example, in a three-dimensional printing quality inspection scenario, the three-dimensional model of the printed object can be obtained by scanning the printed object, and the three-dimensional printing quality inspection can be performed based on the three-dimensional model. Based on this, the scanned object can be an object obtained by three-dimensional printing, and its prior data can be the CAD data used in three-dimensional printing, etc. The scanned object can also be any object with three-dimensional scanning requirements, and its prior data can be point cloud data or mesh data measured by corresponding precision equipment. In addition, mesh data can also be obtained by converting CAD data.

[0058] In order to align the initial reference model with the scanned data of the scanned object, it is necessary to obtain the three-dimensional positioning data of the scanned object. Optionally, the three-dimensional positioning data can be obtained by scanning a positioning object corresponding to the scanned object, a preset area of ​​the scanned object, or a contour of the scanned object, etc.; wherein the positioning object has a preset relative position relationship with the scanned object, and the positioning object has preset features.

[0059] In a possible implementation, the three-dimensional positioning data may be positioning object image data obtained by scanning a positioning object corresponding to the scanned object; the three-dimensional scanner may include a single camera or at least two cameras, based on which, the positioning object image data may be at least two grayscale images obtained by a single camera taking pictures of the positioning object, or at least one grayscale image obtained by at least two cameras each taking pictures of the positioning object (that is, each camera takes at least one grayscale image of the positioning object). The positioning object may be an object with obvious features relative to the scanned object, that is, the preset features of the positioning object are obviously different from the features of the scanned object. The preset features on the positioning object can be set according to actual needs and are not limited here. The positioning object may be an inherent object on the mounting platform where the scanned object is located, or it may be an object artificially placed at a specific position of the scanned object; the positioning object may be an independent object or a combination of multiple sub-objects. For example, the positioning object is a device with a sticker such as Figure 2The cube of the coding point shown in the figure can be attached to one or more faces of the cube, and the coding point is the preset feature on the cube; it should be noted that the coding point can be set according to actual needs and is not limited to Figure 2 Shape shown.

[0060] In another possible implementation, the three-dimensional positioning data may be image data of the preset area of ​​the scanned object obtained by pre-scanning a specific area (i.e., a preset area) of the scanned object. The positioning object image data may be at least two grayscale images obtained by a single camera photographing the preset area of ​​the scanned object, or may be at least one grayscale image obtained by at least two cameras photographing the preset area of ​​the scanned object (i.e., each camera photographs at least one grayscale image of the preset area of ​​the scanned object). The preset area of ​​the scanned object may be an area with a specific pattern, and the specific pattern can uniquely locate the scanned object. In specific implementation, the user may be prompted to pre-scan a specific area of ​​the scanned object to obtain image data of the preset area of ​​the scanned object.

[0061] In another possible implementation, the three-dimensional positioning data may be contour data of the scanned object obtained by scanning the approximate contour of the scanned object in a rapid scanning manner, wherein the scanning speed of the rapid scanning manner is greater than the scanning speed of the normal scanning manner when the scanned object is subsequently three-dimensionally scanned, and the scanning accuracy of the rapid scanning manner is lower than the scanning accuracy of the normal scanning manner.

[0062] Step S120, aligning the initial reference model with the three-dimensional positioning data to obtain a target reference model.

[0063] The above alignment refers to converting the initial reference model to a specific target coordinate system, which may be a coordinate system corresponding to the three-dimensional positioning data. In the subsequent point cloud registration, the corresponding data may be converted to the three-dimensional positioning data to facilitate subsequent data processing. Considering that there will be position movement when a three-dimensional scanner scans an object, one scanning position corresponds to one coordinate system, so the three-dimensional positioning data may involve one coordinate system (such as the three-dimensional positioning data is the positioning object image data or the preset area image data of the scanned object taken by at least two cameras at one position), or may involve multiple coordinate systems (such as the three-dimensional positioning data is the positioning object image data, the preset area image data of the scanned object or the contour data of the scanned object taken by a single camera or at least two cameras at at least two positions). When multiple coordinate systems are involved, one of the coordinate systems can be selected as the target coordinate system (such as the coordinate system corresponding to the first scanning position, and of course other coordinate systems can also be selected), and the three-dimensional positioning data can be pre-aligned, and then the initial reference model is aligned with the pre-aligned three-dimensional positioning data to obtain the target reference model.

[0064] In some possible embodiments, the above step S120 may include: performing feature recognition on the three-dimensional positioning data to obtain target positioning features; wherein the target positioning features include preset features corresponding to the positioning object, area features corresponding to the preset area, or contour features of the scanned object; based on the target positioning features, converting the initial reference model to the target coordinate system corresponding to the three-dimensional positioning data to obtain a target reference model. The above three-dimensional positioning data may be pre-aligned three-dimensional positioning data.

[0065] Optionally, the coordinate conversion matrix between the coordinate system of the initial reference model and the target coordinate system corresponding to the three-dimensional positioning data can be determined based on the target positioning feature, and then the initial reference model can be converted to the target coordinate system based on the coordinate conversion matrix to obtain the target reference model. In specific implementation, if the three-dimensional positioning data is the above-mentioned positioning object image data, the position information of the preset features corresponding to the positioning object in the coordinate system of the initial reference model can be determined based on the relative position relationship between the positioning object and the scanned object, and then the coordinate conversion matrix between the coordinate system of the initial reference model and the target coordinate system corresponding to the positioning object image data can be obtained based on the position information of the preset features in the coordinate system of the initial reference model and the position information of the preset features in the positioning object image data, and the initial reference model can be converted to the target coordinate system based on the coordinate conversion matrix to obtain the target reference model. If the three-dimensional positioning data is preset area image data of the scanned object or contour data of the scanned object, the target positioning features in the three-dimensional positioning data (i.e., area features corresponding to the preset area or contour features of the scanned object) can be first spliced ​​(i.e., matched) with the initial reference model to obtain the position information of the target positioning features in the initial reference model, and then based on the position information of the target positioning features in the initial reference model and the position information of the target positioning features in the three-dimensional positioning data, a coordinate transformation matrix between the coordinate system of the initial reference model and the target coordinate system corresponding to the three-dimensional positioning data is obtained, and based on the coordinate transformation matrix, the initial reference model is transformed into the target coordinate system to obtain the target reference model.

[0066] Step S130, based on the target reference model, three-dimensional scanning processing is performed on the scanned object to obtain surface information of the target object.

[0067] After the target reference model is obtained through the alignment operation of step S120, the object to be scanned can be scanned three-dimensionally, and the information in the target reference model can be used to optimize the algorithms in the scanning process. In the case of importing the initial reference model during the scanning process, the scanned data before the initial reference model is imported can be processed first, or the scanned data can be discarded and the object to be scanned can be scanned three-dimensionally again.

[0068] In some possible embodiments, during the scanning process, the target reference model can be used to perform real-time data processing on the real-time image data obtained by three-dimensional scanning of the scanned object, and at the end of the scanning, the target reference model can be used to perform non-real-time data processing on all target real-time data obtained after the real-time data processing, so as to obtain the surface information of the target object. Based on this, step S130 can include the following steps S131 to S133:

[0069] Step S131, acquiring real-time image data obtained by three-dimensionally scanning the scanned object.

[0070] The real-time image data may include image data obtained by photographing a position of the scanned object. If the 3D scanner uses a single camera, the real-time image data may be at least two grayscale images obtained by the 3D scanner photographing the current position of the scanned object at least twice (the 3D scanner is at different positions at different photographing times); if the 3D scanner uses at least two cameras, the real-time image data may be at least two grayscale images obtained by the 3D scanner photographing the current position of the scanned object once.

[0071] Step S132, based on the target reference model, real-time data processing is performed on the real-time image data to obtain target real-time data; wherein the data processing includes three-dimensional reconstruction and point cloud registration, and also includes real-time gridding or point cloud fusion.

[0072] The above step S132 is a real-time processing process, which has a high requirement on the processing speed. Real-time gridding or point cloud fusion can be selected according to actual needs.

[0073] Step S133, based on the target reference model, non-real-time data processing is performed on all acquired real-time target data to obtain surface information of the target object; wherein the non-real-time data processing includes mesh optimization processing corresponding to real-time meshing, or non-real-time meshing and mesh optimization processing corresponding to point cloud fusion, and the mesh optimization processing includes mesh accuracy optimization and / or mesh filtering and denoising processing.

[0074] The above step S133 is not a real-time processing process, and has a low processing speed requirement. It is mainly used to optimize the surface features and edge corner areas of the model, and improve the overall effect of 3D scanning through optimization processing. The optimization processing result of all the target real-time data corresponding to the scanned object is the target object surface information of the scanned object.

[0075] In this embodiment, through the above-mentioned optimization process of introducing prior data, the scanning accuracy is improved, the complexity of all algorithms in the scanning data processing pipeline is reduced, the efficiency is optimized, and the user experience is improved.

[0076] The three-dimensional scanning processing method based on prior data provided by the embodiment of the present invention obtains the initial reference model and three-dimensional positioning data of the scanned object when performing three-dimensional scanning on the scanned object; wherein the initial reference model is the prior data used to represent the surface information of the scanned object; the initial reference model is aligned with the three-dimensional positioning data to obtain the target reference model; based on the target reference model, the scanned object is subjected to three-dimensional scanning processing to obtain the surface information of the target object. When performing three-dimensional scanning processing on the scanned object, prior data representing the surface information of the scanned object is introduced, so that the use of prior data can improve the scanning accuracy and scanning efficiency, reduce the complexity of the scanning data processing algorithm, and thus improve the user experience.

[0077] The embodiment of the present invention provides two three-dimensional scanning processing methods. The processing flow of method one mainly includes the steps of three-dimensional reconstruction, point cloud registration, real-time meshing and mesh optimization processing, and the processing flow of method two mainly includes the steps of three-dimensional reconstruction, point cloud registration, point cloud fusion, non-real-time meshing and mesh optimization processing. Under normal circumstances, considering that real-time meshing has high requirements on device performance, when the performance of the computer device executing the above-mentioned three-dimensional scanning processing method based on prior data is poor, the processing flow of method two corresponding to non-real-time meshing can be selected; conversely, when the performance of the computer device is better, the processing flow of method one corresponding to real-time meshing can be selected. It should be noted that the processing flows of these two methods are optional flows and can be selected according to user needs. Even if the performance of the computer device is better, the processing flow of method two can also be selected.

[0078] For ease of understanding, the above two methods are introduced in detail below.

[0079] Method 1:

[0080] Real-time image data can first be reconstructed in three dimensions to obtain initial point cloud data of a single frame, then point cloud registration can be performed to obtain point cloud data in the target coordinate system, and then real-time gridding processing can be performed to obtain grid data, and finally grid optimization processing can be performed to obtain surface information of the target object.

[0081] In the first embodiment, the above-mentioned step S132 can be implemented through the following process: based on the object surface position information and the object surface normal information of the target reference model, the real-time image data is three-dimensionally reconstructed to obtain the first point cloud data; based on the object surface geometric information of the target reference model, the vertex weight of the first point cloud data is optimized to obtain the second point cloud data; the second point cloud data is point cloud registered to obtain the third point cloud data; based on the closest distance from the point to the object surface in the target reference model, each point in the third point cloud data is identified and removed to obtain the fourth point cloud data; the fourth point cloud data is meshed in real time to obtain initial mesh data; based on the geometric information of the target reference model, the mesh accuracy of the target area of ​​the initial mesh data is optimized to obtain the target real-time data; wherein, the mesh accuracy optimization of the target area includes downsampling of mesh vertices in the flat area.

[0082] In the first embodiment, the above step S133 can be implemented through the following process: based on the geometric information of the target reference model, down-sampling of mesh vertices in the flat area of ​​all target real-time data, and up-sampling of mesh vertices in the feature area or point distance reduction are performed to obtain initial optimized data; based on the geometric information of the target reference model, grid filtering and denoising processing is performed on all initial optimized data to obtain surface information of the target object; wherein, during the grid filtering and denoising processing, neighborhood calculation is performed along the direction of the minimum principal curvature of the point to be processed.

[0083] The processing flow of the above method 1 is described in detail below.

[0084] The target reference model can be used to improve the accuracy and efficiency of the laser line reconstruction three-dimensional point algorithm. Specifically, the three-dimensional reconstruction of real-time image data can be performed using the object surface position information and object surface normal information of the target reference model, which can reduce mismatching in the three-dimensional reconstruction process, improve matching efficiency, and optimize the scanning frame rate. The above-mentioned vertex weight optimization process is an online matching process for three-dimensional reconstruction. When optimizing the vertex weights of the first point cloud data using the object surface geometry information of the target reference model, the problem of affecting vertex accuracy can be solved by reducing the vertex weights in areas such as convex corners (to reduce the impact of energy overflow here), concave corners (to reduce the impact of mirror reflection here), and edges (to reduce the impact of line missing here) according to the physical characteristics of the laser line (the laser line scan camera in the three-dimensional scanner scans by emitting laser lines) to optimize the accuracy of holes, edges and other positions on the scanned object, thereby improving the overall accuracy. Among them, the vertex weight is used to characterize the quality and accuracy of the corresponding point. For example:

[0085] When calculating the normal vector of a laser point (the input data required for gridding processing), traditional methods generally need to find surrounding neighboring points, determine the direction of the laser line based on the neighboring points, and use the directions of multiple laser lines to calculate the normal vector. This embodiment can directly find the corresponding surface in the target reference model based on the laser point coordinates, and use the known normal vector of the surface in the target reference model.

[0086] By using the closest distance from a point to the surface of an object in the target reference model, it is possible to remove noise points caused by various errors and optimize the efficiency of noise point removal. Specifically, when performing noise point identification and noise point removal, each point in the third point cloud data can be first converted to the target reference model, and the closest distance from each point to the surface of the object in the target reference model is calculated to determine whether the closest distance reaches a preset distance threshold. If so, the point is determined to be a noise point and removed; if not, the point is determined not to be a noise point to retain the point. The distance threshold can be set according to actual needs and is not limited here.

[0087] When optimizing the mesh accuracy of the target area, the number of mesh vertices in the flat area can be reduced by downsampling the mesh vertices in the flat area, thereby improving the mesh generation efficiency; when the computer device executing the method meets the performance requirements, the accuracy of the feature area can be improved by using a smaller point pitch for the feature area (to improve the fineness of the mesh), or the number of vertices in the feature area can be appropriately increased, thereby optimizing the three-dimensional scanning effect of the feature area, thereby achieving the optimization of mesh generation efficiency and quality. Based on this, the above-mentioned target real-time data can be realized by the following process: based on the geometric information of the target reference model, the initial mesh data is identified as the target area; wherein the target area includes a flat area and a feature area; the identified target area is optimized for the corresponding mesh accuracy to obtain the target real-time data; wherein the mesh accuracy optimization of the target area also includes upsampling or reducing the point pitch of the mesh vertices in the feature area. Among them, the flat area refers to the area with relatively consistent visual characteristics in the image, which may lack obvious color, texture or shape changes and look relatively uniform. The flat area does not contain too many visual details or information, and has relatively little effect on image analysis and understanding, so the number of mesh vertices can be appropriately reduced. Feature regions refer to regions in an image that have specific visual attributes. These regions may be distinctive due to their color, texture, shape, or other visual characteristics. Feature regions may include, but are not limited to, edges, textures, and color changes. By improving the accuracy of feature regions, the 3D scanning effect can be improved.

[0088] Optionally, when downsampling the mesh vertices in the flat area of ​​all target real-time data, the downsampling multiple can be greater than the downsampling multiple when downsampling the mesh vertices in the flat area of ​​the initial mesh data, that is, the number of mesh vertices reduced in the flat area during the real-time processing (corresponding to step S132) is greater than the number of mesh vertices reduced in the flat area during the non-real-time processing (corresponding to step S133), for example, the number of vertices in the flat area can be reduced by 30% during the real-time processing, and the number of vertices in the flat area can be reduced by 80% during the non-real-time processing. In this way, the real-time requirements of the real-time processing process can be guaranteed, while the processing efficiency of the real-time processing process can be improved, and the mesh generation effect can be further improved, that is, the optimization of the mesh generation effect and efficiency is achieved.

[0089] During the grid filtering denoising process, the neighborhood calculation can be along the direction of the minimum principal curvature of the feature, that is, only the direction of the smallest surface change is considered, which can enhance the edge angle effect. Among them, the direction of the minimum principal curvature means: there are infinite orthogonal curvatures at a certain point on the surface, among which there is a curve that makes the curvature of the curve maximum, and the curvature perpendicular to the maximum curvature surface is the minimum value Kmin. The direction of the minimum value Kmin curvature is the direction of the minimum principal curvature. For example, Figure 3 As shown in the figure, the curvature of point P on the broken line along the arrow direction is 0, and the arrow direction is the minimum principal curvature direction of point P. When calculating the neighborhood of point P, only the points in the minimum principal curvature direction are taken for calculation, that is, when calculating the neighborhood, only the neighboring points of point P along the broken line direction shown by the arrow are considered, so that the best corner feature retention effect can be achieved. In this way, through the mesh filtering and denoising processing based on the geometric information of the target reference model, the influence of point cloud noise can be reduced, the stability of mesh generation quality can be improved, and the clutter and noise that are prone to appear in detail features, model corners, thin walls and other areas can be avoided to a certain extent.

[0090] It should be noted that the geometric information of the target reference model provides the basic framework for the target reference model, and the surface geometric information of the target reference model is a further refinement and improvement of this basic framework. The two together determine the visual performance and physical properties of the target reference model. The surface geometric information of the target reference model can include information such as the normal direction of each face, the roughness of the surface, and the texture coordinates; the geometric information of the target reference model can include information such as the vertex coordinates of all faces, the length of the edges, and the shape of the faces.

[0091] Method 2:

[0092] Real-time image data can first be reconstructed in three dimensions to obtain initial point cloud data of a single frame, and then point cloud registration can be used to obtain point cloud data in the target coordinate system. Point cloud fusion is then performed to merge point cloud data from multiple perspectives, and then non-real-time gridding is performed to obtain grid data. Finally, grid optimization is performed to obtain surface information of the target object.

[0093] Under method two, the above-mentioned step S132 can be implemented through the following process: based on the object surface position information and the object surface normal information of the target reference model, the real-time image data is three-dimensionally reconstructed to obtain the first point cloud data; based on the object surface geometric information of the target reference model, the first point cloud data is optimized for vertex weights to obtain the second point cloud data; the second point cloud data is point cloud registered to obtain the third point cloud data; based on the closest distance from the point to the object surface in the target reference model, each point in the third point cloud data is identified and removed for noise to obtain the fourth point cloud data; based on one or more of the object surface position information, the object surface normal information and the object surface geometric information in the target reference model, the fourth point cloud data is point cloud fused to obtain the target real-time data.

[0094] Under the second method, the above step S133 can be implemented through the following process: non-real-time meshing processing is performed on all target real-time data to obtain initial mesh data; based on the geometric information of the target reference model, the mesh vertices in the flat area of ​​the initial mesh data are down-sampled, and the mesh vertices in the feature area are up-sampled or the point distance is reduced to obtain initial optimized data; based on the geometric information of the target reference model, the initial optimized data is mesh filtered and denoised to obtain the surface information of the target object; wherein, during the mesh filtering and denoising process, neighborhood calculation is performed along the direction of the minimum principal curvature of the point to be processed.

[0095] For the parts not described in detail in the above method 2, please refer to the corresponding contents in method 1, and will not be repeated here.

[0096] For ease of understanding, the implementation process of the above three-dimensional scanning processing method based on prior data is introduced below by taking method 1 as an example.

[0097] 1. Before or during scanning, import the initial reference model (CAD data, point cloud data or mesh data) of the scanned object.

[0098] 2. Pre-align the initial reference model with the scan data to obtain the target reference model.

[0099] Alignment methods include but are not limited to one of the following:

[0100] 2.1. Use the positioning objects on the mounting table of the scanned object for alignment;

[0101] 2.2. Use artificially added aids for alignment;

[0102] 2.3. Prompt the user to pre-scan specific areas of the scanned object for stitching;

[0103] 2.4. The user uses a faster scanning method to quickly scan the approximate outline of the scanned object and uses this outline for splicing.

[0104] 3. After alignment, start scanning and use the information in the target reference model to optimize the algorithms in the scanning process, including:

[0105] 3.1. Improvement of accuracy and efficiency of laser line reconstruction 3D point algorithm

[0106] Based on the object surface position information and object surface normal information of the target reference model, mismatching is reduced and matching efficiency is improved.

[0107] Based on the surface geometry information of the target reference model, during the online matching process, in order to address issues affecting vertex accuracy, the vertex weights in areas such as convex corners (energy overflow), concave corners (specular reflection), and edges (line missing) are reduced according to the physical properties of the laser line to improve overall accuracy.

[0108] 3.2. Scanning noise removal

[0109] Based on the shortest distance from the generated point to the surface of the object in the target reference model, the noise points caused by various errors are removed.

[0110] 3.3. Optimization of mesh generation effect and efficiency

[0111] Based on the geometric information of the target reference model, the number of mesh vertices in flat areas is reduced to improve mesh generation efficiency.

[0112] 3.4. Optimization of model surface features and edge corner areas

[0113] Based on the geometric information of the target reference model, use a smaller point distance for the feature area, or increase the number of vertices in the feature area appropriately to optimize the effect of the feature area.

[0114] During the grid filtering denoising process, the neighborhood calculation is along the direction of the minimum principal curvature of the feature to enhance the edge angle effect.

[0115] Compared with a three-dimensional scanning solution without prior data, the embodiments of the present invention can improve scanning efficiency, improve scanning accuracy, reduce scanning noise points, and improve data quality at the details of the three-dimensional model of the scanned object.

[0116] Corresponding to the above-mentioned three-dimensional scanning processing method based on prior data, an embodiment of the present invention further provides a three-dimensional scanning processing device based on prior data. Figure 4 A schematic structural diagram of a three-dimensional scanning processing device based on prior data is shown, the device comprising:

[0117] The acquisition module 401 is used to acquire an initial reference model and three-dimensional positioning data of the scanned object; wherein the initial reference model is a priori data used to represent the surface information of the scanned object;

[0118] An alignment module 402 is used to align the initial reference model with the three-dimensional positioning data to obtain a target reference model;

[0119] The processing module 403 is used to perform three-dimensional scanning processing on the scanned object based on the target reference model to obtain surface information of the target object.

[0120] The three-dimensional scanning and processing device based on prior data provided by the embodiment of the present invention, when performing three-dimensional scanning on the scanned object, obtains the initial reference model and three-dimensional positioning data of the scanned object through the acquisition module 401; wherein the initial reference model is the prior data used to represent the surface information of the scanned object; the initial reference model is aligned with the three-dimensional positioning data through the alignment module 402 to obtain the target reference model; and the scanned object is subjected to three-dimensional scanning processing based on the target reference model through the processing module 403 to obtain the surface information of the target object. When performing three-dimensional scanning processing on the scanned object, the prior data representing the surface information of the scanned object is introduced, so that the use of the prior data can improve the scanning accuracy and scanning efficiency, reduce the complexity of the scanning data processing algorithm, and thus improve the user experience.

[0121] Furthermore, the above-mentioned three-dimensional positioning data is obtained by scanning a positioning object corresponding to the scanned object, a preset area of ​​the scanned object, or a contour of the scanned object; wherein the positioning object and the scanned object have a preset relative position relationship, and the positioning object has preset characteristics.

[0122] Furthermore, the above-mentioned alignment module 402 is specifically used to: perform feature recognition on the three-dimensional positioning data to obtain target positioning features; wherein the target positioning features include preset features corresponding to the positioning object, area features corresponding to the preset area, or contour features of the scanned object; based on the target positioning features, the initial reference model is converted to the target coordinate system corresponding to the three-dimensional positioning data to obtain the target reference model.

[0123] Furthermore, the above-mentioned processing module 403 is specifically used to: obtain real-time image data obtained by performing three-dimensional scanning on the scanned object; based on the target reference model, perform real-time data processing on the real-time image data to obtain target real-time data; wherein the data processing includes three-dimensional reconstruction and point cloud alignment, and also includes real-time gridding or point cloud fusion; based on the target reference model, perform non-real-time data processing on all acquired target real-time data to obtain target object surface information; wherein the non-real-time data processing includes grid optimization processing corresponding to real-time gridding, or non-real-time gridding and grid optimization processing corresponding to point cloud fusion, and the grid optimization processing includes grid accuracy optimization and / or grid filtering and denoising processing.

[0124] Furthermore, real-time data processing includes three-dimensional reconstruction, point cloud registration and real-time meshing; the above-mentioned processing module 403 is also used to: perform three-dimensional reconstruction of real-time image data based on the object surface position information and object surface normal information of the target reference model to obtain first point cloud data; perform vertex weight optimization on the first point cloud data based on the object surface geometric information of the target reference model to obtain second point cloud data; perform point cloud registration on the second point cloud data to obtain third point cloud data; perform clutter identification and clutter removal on each point in the third point cloud data based on the closest distance from the point to the object surface in the target reference model to obtain fourth point cloud data; perform real-time meshing on the fourth point cloud data to obtain initial mesh data; perform mesh accuracy optimization on the target area of ​​the initial mesh data based on the geometric information of the target reference model to obtain target real-time data; wherein, the mesh accuracy optimization of the target area includes downsampling of mesh vertices in the flat area.

[0125] Furthermore, the above-mentioned processing module 403 is also used to: identify the target area of ​​the initial mesh data based on the geometric information of the target reference model; wherein the target area includes a flat area and a feature area; optimize the corresponding mesh accuracy of the identified target area to obtain real-time target data; wherein the mesh accuracy optimization of the target area also includes upsampling of mesh vertices in the feature area or reduction of point distance.

[0126] Furthermore, the non-real-time data processing includes a mesh optimization process corresponding to the real-time meshing, and the mesh optimization process includes a mesh accuracy optimization and a mesh filtering and denoising process; the above-mentioned processing module 403 is also used for: based on the geometric information of the target reference model, down-sampling the mesh vertices in the flat area of ​​all target real-time data, and up-sampling or reducing the point distance of the mesh vertices in the feature area to obtain initial optimized data; based on the geometric information of the target reference model, mesh filtering and denoising are performed on all initial optimized data to obtain the surface information of the target object; wherein, in the process of mesh filtering and denoising, neighborhood calculation is performed along the direction of the minimum principal curvature of the point to be processed.

[0127] The three-dimensional scanning processing device based on prior data provided in this embodiment has the same implementation principle and technical effects as the aforementioned three-dimensional scanning processing method embodiment based on prior data. For the sake of brief description, for parts not mentioned in the embodiment of the three-dimensional scanning processing device based on prior data, reference may be made to the corresponding contents in the aforementioned three-dimensional scanning processing method embodiment based on prior data.

[0128] like Figure 5 As shown, a computer device 500 provided in an embodiment of the present invention includes: a processor 501, a memory 502 and a bus, the memory 502 stores a computer program that can be run on the processor 501, when the computer device 500 is running, the processor 501 and the memory 502 communicate through the bus, and the processor 501 executes the computer program to implement the above-mentioned three-dimensional scanning processing method based on prior data.

[0129] Specifically, the memory 502 and the processor 501 can be general-purpose memories and processors, which are not specifically limited here.

[0130] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the three-dimensional scanning processing method based on prior data in the above method embodiment is executed. The computer-readable storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk or an optical disk, and other media that can store program codes.

[0131] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0132] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.

[0133] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and a part of a module, a program segment or a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0134] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0135] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A three-dimensional scanning processing method based on prior data, characterized in that: include: Acquire an initial reference model and three-dimensional positioning data of the scanned object; wherein the initial reference model is a priori data for representing the surface information of the scanned object; Aligning the initial reference model with the three-dimensional positioning data to obtain a target reference model; Based on the target reference model, the scanned object is subjected to three-dimensional scanning processing to obtain surface information of the target object.

2. The method according to claim 1, characterized in that: The three-dimensional positioning data is obtained by scanning a positioning object corresponding to the scanned object, a preset area of ​​the scanned object, or a contour of the scanned object; wherein the positioning object has a preset relative position relationship with the scanned object, and the positioning object has preset characteristics.

3. The method according to claim 2, characterized in that The step of aligning the initial reference model with the three-dimensional positioning data to obtain a target reference model includes: Performing feature recognition on the three-dimensional positioning data to obtain target positioning features; wherein the target positioning features include preset features corresponding to the positioning object, area features corresponding to the preset area, or contour features of the scanned object; Based on the target positioning feature, the initial reference model is converted to a target coordinate system corresponding to the three-dimensional positioning data to obtain a target reference model.

4. The method according to claim 1, characterized in that: The step of performing three-dimensional scanning processing on the scanned object based on the target reference model to obtain surface information of the target object includes: Acquiring real-time image data obtained by three-dimensionally scanning the scanned object; Based on the target reference model, real-time data processing is performed on the real-time image data to obtain target real-time data; wherein the data processing includes three-dimensional reconstruction and point cloud registration, and also includes real-time gridding or point cloud fusion; Based on the target reference model, non-real-time data processing is performed on all the acquired real-time data of the target to obtain surface information of the target object; wherein the non-real-time data processing includes mesh optimization processing corresponding to the real-time meshing, or non-real-time meshing and mesh optimization processing corresponding to the point cloud fusion, and the mesh optimization processing includes mesh accuracy optimization and / or mesh filtering and denoising processing.

5. The method according to claim 4, characterized in that The real-time data processing includes three-dimensional reconstruction, point cloud registration and real-time gridding; the real-time data processing of the real-time image data based on the target reference model to obtain target real-time data includes: Based on the object surface position information and the object surface normal information of the target reference model, three-dimensionally reconstruct the real-time image data to obtain first point cloud data; Based on the object surface geometric information of the target reference model, performing vertex weight optimization on the first point cloud data to obtain second point cloud data; Performing point cloud registration on the second point cloud data to obtain third point cloud data; Based on the shortest distance from the point to the surface of the object in the target reference model, performing clutter identification and clutter removal on each point in the third point cloud data to obtain fourth point cloud data; Performing real-time gridding processing on the fourth point cloud data to obtain initial grid data; Based on the geometric information of the target reference model, the initial mesh data is optimized for mesh accuracy in the target area to obtain target real-time data; wherein the mesh accuracy optimization in the target area includes downsampling of mesh vertices in a flat area.

6. The method according to claim 5, characterized in that The step of optimizing the mesh accuracy of the target area of ​​the initial mesh data based on the geometric information of the target reference model to obtain the target real-time data includes: Based on the geometric information of the target reference model, target area recognition is performed on the initial grid data; wherein the target area includes the flat area and the feature area; The identified target area is subjected to corresponding mesh precision optimization to obtain target real-time data; wherein the mesh precision optimization of the target area further includes upsampling of mesh vertices or reduction of point distance in the feature area.

7. The method according to claim 4, characterized in that The non-real-time data processing includes a grid optimization process corresponding to the real-time gridding, and the grid optimization process includes grid accuracy optimization and grid filtering and denoising process; The non-real-time data processing is performed on all the acquired real-time data of the target based on the target reference model to obtain the surface information of the target object, including: Based on the geometric information of the target reference model, down-sampling of mesh vertices in a flat area and up-sampling or point distance reduction of mesh vertices in a feature area are performed on all the target real-time data to obtain initial optimized data; Based on the geometric information of the target reference model, all the initial optimization data are subjected to grid filtering and denoising processing to obtain the surface information of the target object; wherein, during the grid filtering and denoising processing, neighborhood calculation is performed along the direction of the minimum principal curvature of the point to be processed.

8. A three-dimensional scanning processing device based on prior data, characterized in that: include: An acquisition module, used to acquire an initial reference model and three-dimensional positioning data of the scanned object; wherein the initial reference model is a priori data for representing the surface information of the scanned object; An alignment module, used for aligning the initial reference model with the three-dimensional positioning data to obtain a target reference model; The processing module is used to perform three-dimensional scanning processing on the scanned object based on the target reference model to obtain surface information of the target object.

9. A computer device, characterized in that: It comprises a memory and a processor; the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, it implements the three-dimensional scanning processing method based on prior data as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the three-dimensional scanning processing method based on prior data described in any one of claims 1 to 7 is executed.

Images