Three-dimensional scanning and reconstruction method and apparatus, device, and storage medium

Abstract

Embodiments of the present disclosure relate to the technical field of three-dimensional scanners, and provide a three-dimensional scanning and reconstruction method and apparatus, a device, and a storage medium. The method comprises: acquiring a current-frame image of the surface of an object to be measured; performing real-time processing on the current-frame image to obtain current-frame three-dimensional point data, wherein the current-frame three-dimensional point data is configured to represent discrete point data of the current surface shape contour of said object; and performing real-time mesh reconstruction on the basis of the current-frame three-dimensional point data to obtain current three-dimensional mesh updating data, wherein the current three-dimensional mesh updating data is configured to represent continuous surface data of the surface shape contour of said object. By using the present technical solution, the accuracy of the scanning result of an object to be measured scanned by a handheld three-dimensional scanner can be higher, and the final scanning result can be reflected more directly.

Description

Three-dimensional scanning reconstruction method, device, equipment and storage medium

[0001] This disclosure claims priority to the Chinese patent application filed with the China Patent Office on December 29, 2023, with application number 202311849131.2, and entitled “Three-dimensional scanning reconstruction method, device, equipment and storage medium”, the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0002] The embodiments of the present disclosure relate to the technical field of three-dimensional scanners, and in particular to a three-dimensional scanning and reconstruction method, apparatus, device, and storage medium. Background Art

[0003] As a non-contact 3D measurement tool, handheld 3D scanners have been widely used in industries such as industry, healthcare, and education. When scanning with a handheld 3D scanner, it is often necessary to display the scanned surface of the object to be measured in real time to help users determine which areas have been scanned and which have not.

[0004] Currently, common handheld 3D scanners on the market generally select 3D points on the surface of an object for display. However, using only 3D points to represent the object to be measured has its limitations.

[0005] Therefore, there is an urgent need for a three-dimensional scanning reconstruction method that can make the scanning results of the object to be measured scanned by a handheld three-dimensional scanner more accurate and can more directly reflect the final scanning results. Summary of the Invention

[0006] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide a three-dimensional scanning reconstruction method, device, equipment and storage medium.

[0007] A first aspect of an embodiment of the present disclosure provides a three-dimensional scanning and reconstruction method, the method comprising:

[0008] Obtain the current frame image of the surface of the object to be measured;

[0009] Processing the current frame image in real time to obtain three-dimensional point data of the current frame, wherein the three-dimensional point data of the current frame is configured as discrete point data representing the current surface shape contour of the object to be measured;

[0010] Real-time mesh reconstruction is performed based on the current frame three-dimensional point data to obtain current three-dimensional mesh update data; wherein the current three-dimensional mesh update data is configured as continuous surface data representing the surface shape contour of the object to be measured.

[0011] In one example, when the current frame image is not the first frame image, after obtaining the current frame three-dimensional point data, the method further includes:

[0012] The current frame 3D point data is spliced ​​and fused with the current 3D point data in real time to obtain current 3D point update data; the current 3D point data is the 3D point data obtained by splicing and fusion of one or more frames of 3D point data obtained before the current frame 3D point data.

[0013] In one example, when the current frame image is not the first frame image, performing real-time mesh reconstruction based on the current frame three-dimensional point data to obtain current three-dimensional mesh update data includes:

[0014] Determining, in the current three-dimensional point update data, a portion of data corresponding to the three-dimensional point data of the current frame;

[0015] Based on the current three-dimensional grid data, the partial data is reconstructed in real time to obtain current three-dimensional grid update data; wherein the current three-dimensional grid data is three-dimensional grid data generated from the current three-dimensional point data.

[0016] In one example, after obtaining the current three-dimensional grid update data, the method further includes:

[0017] The current three-dimensional grid update data is rendered to generate a three-dimensional grid display image.

[0018] In one example, the real-time mesh reconstruction of the partial data to obtain current three-dimensional mesh update data includes:

[0019] Based on the partial data, topological relationships between neighborhoods of the three-dimensional points are constructed in parallel to obtain current three-dimensional grid update data.

[0020] In one example, the step of constructing the topological relationship between the neighborhoods of the three-dimensional points based on the partial data in parallel to obtain the current three-dimensional grid update data includes:

[0021] Dividing the portion of data into a plurality of spatial blocks;

[0022] Constructing a topological relationship between the neighborhood of each three-dimensional point in each of the spatial blocks to obtain a plurality of sub-grids;

[0023] The current three-dimensional grid update data is obtained based on the multiple sub-grids constructed by each of the spatial blocks.

[0024] In one example, constructing a topological relationship between the neighborhood of each three-dimensional point in each of the spatial blocks to obtain multiple sub-grids includes:

[0025] For each of the spatial blocks, determining an initial edge as a current topological edge based on the three-dimensional points in the spatial block;

[0026] Traversing each 3D point outside the current topological edge in the spatial block, determining a triangle formed by each 3D point and the current topological edge, and determining the triangle that meets a preset condition as a facet of the sub-mesh;

[0027] Two edges other than the initial edge in the patch are updated as current topological edges.

[0028] In one example, determining an initial edge as a current topological edge based on a three-dimensional point in the spatial block includes:

[0029] Determine any three-dimensional point that does not belong to any facet within the spatial block as an initial point;

[0030] Traversing each 3D point other than the initial point in the spatial block, calculating the distance between each 3D point and the initial point, and determining the 3D point with the smallest distance as the first connected 3D point;

[0031] An edge formed by the initial point and the first connected three-dimensional point is determined as the initial edge.

[0032] In one example, obtaining the current three-dimensional grid update data based on the multiple sub-grids constructed based on each of the spatial blocks includes:

[0033] Searching for an edge in any of the spatial blocks that is not shared by multiple facets as a current topological edge;

[0034] Each three-dimensional point in the neighborhood spatial block of the spatial block is traversed to determine the triangle formed by each three-dimensional point and the current topological edge, and the triangle that meets the preset conditions is determined as the facet of the current three-dimensional grid.

[0035] A second aspect of the embodiments of the present disclosure provides a three-dimensional scanning and reconstruction device, the device comprising:

[0036] An acquisition module is configured to acquire a current frame image of the surface of the object to be measured;

[0037] a processing module configured to perform real-time processing on the current frame image to obtain three-dimensional point data of the current frame, wherein the three-dimensional point data of the current frame is configured as discrete point data representing the current surface shape contour of the object to be measured;

[0038] The reconstruction module is configured to perform real-time mesh reconstruction based on the current frame three-dimensional point data to obtain current three-dimensional mesh update data; wherein the current three-dimensional mesh update data is configured as continuous surface data representing the surface shape contour of the object to be measured.

[0039] In one example, when the current frame image is not the first frame image, after obtaining the current frame three-dimensional point data, the apparatus further includes:

[0040] The fusion module is configured to splice and fuse the current frame 3D point data with the current 3D point data in real time to obtain current 3D point update data; the current 3D point data is the 3D point data obtained by splicing and fusing one or more frames of 3D point data obtained before the current frame 3D point data.

[0041] In one example, when the current frame image is not the first frame image, the reconstruction module includes:

[0042] a determining unit configured to determine, in the current 3D point update data, a portion of data corresponding to the current frame 3D point data;

[0043] The reconstruction unit is configured to perform real-time mesh reconstruction on the partial data based on the current three-dimensional mesh data to obtain current three-dimensional mesh update data; wherein the current three-dimensional mesh data is three-dimensional mesh data generated from the current three-dimensional point data.

[0044] In one example, after obtaining the current three-dimensional grid update data, the apparatus further includes:

[0045] The rendering module is configured to render the current three-dimensional grid update data and generate a three-dimensional grid display image.

[0046] In one example, the reconstruction module includes:

[0047] The construction unit is configured to construct, based on the partial data, a topological relationship between the neighborhoods of each three-dimensional point in parallel to obtain current three-dimensional grid update data.

[0048] In one example, the building block includes:

[0049] a partitioning subunit, configured to partition the portion of data into a plurality of spatial blocks;

[0050] A construction sub-unit is configured to construct a topological relationship between the neighborhood of each three-dimensional point in each of the spatial blocks to obtain a plurality of sub-grids;

[0051] The determination subunit is configured to obtain the current three-dimensional grid update data based on the multiple sub-grids constructed by each of the spatial blocks.

[0052] In one example, the subunit is constructed and specifically configured as follows:

[0053] For each of the spatial blocks, determining an initial edge as a current topological edge based on the three-dimensional points in the spatial block;

[0054] Traversing each 3D point outside the current topological edge in the spatial block, determining a triangle formed by each 3D point and the current topological edge, and determining the triangle that meets a preset condition as a facet of the sub-mesh;

[0055] Two edges other than the initial edge in the patch are updated as current topological edges.

[0056] In one example, the subunit is constructed and specifically configured as follows:

[0057] Determine any three-dimensional point that does not belong to any facet within the spatial block as an initial point;

[0058] Traversing each 3D point other than the initial point in the spatial block, calculating the distance between each 3D point and the initial point, and determining the 3D point with the smallest distance as the first connected 3D point;

[0059] An edge formed by the initial point and the first connected three-dimensional point is determined as the initial edge.

[0060] In one example, the determining subunit is specifically configured to:

[0061] Searching for an edge in any of the spatial blocks that is not shared by multiple facets as a current topological edge;

[0062] Each three-dimensional point in the neighborhood spatial block of the spatial block is traversed to determine the triangle formed by each three-dimensional point and the current topological edge, and the triangle that meets the preset conditions is determined as the facet of the current three-dimensional grid.

[0063] A third aspect of an embodiment of the present disclosure provides an electronic device, which includes: a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the method of the first aspect above.

[0064] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method of the first aspect described above can be implemented.

[0065] The disclosed embodiments provide a three-dimensional scanning and reconstruction method, apparatus, device, and storage medium. The method obtains a current frame image of the surface of the object to be measured, processes the current frame image in real time, and obtains current frame three-dimensional point data, wherein the current frame three-dimensional point data is configured as discrete point data representing the current surface shape contour of the object to be measured, and performs real-time mesh reconstruction based on the current frame three-dimensional point data to obtain current three-dimensional mesh update data; wherein the current three-dimensional mesh update data is configured as continuous surface data representing the surface shape contour of the object to be measured. The use of this technical solution can make the scanning results of the object to be measured scanned by a handheld three-dimensional scanner more accurate and can more directly reflect the final scanning results. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0067] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0068] FIG1 is a schematic flow chart of a three-dimensional scanning and reconstruction method provided in a first embodiment of the present disclosure;

[0069] FIG2 is a flow chart of another three-dimensional scanning and reconstruction method provided in a second embodiment of the present disclosure;

[0070] FIG3 is a schematic structural diagram of a space block provided in a second embodiment of the present disclosure;

[0071] FIG4 is a schematic diagram of a three-dimensional grid display image provided by the second embodiment of the present disclosure;

[0072] FIG5 is a schematic structural diagram of a three-dimensional scanning and reconstruction device provided in a third embodiment of the present disclosure;

[0073] FIG6 is a schematic structural diagram of a three-dimensional scanning and reconstruction device provided in a fourth embodiment of the present disclosure;

[0074] FIG7 is a schematic structural diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0075] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0076] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0077] FIG1 is a flow chart of a three-dimensional scanning and reconstruction method provided in a first embodiment of the present disclosure. The method can be performed by an electronic device. The electronic device can be exemplarily understood as a device such as a mobile phone, tablet computer, laptop computer, desktop computer, smart TV, etc. As shown in FIG1 , the method provided in this embodiment includes the following steps:

[0078] S101: Acquire a current frame image of the surface of an object to be measured.

[0079] In this embodiment, the object to be measured can be of various shapes. The current frame image includes a first frame image and non-first frame images. Specifically, a 3D scanner is used to scan the surface of the object to be measured, thereby obtaining a current frame image of the surface of the object to be measured. The 3D scanner then transmits the current frame image of the surface of the object to be measured to a graphics card. The graphics card is the execution entity in this embodiment.

[0080] S102 : Process the current frame image in real time to obtain three-dimensional point data of the current frame, wherein the three-dimensional point data of the current frame is configured as discrete point data representing the current surface shape contour of the object to be measured.

[0081] In this embodiment, the current frame 3D point data is configured as discrete point data representing the current surface shape contour of the object to be measured. If the current frame image is the first frame image, the current frame 3D point data is the first frame 3D point data. If the current frame image is not the first frame image, the current frame 3D point data is the non-first frame 3D point data. The current frame image is processed in real time by the graphics card, and then the discrete point data of the current surface shape contour of the object to be measured can be obtained. The above-mentioned discrete point data is collected as discontinuous point data.

[0082] S103 , performing real-time mesh reconstruction based on the current frame three-dimensional point data to obtain current three-dimensional mesh update data; wherein the current three-dimensional mesh update data is configured as continuous surface data representing the surface shape contour of the object to be measured.

[0083] In one example, the current three-dimensional network update data represents the surface data obtained by constructing the discrete point data of the current surface shape contour of the object to be measured, that is, the current three-dimensional network update data represents the continuous surface data of the surface shape contour of the object to be measured.

[0084] In this embodiment, each time the current frame 3D point data is obtained, the current frame 3D point data is processed in real time. The specific processing method is mesh reconstruction, thereby obtaining the current 3D mesh update data.

[0085] The disclosed embodiments provide a three-dimensional scanning and reconstruction method. The method obtains a current frame image of the surface of an object to be measured, processes the current frame image in real time, and obtains three-dimensional point data for the current frame. The current frame three-dimensional point data is configured as discrete point data representing the current surface shape contour of the object to be measured. Based on the current frame three-dimensional point data, a real-time mesh reconstruction is performed to obtain current three-dimensional mesh update data. The current three-dimensional mesh update data is configured as continuous surface data representing the surface shape contour of the object to be measured. The use of this technical solution can improve the accuracy of the scan results of the object to be measured scanned by a handheld three-dimensional scanner and more directly reflect the final scan results.

[0086] Figure 2 shows a flow chart of another three-dimensional scanning and reconstruction method provided by the second embodiment of the present disclosure. The present embodiment is optimized based on the above embodiment, and the present embodiment can be combined with various optional solutions in one or more of the above embodiments.

[0087] As shown in FIG2 , the three-dimensional scanning reconstruction method may include the following steps:

[0088] S201: Acquire a current frame image of the surface of the object to be measured.

[0089] For example, this step may refer to the above-mentioned step S101 and will not be described in detail.

[0090] S202 : Process the current frame image in real time to obtain three-dimensional point data of the current frame, wherein the three-dimensional point data of the current frame is configured as discrete point data representing the current surface shape contour of the object to be measured.

[0091] For example, this step may refer to the above-mentioned step S102 and will not be described in detail.

[0092] S203. When the current frame image is not the first frame image, the current frame 3D point data is spliced ​​and fused with the current 3D point data in real time to obtain current 3D point update data; the current 3D point data is the 3D point data obtained by splicing and fusion of one or more frames of 3D point data obtained before the current frame 3D point data.

[0093] In this embodiment, when the current frame image is the first frame image, the first frame image is directly processed in real time to obtain the first frame 3D point data, and then the first frame 3D point data is meshed in real time to obtain the current 3D mesh update data.

[0094] In this embodiment, when the current frame image is not the first frame image, and the non-first frame image is the second frame image, the current frame three-dimensional point data is the second frame three-dimensional point data, and the current three-dimensional point data is the first frame three-dimensional point data, then the current three-dimensional point update data is obtained by real-time splicing and fusion of the first frame three-dimensional point data and the second frame three-dimensional point data.

[0095] In this embodiment, when the current frame image is not the first frame image, and the non-first frame image is the third frame image, the current frame 3D point data is the third frame 3D point data, and the current 3D point data is the first frame 3D point data and the second frame 3D point data. Then, the current 3D point update data is obtained by real-time splicing and fusion of the first frame 3D point data, the second frame 3D point data, and the third frame 3D point data. When the current frame image is not the first frame image, and the non-first frame image is the Nth frame image, where N is greater than 3, the process for determining the current 3D mesh update data is the same as the process for determining the current 3D mesh update data for the third frame image.

[0096] S204: Determine, in the current 3D point update data, a portion of data corresponding to the current frame 3D point data.

[0097] In this embodiment, if the current frame three-dimensional point data is the third frame three-dimensional point data, since the current three-dimensional point update data includes the first frame three-dimensional point data, the second frame three-dimensional point data and the third frame three-dimensional point data, it is necessary to determine the third frame three-dimensional point data as partial data in the current three-dimensional point update data.

[0098] S205 . Based on the current three-dimensional grid data, perform real-time grid reconstruction on part of the data to obtain current three-dimensional grid update data; wherein the current three-dimensional grid data is three-dimensional grid data generated from the current three-dimensional point data.

[0099] In this embodiment, if the current frame three-dimensional point data is the third frame three-dimensional point data, the current three-dimensional mesh data is obtained from the first frame three-dimensional point data and the second frame three-dimensional point data, then based on the current three-dimensional mesh data, the third frame three-dimensional point data is reconstructed in real time to obtain the current three-dimensional mesh update data.

[0100] In one example, real-time mesh reconstruction is performed on part of the data to obtain current 3D mesh update data, including:

[0101] Based on partial data, the topological relationship between the neighborhoods of each 3D point is constructed in parallel to obtain the current 3D grid update data.

[0102] In this embodiment, the graphics card processes some of the data in parallel, specifically by constructing the topological relationships between the neighborhoods of each 3D point in the data in parallel to obtain the current 3D grid update data. This arrangement has the advantage of improving the processing efficiency of the graphics card.

[0103] In one example, based on partial data, topological relationships between neighborhoods of 3D points are constructed in parallel to obtain current 3D grid update data, including:

[0104] Divide some data into multiple spatial blocks;

[0105] Construct the topological relationship between the neighborhood of each 3D point in each spatial block to obtain multiple sub-grids;

[0106] Based on the multiple sub-grids constructed for each spatial block, the current three-dimensional grid update data is obtained.

[0107] In one example, a spatial block is composed of one or more voxels in the voxel grid where the current three-dimensional point update data is located. For better explanation, please refer to the structural diagram of a spatial block shown in Figure 3. A three-dimensional point neighborhood refers to a point within a preset range of a three-dimensional point. A topological relationship is a connection structure relationship. Through the topological relationship between each three-dimensional point neighborhood in each spatial block, multiple sub-grids are obtained, where the sub-grid can be a triangular sub-grid or a quadrilateral sub-grid. The current three-dimensional grid update data is then constructed based on the above sub-grids and historical sub-grids.

[0108] In one example, a topological relationship between the neighborhood of each 3D point in each spatial block is constructed to obtain multiple sub-grids, including:

[0109] For each spatial block, determine the initial edge as the current topological edge based on the 3D points in the spatial block;

[0110] Traverse each 3D point outside the current topological edge in the spatial block, determine the triangle formed by each 3D point and the current topological edge, and determine the triangle that meets the preset conditions as the sub-mesh face;

[0111] Update the two edges in the patch other than the initial edge as the current topology edges.

[0112] In one example, the initial edge is the first edge determined, for example, edge AB. Then, each 3D point outside edge AB is traversed within the spatial block, which is denoted as C for ease of explanation. BC and AC are the current topological edges. It is then determined whether the triangle formed by A, B, and C meets a preset condition, where the preset condition includes at least one of the following:

[0113] 1)d c =|AB|+|AC|+|BC|-min(|AB|, |AC|, |BC|);

[0114] 2) For non-initial patches, there is no overlap between the patches obtained later and the patches obtained earlier.

[0115] In one example, determining an initial edge as a current topological edge based on a 3D point in a spatial block includes:

[0116] Determine any three-dimensional point that does not belong to any face in the spatial block as the initial point;

[0117] Traversing each 3D point except the initial point in the spatial block, calculating the distance between each 3D point and the initial point, and determining the 3D point with the smallest distance as the first connected 3D point;

[0118] An edge formed by the initial point and the first connected three-dimensional point is determined as the initial edge.

[0119] In one example, any three-dimensional point that does not belong to any facet is determined in the spatial block as the initial point A, and each three-dimensional point other than the initial point is traversed in the spatial block. Then, the three-dimensional point with the smallest distance value is used as the first connected three-dimensional point B, and AB is determined as the initial edge.

[0120] In one example, based on the multiple subgrids constructed for each spatial block, current 3D grid update data is obtained, including:

[0121] In any spatial block, find an edge that is not shared by multiple faces as the current topological edge;

[0122] Traverse each 3D point in the neighborhood space block of the space block, determine the triangle formed by each 3D point and the current topological edge, and determine the triangle that meets the preset conditions as the face of the current 3D grid.

[0123] In one example, any spatial block includes the current spatial block and other spatial blocks. For example, if EF is an edge that is not shared by multiple facets, then EF is the current topological edge. Then, each 3D point G in the neighboring spatial blocks of the spatial block is traversed to determine the triangle formed by EFG, and the triangle satisfies a preset condition, wherein the preset condition includes at least one of the following:

[0124] 1)d c =|EF|+|EG|+|FG|-min(|EF|, |EG|, |FG|);

[0125] 2) For non-initial patches, there is no overlap between the patches obtained later and the patches obtained earlier.

[0126] In one example, determining an initial edge as a current topological edge based on a 3D point in a spatial block includes:

[0127] Determine any three-dimensional point that does not belong to any face in the spatial block as the initial point;

[0128] Traversing each 3D point except the initial point in the spatial block, calculating the distance between each 3D point and the initial point, and determining the 3D point with the smallest distance as the first connected 3D point;

[0129] An edge formed by the initial point and the first connected three-dimensional point is determined as the initial edge.

[0130] In one example, any three-dimensional point that does not belong to any facet is determined in the spatial block as the initial point A, and each three-dimensional point other than the initial point is traversed in the spatial block. Then, the three-dimensional point with the smallest distance value is used as the first connected three-dimensional point B, and AB is determined as the initial edge.

[0131] In one example, based on the multiple subgrids constructed for each spatial block, current 3D grid update data is obtained, including:

[0132] In any spatial block, find an edge that is not shared by multiple faces as the current topological edge;

[0133] Traverse each 3D point in the neighborhood space block of the space block, determine the triangle formed by each 3D point and the current topological edge, and determine the triangle that meets the preset conditions as the face of the current 3D grid.

[0134] In one example, any spatial block includes the current spatial block and other spatial blocks. For example, if EF is an edge that is not shared by multiple facets, then EF is the current topological edge. Then, each 3D point G in the neighboring spatial blocks of the spatial block is traversed to determine the triangle formed by EFG, and the triangle satisfies a preset condition, wherein the preset condition includes at least one of the following:

[0135] 1)d c =|EF|+|EG|+|FG|-min(|EF|, |EG|, |FG|);

[0136] It should be noted that the spatial block refers to the spatial block corresponding to the current three-dimensional point update data. It can be the spatial block corresponding to part of the data in the current three-dimensional point update data, or it can be the spatial block corresponding to the current three-dimensional point data in the current three-dimensional point update data. In this way, the newly added three-dimensional grid data and the existing three-dimensional grid data can form a continuous grid.

[0137] 2) For non-initial patches, there is no overlap between the patches obtained later and the patches obtained earlier.

[0138] S206: Render the current three-dimensional grid update data and generate a three-dimensional grid display image.

[0139] In one example, a schematic diagram of a three-dimensional grid display image is shown in Figure 4. Specifically, the three-dimensional grid display image is obtained by rendering continuous surface data of the surface shape contour of the object to be measured.

[0140] The disclosed embodiments provide a 3D scanning and reconstruction method. The method determines partial data corresponding to the current frame's 3D point data within the current 3D point update data, and then reconstructs the partial data in real time based on the current 3D mesh data to obtain the current 3D mesh update data. The current 3D mesh data is the 3D mesh data generated from the current 3D point data, and the method constructs topological relationships between neighboring 3D points in the partial data in parallel to obtain the current 3D mesh update data. This technical solution enables a graphics card to construct topological relationships between neighboring 3D points in parallel to obtain the current 3D mesh update data, thereby fully utilizing computing device resources and making the obtained current 3D mesh update data more accurate.

[0141] It should be noted that processing some data in parallel, including the construction of subgrids across multiple spatial blocks, means that while one spatial block is constructing a subgrid, another spatial block is also constructing a subgrid. Also, processing some data in parallel, including the construction of a continuous grid across multiple subgrids, means that while one spatial block is constructing a patch with its neighboring spatial blocks, another spatial block is also constructing a patch with its neighboring spatial blocks. This allows for rapid acquisition of updated 3D mesh data, achieving real-time 3D mesh generation.

[0142] FIG5 is a schematic diagram of the structure of a 3D scanning and reconstruction device provided in the third embodiment of the present disclosure. The 3D scanning and reconstruction device can be understood as the above-mentioned electronic device or a part of the functional modules in the above-mentioned electronic device. As shown in FIG5 , the 3D scanning and reconstruction device 50 includes:

[0143] The acquisition module 501 is configured to acquire a current frame image of the surface of the object to be measured.

[0144] The processing module 502 is configured to perform real-time processing on the current frame image to obtain the current frame 3D point data, wherein the current frame 3D point data is configured as discrete point data representing the current surface shape contour of the object to be measured.

[0145] The reconstruction module 503 is configured to perform real-time mesh reconstruction based on the current frame 3D point data to obtain current 3D mesh update data; wherein the current 3D mesh update data is configured as continuous surface data representing the surface shape contour of the object to be measured.

[0146] The device provided in this embodiment can execute the method of any of the above embodiments, and its execution method and beneficial effects are similar, which will not be repeated here.

[0147] FIG6 is a schematic diagram of the structure of a three-dimensional scanning and reconstruction device provided in a fourth embodiment of the present disclosure. The three-dimensional scanning and reconstruction device can be understood as the above-mentioned electronic device or a functional module in the above-mentioned electronic device. As shown in FIG6 , the three-dimensional scanning and reconstruction device 60 includes:

[0148] The acquisition module 601 is configured to acquire a current frame image of the surface of the object to be measured.

[0149] The processing module 602 is configured to perform real-time processing on the current frame image to obtain the current frame 3D point data, wherein the current frame 3D point data is configured as discrete point data representing the current surface shape contour of the object to be measured.

[0150] The reconstruction module 603 is configured to perform real-time mesh reconstruction based on the current frame 3D point data to obtain current 3D mesh update data; wherein the current 3D mesh update data is configured as continuous surface data representing the surface shape contour of the object to be measured.

[0151] In one example, when the current frame image is not the first frame image, after obtaining the three-dimensional point data of the current frame, the apparatus 60 further includes:

[0152] The fusion module 604 is configured to splice and fuse the current frame 3D point data with the current 3D point data in real time to obtain the current 3D point update data; the current 3D point data is the 3D point data obtained by splicing and fusing one or more frames of 3D point data obtained before the current frame 3D point data.

[0153] In one example, when the current frame image is not the first frame image, the reconstruction module 603 includes:

[0154] The determining unit 6031 is configured to determine, in the current 3D point update data, a portion of data corresponding to the 3D point data of the current frame.

[0155] The reconstruction unit 6032 is configured to perform real-time mesh reconstruction on part of the data based on the current 3D mesh data to obtain current 3D mesh update data; wherein the current 3D mesh data is 3D mesh data generated from the current 3D point data.

[0156] In one example, after obtaining the current three-dimensional grid update data, the apparatus 60 further includes:

[0157] The rendering module 605 is configured to render the current 3D mesh update data and generate a 3D mesh display image.

[0158] In one example, the reconstruction module 603 includes:

[0159] The construction unit 6033 is configured to construct the topological relationship between the neighborhoods of each three-dimensional point in parallel based on the partial data to obtain the current three-dimensional grid update data.

[0160] In one example, the construction unit 6033 includes:

[0161] The division subunit 60331 is configured to divide part of the data into multiple spatial blocks.

[0162] The construction subunit 60332 is configured to construct a topological relationship between the neighborhood of each three-dimensional point in each spatial block to obtain multiple sub-grids.

[0163] The determination subunit 60333 is configured to obtain current three-dimensional grid update data based on multiple sub-grids constructed by each spatial block.

[0164] In one example, the construction subunit 60332 is specifically configured as follows:

[0165] For each spatial block, determine the initial edge as the current topological edge based on the 3D points in the spatial block;

[0166] Traverse each 3D point outside the current topological edge in the spatial block, determine the triangle formed by each 3D point and the current topological edge, and determine the triangle that meets the preset conditions as the sub-mesh face;

[0167] Update the two edges in the patch other than the initial edge as the current topology edges.

[0168] In one example, the construction subunit 60332 is specifically configured as follows:

[0169] Determine any three-dimensional point that does not belong to any face in the spatial block as the initial point;

[0170] Traversing each 3D point except the initial point in the spatial block, calculating the distance between each 3D point and the initial point, and determining the 3D point with the smallest distance as the first connected 3D point;

[0171] An edge formed by the initial point and the first connected three-dimensional point is determined as the initial edge.

[0172] In one example, the determination subunit 60333 is specifically configured to:

[0173] In any spatial block, find an edge that is not shared by multiple faces as the current topological edge;

[0174] Traverse each 3D point in the neighborhood space block of the space block, determine the triangle formed by each 3D point and the current topological edge, and determine the triangle that meets the preset conditions as the face of the current 3D grid.

[0175] The device provided in this embodiment can execute the method of any of the above embodiments, and its execution method and beneficial effects are similar, which will not be repeated here.

[0176] An embodiment of the present disclosure further provides an electronic device, comprising: a memory storing a computer program; and a processor configured to execute the computer program. When the computer program is executed by the processor, the method of any of the above embodiments can be implemented.

[0177] For example, FIG7 is a schematic diagram of the structure of an electronic device in an embodiment of the present disclosure. Specific reference will be made to FIG7 below, which shows a schematic diagram of the structure of an electronic device 1000 suitable for implementing an embodiment of the present disclosure. The electronic device 1000 in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), head-mounted display devices, etc., as well as fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG7 is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

[0178] As shown in FIG7 , the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 into a random access memory (RAM) 1003. Various programs and data required for the operation of the electronic device 1000 are also stored in the RAM 1003. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0179] Typically, the following devices can be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an input device 1006 including, for example, a display;

[0180] , speakers, vibrators, and the like; storage devices 1008, such as magnetic tapes and hard disks; and communication devices 1009. Communication devices 1009 may allow electronic device 1000 to communicate with other devices wirelessly or by wire to exchange data. While FIG7 illustrates electronic device 1000 with various devices, it should be understood that not all illustrated devices are required to be implemented or present. More or fewer devices may alternatively be implemented or present.

[0181] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code configured to execute the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1009, or installed from the storage device 1008, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0182] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or convey a program configured for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be conveyed using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0183] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0184] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0185] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device is enabled to: obtain a current frame image of the surface of the object to be measured; perform real-time processing on the current frame image to obtain current frame three-dimensional point data, wherein the current frame three-dimensional point data is configured as discrete point data representing the current surface shape contour of the object to be measured; perform real-time mesh reconstruction based on the current frame three-dimensional point data to obtain current three-dimensional mesh update data; wherein the current three-dimensional mesh update data is configured as continuous surface data representing the surface shape contour of the object to be measured.

[0186] Computer program code configured to perform the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0187] The flow charts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions configured to implement the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0188] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0189] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0190] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0191] The embodiments of the present disclosure also provide a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method of any of the above embodiments can be implemented. The execution method and beneficial effects are similar and will not be repeated here.

[0192] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0193] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein. Industrial Applicability

[0194] The 3D scanning and reconstruction method provided by the present disclosure obtains a current frame image of the surface of the object to be measured, processes the current frame image in real time, and obtains current frame 3D point data, wherein the current frame 3D point data is configured as discrete point data representing the current surface shape contour of the object to be measured, and performs real-time mesh reconstruction based on the current frame 3D point data to obtain current 3D mesh update data; wherein the current 3D mesh update data is configured as continuous surface data representing the surface shape contour of the object to be measured. The adoption of this technical solution can make the scanning results of the object to be measured scanned by a handheld 3D scanner more accurate, can more directly reflect the final scanning results, and has strong industrial practicality.

Claims

1. A three-dimensional scanning and reconstruction method, characterized in that, Including: Obtain the current frame image of the surface of the object to be measured; Perform real-time processing on the current frame image to obtain current frame three-dimensional point data, where the current frame three-dimensional point data is configured to represent discrete point data of the current surface shape contour of the object to be measured; Perform real-time mesh reconstruction based on the current frame three-dimensional point data to obtain current three-dimensional mesh update data; wherein, the current three-dimensional network update data is configured to represent continuous surface data of the surface shape contour of the object to be measured.

2. The method according to claim 1, characterized in that When the current frame image is not the first frame image, after obtaining the current frame three-dimensional point data, the method further includes: Perform real-time stitching and fusion of the current frame three-dimensional point data and the current three-dimensional point data to obtain current three-dimensional point update data; the current three-dimensional point data is the three-dimensional point data after stitching and fusion of one or more frames of three-dimensional point data obtained before the current frame three-dimensional point data.

3. The method according to claim 2, wherein When the non-first frame image is the second frame image, the current frame three-dimensional point data is the second frame three-dimensional point data, the current three-dimensional point data is the first frame three-dimensional point data, and the current three-dimensional point update data is obtained by real-time stitching and fusion of the first frame three-dimensional point data and the second frame three-dimensional point data.

4. The method according to claim 2, wherein When the non-first frame image is the third frame image, the current frame three-dimensional point data is the third frame three-dimensional point data, the current three-dimensional point data is the first frame three-dimensional point data and the second frame three-dimensional point data, and the current three-dimensional point update data is obtained by real-time stitching and fusion of the first frame three-dimensional point data, the second frame three-dimensional point data, and the third frame three-dimensional point data.

5. The method according to claim 2, wherein When the current frame image is not the first frame image, the performing real-time mesh reconstruction based on the current frame three-dimensional point data to obtain current three-dimensional mesh update data includes: Determine the partial data corresponding to the current frame three-dimensional point data in the current three-dimensional point update data; Perform real-time mesh reconstruction on the partial data based on the current three-dimensional mesh data to obtain current three-dimensional mesh update data; wherein, the current three-dimensional mesh data is the three-dimensional mesh data generated from the current three-dimensional point data.

6. The method according to claim 1 or 5, characterized in that, After obtaining the current three-dimensional mesh update data, the method further includes: Render the current three-dimensional mesh update data and generate a three-dimensional mesh display image.

7. The method according to claim 5, wherein The performing real-time mesh reconstruction on the partial data to obtain current three-dimensional mesh update data includes: Parallelly construct the topological relationship between the neighborhoods of each three-dimensional point based on the partial data to obtain current three-dimensional mesh update data.

8. The method according to claim 7, characterized in that The parallelly constructing the topological relationship between the neighborhoods of each three-dimensional point based on the partial data to obtain current three-dimensional mesh update data includes: Divide the partial data into multiple spatial volume blocks; Construct the topological relationship between the neighborhoods of each three-dimensional point within each spatial volume block to obtain multiple sub-meshes; Based on the multiple sub-meshes constructed for each spatial volume block, obtain the current three-dimensional mesh update data.

9. The method according to claim 8, characterized in that, The constructing the topological relationship between the neighborhoods of each three-dimensional point within each spatial volume block to obtain multiple sub-meshes includes: For each spatial volume block, determine an initial edge based on the three-dimensional points within the spatial volume block as the current topological edge; Traverse every three-dimensional point outside the current topological edge within the spatial volume, determine the triangles formed by each three-dimensional point and the current topological edge, and determine the triangles that meet the preset conditions as the patches of the sub-grid; Update the two edges other than the initial edge in the patches as the current topological edge.

10. The method according to claim 9, wherein The determining the initial edge as the current topological edge based on the three-dimensional points within the spatial volume includes: Determine any three-dimensional point that does not belong to any patch within the spatial volume as the initial point; Traverse every three-dimensional point outside the initial point within the spatial volume, calculate the distance value between each three-dimensional point and the initial point, and determine the three-dimensional point with the minimum distance value as the first connected three-dimensional point; Determine the edge formed by the initial point and the first connected three-dimensional point as the initial edge.

11. The method according to claim 9, wherein Use AB to represent the initial edge, C to represent traversing every three-dimensional point outside the AB edge within the spatial volume, and BC and AC to represent the current topological edge. Then, the triangle formed by A, B, and C needs to meet the preset conditions, and the preset conditions include: d c = |AB| + |AC| + |BC| - min(|AB|, |AC|, |BC|), and / or, for non-initial patches, there is no overlap between the subsequently obtained patches and the previously obtained patches.

12. The method according to claim 8, wherein The obtaining the current three-dimensional grid update data based on the multiple sub-grids constructed for each spatial volume includes: Find the edge that is not shared by multiple patches in any one of the spatial volumes as the current topological edge; Traverse every three-dimensional point within the neighborhood spatial volume of the spatial volume, determine the triangles formed by each three-dimensional point and the current topological edge, and determine the triangles that meet the preset conditions as the patches of the current three-dimensional grid.

13. The method according to claim 12, wherein Use EF to represent the current topological edge, G to represent traversing every three-dimensional point within the neighborhood spatial volume of the spatial volume, EG to represent the edge formed by the three-dimensional point G and one end of the current topological edge EF, and FG to represent the edge formed by the three-dimensional point G and the other end of the current topological edge EF. Then, the triangle formed by E, F, and G needs to meet the preset conditions, and the preset conditions include d c = |EF| + |EG| + |FG| - min(|EF|, |EG|, |FG|), and / or, for non-initial patches, there is no overlap between the subsequently obtained patches and the previously obtained patches.

14. The method according to claim 8, characterized in that, Each spatial volume is composed of one voxel or multiple voxels in the voxel grid where the current three-dimensional point update data is located.

15. The method according to claim 8, wherein The spatial volume is the spatial volume corresponding to the current three-dimensional point update data; it includes the spatial volume corresponding to part of the data in the current three-dimensional point update data and / or the spatial volume corresponding to the current three-dimensional point data in the current three-dimensional point update data, so that the newly added three-dimensional grid data and the existing three-dimensional grid data form a continuous grid.

16. A three-dimensional scanning and reconstruction device, characterized in that, The device includes: An acquisition module configured to acquire the current frame image of the surface of the object to be measured; A processing module configured to perform real-time processing on the current frame image to obtain the current frame three-dimensional point data, where the current frame three-dimensional point data is configured to represent the discrete point data of the current surface shape contour of the object to be measured; A reconstruction module configured to perform real-time grid reconstruction based on the current frame three-dimensional point data to obtain the current three-dimensional grid update data; where the current three-dimensional network update data is configured to represent the continuous surface data of the surface shape contour of the object to be measured.

17. An electronic device, characterized in that, It includes: A processor and a memory, where a computer program is stored in the memory. When the computer program is executed by the processor, the processor executes the method according to any one of claims 1-15.

18. A computer-readable storage medium, characterized in that, A computer program is stored in the storage medium, and when the computer program is executed by a processor, the method described in any one of claims 1-15 is implemented.

Images