Marker point optimization method, device, system, computer equipment and storage medium

Abstract

The application relates to a marker point optimization method, device, system, computer equipment and a storage medium, wherein the method comprises the following steps: dividing a scanning object into multiple scanning areas; determining first optimization marker point data corresponding to each scanning area when a closed-loop marker point structure is formed between the multiple scanning areas; splicing the first optimization marker point data corresponding to each scanning area based on a topology structure in each group of first optimization marker point data to obtain global marker point data; the topology structure is formed by at least four marker points; and performing global optimization processing on the global marker point data according to a preset second accuracy threshold and the topology structure to obtain target marker point data. Through the application, the problem that the cumulative error of marker point splicing is large in the related art and the scanning of a large-size scanning object cannot be applied is solved, the cumulative error of marker point splicing is reduced, and the scanning of a large-size scanning object can be applied.

Description

Technical Field

[0001] This application relates to the field of 3D scanning technology, and in particular to marker point optimization methods, apparatus, systems, computer equipment, and storage media. Background Technology

[0002] When scanning large objects, structured light 3D scanners require marking points on the object before scanning. During the scanning process, photogrammetry is used to capture images of the marking points frame by frame, and then the images from each frame are stitched together. However, as the scanning progresses, stitching errors accumulate and propagate from frame to frame, and these errors are positively correlated with the number of scanned frames. If these errors are not eliminated, they will severely affect the overall scanning result, such as causing distortion or breakage of the scanned model, ultimately rendering the scanned data unusable.

[0003] There is currently no effective solution to the problem that the accumulated error of marker stitching in related technologies is large and cannot be applied to scanning large-sized objects. Summary of the Invention

[0004] This embodiment provides a marker point optimization method, apparatus, system, computer device, and storage medium to solve the problem in related technologies where the cumulative error of marker point stitching is large and cannot be applied to scanning large-size objects.

[0005] Firstly, this embodiment provides a marker point optimization method, including:

[0006] The object to be scanned is divided into multiple scanning regions; each of the scanning regions has a marker point on the object being scanned.

[0007] When forming a closed-loop marker structure between multiple scanning regions, first optimized marker data corresponding to each scanning region is determined; the optimization accuracy of the first optimized marker data meets a preset first accuracy threshold.

[0008] Based on the topology in each group of the first optimized marker point data, the first optimized marker point data corresponding to each scanned region are spliced ​​together to obtain global marker point data; the topology is formed by at least four marker points;

[0009] Based on the preset second precision threshold and the topology, the global marker point data is subjected to global optimization processing to obtain the target marker point data.

[0010] In some embodiments, the step of performing global optimization processing on the global marker point data according to a preset second precision threshold and the topology to obtain target marker point data includes:

[0011] Based on the topology in each of the first optimized marker point data, a first transformation relationship is determined between the first optimized marker point data corresponding to two adjacent scan areas;

[0012] Based on the first transformation relationship, the first optimized marker point data corresponding to each scanned area are spliced ​​together to obtain the first spliced ​​marker point data;

[0013] The first stitching accuracy between each scanning area is determined based on the distance between the same topological structure in the first stitching marker data.

[0014] When the first splicing accuracy meets the preset second accuracy threshold, the splicing marker point data is used as the target marker point data.

[0015] In some of these embodiments, it also includes:

[0016] When the first splicing accuracy does not meet the preset second accuracy threshold, each of the first conversion relationships is updated according to the first splicing accuracy and the second accuracy threshold to obtain the second conversion relationship;

[0017] Based on the second transformation relationship, the first optimized marker point data corresponding to each scanned area are spliced ​​together to obtain the second spliced ​​marker point data;

[0018] The second stitching accuracy between each scanning area is determined based on the distance between the same topology in the second stitching marker data.

[0019] The second stitching mark data is used as the target mark data until the second stitching accuracy meets the preset second accuracy threshold.

[0020] In some embodiments, the step of performing global optimization processing on the global marker point data according to a preset second precision threshold and the topology to obtain target marker point data includes:

[0021] The marker points in the global marker point data are reconstructed in three dimensions, and the three-dimensional reconstruction results are reprojected to obtain the second reprojected marker point data;

[0022] The second optimization accuracy of the global marker point data is determined based on the distance difference between the same topology in the second reprojection marker point data and the global marker point data;

[0023] When the second optimization accuracy meets the preset second accuracy threshold, the second reprojection marker data is used as the target marker data.

[0024] In some embodiments, determining the first optimized marker point data corresponding to each of the scanned regions includes:

[0025] Based on the scanning area, the scanning object is scanned in sections to obtain a two-dimensional image corresponding to each scanning area;

[0026] Based on the first precision threshold, the marker points in each group of two-dimensional images are optimized to determine the first optimized marker point data corresponding to each scanned area.

[0027] In some embodiments, the step of optimizing the marker points in each group of two-dimensional images based on the first precision threshold to determine the first optimized marker point data corresponding to each scanned region includes:

[0028] The marker points in each set of two-dimensional images are reconstructed in three dimensions, and the three-dimensional reconstruction results are reprojected to obtain the first reprojected marker point data;

[0029] The first optimization accuracy of each group of two-dimensional images is determined based on the distance difference between the first reprojection marker point data and the same topological structure in the two-dimensional image;

[0030] When the first optimization accuracy meets the preset first accuracy threshold, the first reprojection marker point data is used as the first optimized marker point data corresponding to the scanning area;

[0031] When the optimization accuracy does not meet the preset first accuracy threshold, the first reprojection marker data is optimized until the optimization accuracy meets the preset first accuracy threshold.

[0032] In some embodiments, after dividing the scanned object into multiple scan regions, the method further includes:

[0033] When an open-loop marker structure is formed between multiple scanning regions, a two-dimensional image of the scanning aid is acquired as a two-dimensional image of the open-loop region; the scanning aid is disposed in the open-loop region, and the marker is disposed on the scanning aid.

[0034] In some of these embodiments, it also includes:

[0035] Based on the target marker point data, the scan data corresponding to each scan area is stitched together to complete the three-dimensional scan.

[0036] Secondly, this embodiment provides a marker point optimization device, including: a division module, a local optimization module, a stitching module, and a global optimization module;

[0037] The segmentation module is used to divide the scanned object into multiple scan regions; each scanned object in the scan region is provided with a marker point;

[0038] The local optimization module is used to determine the first optimized marker point data corresponding to each of the multiple scanning regions when forming a closed-loop marker point structure between the multiple scanning regions; the optimization accuracy of the first optimized marker point data meets a preset first accuracy threshold.

[0039] The stitching module is used to stitch together the first optimized marker point data corresponding to each scanning area based on the topological structure in each group of the first optimized marker point data to obtain global marker point data; the topological structure is formed by at least four marker points;

[0040] The global optimization module is used to perform global optimization processing on the global marker point data according to the preset second precision threshold and the topology to obtain the target marker point data.

[0041] Thirdly, this embodiment provides a marker point optimization system, including: a 3D scanning device and a processing device; wherein the 3D scanning device and the processing device are connected.

[0042] The three-dimensional scanning device is used to scan the object being scanned;

[0043] The processing device is used to execute the marker point optimization method described in the first aspect above.

[0044] In some embodiments, the system further includes a scanning aid with marked points;

[0045] The scanning aid is set in the open-loop region of the scanned object and is used to combine multiple scanning regions to form a closed-loop marker point structure.

[0046] Thirdly, this embodiment provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the marker point optimization method described in the first aspect above.

[0047] Fourthly, this embodiment provides a storage medium storing a computer program that, when executed by a processor, implements the marker optimization method described in the first aspect above.

[0048] Compared with related technologies, the marker optimization method, apparatus, system, computer equipment, and storage medium provided in this embodiment divide the scanned object into multiple scan regions; each scanned object in each scan region is provided with marker points; when a closed-loop marker point structure is formed between multiple scan regions, the first optimized marker point data corresponding to each scan region is determined; the optimization accuracy of the first optimized marker point data meets a preset first accuracy threshold; based on the topology in each group of first optimized marker point data, the first optimized marker point data corresponding to each scan region are spliced ​​together to obtain global marker point data; the topology is formed by at least four marker points; according to the preset second accuracy threshold and the topology, the global marker point data is globally optimized to obtain target marker point data. This solves the problem in related technologies where the cumulative error of marker point splicing is large and cannot be applied to the scanning of large-size scanned objects. By combining preliminary optimization of scanned regions with global optimization, the cumulative error of marker point splicing is reduced, thus making it applicable to the scanning of large-size scanned objects.

[0049] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0050] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0051] Figure 1 This is a structural block diagram of a marker point optimization system provided in an embodiment of this application;

[0052] Figure 2 This is a flowchart of a marker point optimization method provided in an embodiment of this application;

[0053] Figure 3 yes Figure 2 Flowchart of step S240;

[0054] Figure 4 This is a structural block diagram of a marker optimization device provided in an embodiment of this application.

[0055] In the diagram: 1. 3D scanning equipment; 2. Processing equipment; 3. Scanning aids; 210. Dividing module; 220. Local optimization module; 230. Stitching module; 240. Global optimization module. Detailed Implementation

[0056] To better understand the purpose, technical solution, and advantages of this application, the application is described and explained below in conjunction with the accompanying drawings and embodiments.

[0057] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0058] Figure 1 This is a schematic diagram of the structure of an exemplary marker point optimization system according to some embodiments of the present invention. (Reference) Figure 1 As shown, the marker point optimization system includes: a 3D scanning device 1 and a processing device 2; wherein, the 3D scanning device 1 is connected to the processing device 2; the 3D scanning device 1 is used to scan the object; the processing device 2 is used to execute the steps of each marker point optimization method.

[0059] The scanning objects can be industrial products, agricultural products, and construction products, etc. The 3D scanning device 1 is a binocular vision scanner, which can be a handheld 3D scanning device or a tracking scanning device. For example, a marker point optimization system can include multiple 3D scanning devices 1, such as multiple handheld 3D scanning devices, cascaded tracking scanning devices, or one handheld 3D scanning device and another tracking scanning device. The 3D scanning device 1 can have multiple scanning modes, such as a marker point optimization mode and a scanning mode. The marker point optimization mode directly scans the marker points and optimizes them; the marker point optimization method of this application can be directly applied to this mode. The 3D scanning mode performs marker point optimization and 3D modeling together; it can also be performed after applying a rapid calibration mode. The marker point optimization method of this application can be directly applied to this mode.

[0060] The processor device may include, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The processing device may include one or more processors and a memory for storing data. The memory may be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the marker point optimization method in this embodiment. The processor executes various functional applications and data processing by running the computer programs stored in the memory, thereby implementing the aforementioned method. The memory may include high-speed random access memory (RAM) and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks (LANs), mobile communication networks, and combinations thereof.

[0061] In some embodiments, the marker optimization system further includes a scanning aid 3 with markers;

[0062] Scanning aid 3 is set in the open-loop area of ​​the scanned object to form a closed-loop marker point structure by combining multiple scan areas.

[0063] Scanning aids 3 include, but are not limited to, calibration plates, calibration rods, calibration balls, and handheld 3D scanning devices, etc., used for calibration; users can choose their own scanning aids 3. When scanning certain objects, the position of the object can affect the scanning process. For example, if the object is laid horizontally, the 3D scanning device 1 cannot scan the bottom of the object, and the markers pasted on the bottom cannot be scanned, thus forming an open-loop marker structure that affects the scanning. Therefore, scanning aids 3 can be placed at this open-loop position to combine multiple scanning areas to form a closed-loop marker structure.

[0064] This embodiment provides a marker point optimization method. Figure 2 This is a flowchart of the marker optimization method in this embodiment, as follows: Figure 2 As shown, the process includes the following steps:

[0065] Step S210: Divide the scanned object into multiple scan areas; wherein, each scanned object in the scan area is marked with a marker point;

[0066] Step S220: When forming a closed-loop marker point structure between multiple scanning areas, determine the first optimized marker point data corresponding to each scanning area; the optimization accuracy of the first optimized marker point data meets the preset first accuracy threshold.

[0067] Step S230: Based on the topology in each group of first optimized marker point data, the first optimized marker point data corresponding to each scan area are spliced ​​together to obtain global marker point data; the topology is formed by at least four marker points.

[0068] Step S240: Based on the preset second precision threshold and topology, perform global optimization processing on the global marker point data to obtain the target marker point data.

[0069] Specifically, based on the size and shape of the scanned object, it can be divided into at least two scanning regions. For example, if the scanned object is a hexahedron, it can be divided into six scanning regions for each face. Alternatively, some faces can be divided into two or more scanning regions, resulting in at least seven scanning regions for the hexahedron. Users can divide the scanning regions as needed without restriction, as long as the marker points between these scanning regions can form a closed-loop marker point structure. Although the scanned object is divided into multiple scanning regions, the directly scanned 2D image will cover adjacent scanning regions during the region-based scanning process, resulting in the same topological structure between these regions. However, when determining the first optimized marker point data for each scanning region, the 2D image of the target scanning region is extracted from the 2D images covering the adjacent scanning regions and optimized accordingly.

[0070] A closed-loop marker structure refers to a structure where at least four identical marker points exist between the first and last scanned 2D image frames; alternatively, it can be considered that the first and last 2D image frames share the same topological structure. Given a closed-loop marker structure across multiple scan regions, the marker data in each scan region is first optimized for accuracy to determine the first optimized marker data. This ensures that the accuracy of the marker data in each scan region meets a preset first accuracy threshold. The first accuracy threshold is the optimized standard accuracy of the marker data in each scan region; the second accuracy threshold is the optimized standard accuracy of the global stitching cumulative error. Both the first and second accuracy thresholds can be determined based on the specific application and are not subject to restriction.

[0071] The 3D coordinates of the marker points in each group of first-optimized marker point data form a topological structure. For example, the topological structure is determined by calculating the distances, angles, and other relationships between each marker point and other marker points, using each marker point as the center. Then, based on the preset second precision threshold and the topological structure, various methods can be used to perform global optimization processing on the global marker point data to obtain the target marker point data. The target marker point data can be considered as the 3D coordinates and transformation matrix (RT matrix) of each marker point after global optimization. For example, if the left camera coordinate system in the binocular system is taken as the world coordinate system, then the transformation matrix is ​​the RT matrix of each 2D image from the single-camera coordinate system to the world coordinate system.

[0072] In related technologies, marker points are captured frame by frame, and then the images from each frame are stitched together. However, as scanning progresses, the stitching error of each frame accumulates and propagates, and is positively correlated with the number of scan frames. This can lead to distortion and breakage of the scanned model, resulting in scanned data that fails to meet usage requirements. This application, through the aforementioned steps, forms a closed-loop marker point structure between multiple scan areas. Preliminary optimization of each scan area yields first optimized marker point data that meets a first precision threshold, controlling the error in each scan area. Then, combined with a preset second precision threshold and topology, global optimization processing is performed on the global marker point data to reduce the overall cumulative stitching error, making it suitable for scanning large-size objects. This solves the problem in related technologies where the cumulative error of marker point stitching is large, making it unsuitable for scanning large-size objects.

[0073] The following section explains the implementation methods of various global optimizations.

[0074] One implementation method involves adjusting the first transformation relationship between the first optimized marker point data of each scan area as a whole. This can be considered as adjusting only the overall scan area, modifying the position of the first optimized marker point data within each scan area, without individually adjusting the positions of the individual first optimized marker point data within each scan area.

[0075] In some of these embodiments, such as Figure 3 As shown, step S240, which involves performing global optimization processing on the global marker point data based on a preset second precision threshold and topology to obtain the target marker point data, includes the following steps:

[0076] Step S241: Based on the topological structure in each first optimized marker point data, determine the first transformation relationship between the first optimized marker point data corresponding to two adjacent scanning areas;

[0077] Step S242: Based on the first transformation relationship, the first optimized marker point data corresponding to each scan area are spliced ​​together to obtain the first spliced ​​marker point data;

[0078] Step S243: Determine the first stitching accuracy between each scanning area based on the distance between the same topological structure in the first stitching marker point data.

[0079] Step S244: When the first stitching accuracy meets the preset second accuracy threshold, the stitching marker data is used as the target marker data.

[0080] Specifically, since scanning adjacent scanning regions results in the obtained 2D image covering the corresponding adjacent scanning regions, for example, scanning adjacent regions of scanning region A and scanning region B will result in a 2D image covering both scanning regions A and B. Due to the high scanning rate, adjacent frames of 2D images will have the same topological structure. Therefore, the first transformation relationship between the first optimized marker point data corresponding to two adjacent scanning regions is determined by using the similarity or identical topological structure in the first optimized marker point data of each scanning region as a matching condition. For example, given 6 scanning regions (scanning region A, scanning region B, scanning region C, scanning region D, scanning region E, scanning region F, and a loop between scanning regions A and F), the first transformation relationship between scanning region A and scanning region B is denoted by RTa, the first transformation relationship between scanning region B and scanning region C is denoted by RTb, similarly, the first transformation relationship between scanning region C and scanning region D is denoted by RTc, the first transformation relationship between scanning region D and scanning region E is RTd, the first transformation relationship between scanning region E and scanning region F is denoted by RTe, and the first transformation relationship between scanning region F and scanning region A is denoted by RTf.

[0081] Based on the first transformation relationship described above, the first optimized marker point data corresponding to each scanning area is stitched together to obtain the first stitched marker point data. The stitching can be sequential, for example: from scanning area A to scanning area C to scanning area F to scanning area A; or from scanning area A to scanning area C; from scanning area A to scanning area F to scanning area D; examples are not provided here. Next, the distance difference between scanning areas for the same topological structure in the first stitched marker point data is calculated, and the average of these distance differences is taken as the first stitching accuracy. The position of the overall first optimized marker point data for each scanning area is continuously adjusted based on the average until the first stitching accuracy meets a preset second accuracy threshold. The stitched marker point data is then used as the target marker point data.

[0082] In some embodiments, step S240, which involves performing global optimization processing on the global marker point data based on a preset second precision threshold and topology to obtain target marker point data, further includes the following steps:

[0083] When the first stitching accuracy does not meet the preset second accuracy threshold, each first transformation relationship is updated according to the first stitching accuracy and the second accuracy threshold to obtain the second transformation relationship;

[0084] Based on the second transformation relationship, the first optimized marker point data corresponding to each scan area are spliced ​​together to obtain the second spliced ​​marker point data.

[0085] The second stitching accuracy between each scanning area is determined based on the distance between the same topology in the second stitching marker data.

[0086] The second stitching mark data is used as the target mark data until the second stitching accuracy meets the preset second accuracy threshold.

[0087] Specifically, in practical applications, it's rare for the first stitching accuracy to meet the second accuracy threshold in a single stitch. In such cases, each first transformation relationship needs to be updated to obtain the second transformation relationship before stitching continues. This process is repeated until the stitching accuracy meets the preset second accuracy threshold or the number of iterations reaches a certain number; otherwise, continuous optimization continues. For example, if the first stitching accuracy does not meet the second accuracy threshold, the first transformation relationships RTa, RTb, RTc, RTd, and RTe need to be adjusted appropriately. Then, the second transformation relationships RTa', RTb', RTc', RTd', and RTe' are updated by averaging the first and second accuracy thresholds. The adjusted transformation relationships are then used to stitch again to determine the second stitching accuracy. If the second stitching accuracy still does not meet the preset second accuracy threshold, the second transformation relationships RTa', RTb', RTc', RTd', and RTe' are updated again to obtain the third transformation relationships RTa”, RTb”, RTc”, RTd”, and RTe” until the preset second accuracy threshold is met.

[0088] In this embodiment, the first transformation relationship between the first optimized marker point data of each scanning area is adjusted by the topology structure, which greatly improves the optimization efficiency, eliminates the accumulation of cumulative stitching errors, and improves the optimization accuracy.

[0089] Another implementation method is to adjust the position of the first optimized marker point data globally.

[0090] In some embodiments, step S240, which involves performing global optimization processing on the global marker point data based on a preset second precision threshold and topology to obtain the target marker point data, includes the following steps:

[0091] Perform 3D reconstruction on the marker points in the global marker point data, and reproject the 3D reconstruction results to obtain the second reprojected marker point data;

[0092] The second optimization accuracy of the global marker point data is determined based on the distance difference between the same topology in the second projection marker point data and the global marker point data.

[0093] When the second optimization accuracy meets the preset second accuracy threshold, the second reprojection marker data is used as the target marker data.

[0094] Specifically, limit constraints and triangulation principles can be used to perform 3D reconstruction on each marker point in the global marker point data. Then, the 3D coordinates of each reconstructed marker point are reprojected to obtain second-projected marker point data. Next, the distance differences between the second-projected marker point data and the global marker point data for the same or similar topological structures are calculated, and the average of these distance differences is taken as the second optimization accuracy of the global marker point data. When the second optimization accuracy meets a preset second accuracy threshold, the second-projected marker point data is used as the target marker point data. If the second optimization accuracy does not meet the preset second accuracy threshold, the second optimization accuracy is recalculated based on the distance differences between the second-projected marker point data and the global marker point data for the same topological structures until the second optimization accuracy meets the preset second accuracy threshold or the number of iterations reaches a certain number.

[0095] In this embodiment, by adjusting the position of the first optimized marker point data globally through the topology structure, the accumulation of splicing errors can be eliminated, thereby improving the optimization accuracy.

[0096] In some embodiments, determining the first optimized marker point data corresponding to each scan region in step S220 includes the following steps:

[0097] Step S221: Based on the scanning area, the scanning object is scanned in sections to obtain a two-dimensional image corresponding to each scanning area;

[0098] Step S222: Based on the first precision threshold, optimize the marker points in each group of two-dimensional images to determine the first optimized marker point data corresponding to each scan area.

[0099] Specifically, the scanned object is divided into multiple scan regions, each of which can be scanned independently to obtain a corresponding two-dimensional image. These two-dimensional images contain markers that cover adjacent regions. Based on the scanned regions, the corresponding two-dimensional image for each scanned region is extracted from the original two-dimensional image. Then, using various BA optimization algorithms, based on a first precision threshold, the markers in each set of two-dimensional images are optimized to determine the first optimized marker data for each scanned region.

[0100] In this embodiment, scanning in sections first and then optimizing the marker points in each scanning area can maximize the control of the optimization error of the marker points in the scanning area and reduce the cumulative error.

[0101] In some embodiments, step S222, which optimizes the marker points in each group of two-dimensional images based on a first precision threshold to determine the first optimized marker point data corresponding to each scanned region, includes the following steps:

[0102] The marker points in each set of two-dimensional images are reconstructed in three dimensions, and the three-dimensional reconstruction results are reprojected to obtain the first reprojection marker point data;

[0103] The first optimization accuracy of each set of two-dimensional images is determined based on the distance difference between the first projection marker point data and the same topological structure in the two-dimensional image;

[0104] When the first optimization accuracy meets the preset first accuracy threshold, the first reprojection marker point data is used as the first optimization marker point data of the corresponding scanning area;

[0105] If the optimization accuracy does not meet the preset first accuracy threshold, the first reprojection marker data is optimized until the optimization accuracy meets the preset first accuracy threshold.

[0106] Specifically, using limit constraints and triangulation principles, each marker point in each set of two-dimensional images can be reconstructed into three dimensions. Then, the three-dimensional coordinates of each reconstructed marker point are reprojected to obtain the first reprojected marker point data. Next, the distance difference between the first reprojected marker point data and the same or similar topological structures in the two-dimensional image is calculated, and the average of these distance differences is taken as the first optimization accuracy of the two-dimensional image. When the first optimization accuracy meets a preset first accuracy threshold, the first reprojected marker point data is used as the first optimized marker point data. If the first optimization accuracy does not meet the preset first accuracy threshold, the first optimization accuracy is recalculated based on the distance difference between the first reprojected marker point data and the same topological structures in the two-dimensional image until the first optimization accuracy meets the preset first accuracy threshold or the number of iterations reaches a certain number.

[0107] In this embodiment, high-precision and rapid optimization of each scanned area is achieved by utilizing limit constraints and triangulation principles, combined with the topological structure of the marked points.

[0108] In some embodiments, the marker optimization method further includes the following steps:

[0109] After dividing the scanned object into multiple scan areas, when an open-loop marker point structure is formed between the multiple scan areas, the two-dimensional image of the scanning aid is obtained as the two-dimensional image of the open-loop area; the scanning aid is set in the open-loop area, and marker points are set on the scanning aid.

[0110] Specifically, if an open-loop marker point structure is formed between multiple scanning areas, it can be assumed that after scanning all scanning areas, there is no identical topological structure between the first and last 2D image frames. In this case, a scanning aid set in the open-loop area, which has marker points, can be scanned. During scanning, both the first and last frames will cover the scanning aid, allowing the 2D image corresponding to the scanning aid to be used as the 2D image of the open-loop area, thus achieving a closed-loop marker point structure. Finally, the marker points can be optimized based on the marker point optimization methods described in the above embodiments or by using the BA optimization algorithm, reducing cumulative stitching errors and improving scanning accuracy.

[0111] In some embodiments, the marker optimization method further includes the following steps:

[0112] Based on the target marker point data, the scan data corresponding to each scan area is stitched together to complete the 3D scan.

[0113] Specifically, using the target marker point data as the stitching parameter, the scan data corresponding to each scan area is stitched together, thereby converting the scan data corresponding to each scan area into the world coordinate system to achieve 3D modeling, obtain the target model, and complete the 3D scan.

[0114] In this embodiment, the optimized target marker data is directly applied to the stitching of the scan data, which can speed up the 3D scanning efficiency and achieve high scanning accuracy.

[0115] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0116] This embodiment also provides a marker point optimization device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0117] Figure 4 This is a structural block diagram of the marker point optimization device in this embodiment, as shown below. Figure 4 As shown, the device includes: a partitioning module 210, a local optimization module 220, a splicing module 230, and a global optimization module 240;

[0118] The partitioning module 210 is used to divide the scanned object into multiple scan regions; each scanned object in the scan region is marked with marker points;

[0119] The local optimization module 220 is used to determine the first optimized marker point data corresponding to each scanning region when forming a closed-loop marker point structure between multiple scanning regions; the optimization accuracy of the first optimized marker point data meets the preset first accuracy threshold.

[0120] The stitching module 230 is used to stitch together the first optimized marker point data corresponding to each scanning area based on the topology in each group of first optimized marker point data to obtain global marker point data; the topology is formed by at least four marker points;

[0121] The global optimization module 240 is used to perform global optimization processing on the global marker point data according to the preset second precision threshold and topology to obtain the target marker point data.

[0122] The aforementioned device solves the problem in related technologies where the cumulative error of marker point stitching is large and it cannot be used for scanning large-size objects. It reduces the cumulative error of marker point stitching, thus making it suitable for scanning large-size objects.

[0123] In some embodiments, the global optimization module 240 is further configured to determine a first transformation relationship between the first optimized marker point data corresponding to two adjacent scanning areas based on the topology in each first optimized marker point data;

[0124] Based on the first transformation relationship, the first optimized marker point data corresponding to each scan area are spliced ​​together to obtain the first spliced ​​marker point data;

[0125] The first stitching accuracy between each scanning area is determined based on the distance between the same topological structure in the first stitching marker data.

[0126] When the first stitching accuracy meets the preset second accuracy threshold, the stitching marker data is used as the target marker data.

[0127] In some embodiments, the global optimization module 240 is further configured to update each first transformation relationship according to the first stitching accuracy and the second accuracy threshold when the first stitching accuracy does not meet the preset second accuracy threshold, so as to obtain the second transformation relationship;

[0128] Based on the second transformation relationship, the first optimized marker point data corresponding to each scan area are spliced ​​together to obtain the second spliced ​​marker point data.

[0129] The second stitching accuracy between each scanning area is determined based on the distance between the same topology in the second stitching marker data.

[0130] The second stitching mark data is used as the target mark data until the second stitching accuracy meets the preset second accuracy threshold.

[0131] In some embodiments, the global optimization module 240 is further configured to perform three-dimensional reconstruction of the marker points in the global marker point data, and to reproject the three-dimensional reconstruction results to obtain second reprojected marker point data.

[0132] The second optimization accuracy of the global marker point data is determined based on the distance difference between the same topology in the second projection marker point data and the global marker point data.

[0133] When the second optimization accuracy meets the preset second accuracy threshold, the second reprojection marker data is used as the target marker data.

[0134] In some embodiments, the local optimization module 220 is also used to perform regional scanning of the scanned object based on the scanned area to obtain a two-dimensional image corresponding to each scanned area;

[0135] Based on a first precision threshold, the marker points in each group of two-dimensional images are optimized to determine the first optimized marker point data corresponding to each scan area.

[0136] In some embodiments, the local optimization module 220 is also used to perform three-dimensional reconstruction on the marker points in each group of two-dimensional images, and to reproject the three-dimensional reconstruction results to obtain first reprojected marker point data.

[0137] The first optimization accuracy of each set of two-dimensional images is determined based on the distance difference between the first projection marker point data and the same topological structure in the two-dimensional image;

[0138] When the first optimization accuracy meets the preset first accuracy threshold, the first reprojection marker point data is used as the first optimization marker point data of the corresponding scanning area;

[0139] If the optimization accuracy does not meet the preset first accuracy threshold, the first reprojection marker data is optimized until the optimization accuracy meets the preset first accuracy threshold.

[0140] In some embodiments, the marker optimization apparatus further includes an acquisition module;

[0141] The acquisition module is used to acquire a two-dimensional image of the scanning aid as a two-dimensional image of the open-loop region when an open-loop marker point structure is formed between multiple scanning regions; the scanning aid is set in the open-loop region and has marker points.

[0142] In some embodiments, the marker optimization device further includes a scanning module;

[0143] The scanning module is used to stitch together the scanning data corresponding to each scanning area based on the target marker point data to complete the 3D scan.

[0144] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0145] This embodiment also provides a computer device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0146] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0147] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0148] S1, the scanned object is divided into multiple scan areas; each scanned object in each scan area is marked with a marker point;

[0149] S2, when forming a closed-loop marker structure between multiple scanning areas, determine the first optimized marker data corresponding to each scanning area; the optimization accuracy of the first optimized marker data meets the preset first accuracy threshold.

[0150] S3, based on the topology in each group of first optimized marker point data, the first optimized marker point data corresponding to each scan area are spliced ​​together to obtain global marker point data; the topology is formed by at least four marker points;

[0151] S4. Based on the preset second precision threshold and topology, perform global optimization processing on the global marker point data to obtain the target marker point data.

[0152] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0153] Furthermore, in conjunction with the marker optimization methods provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the marker optimization methods in the above embodiments.

[0154] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0155] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0156] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0157] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method for optimizing marker points, characterized in that, include: The object to be scanned is divided into multiple scanning regions; each of the scanning regions has a marker point on the object being scanned. When forming a closed-loop marker structure between multiple scanning regions, first optimized marker data corresponding to each scanning region is determined; the optimization accuracy of the first optimized marker data meets a preset first accuracy threshold; the closed-loop marker structure refers to the fact that the first frame of two-dimensional image and the last frame of two-dimensional image obtained by scanning have the same topological structure. Based on the topology in each group of the first optimized marker point data, the first optimized marker point data corresponding to each scanned region are spliced ​​together to obtain global marker point data; the topology is formed by at least four marker points; Based on the preset second precision threshold and the topology, the global marker point data is subjected to global optimization processing to obtain the target marker point data.

2. The marker point optimization method according to claim 1, characterized in that, The step of performing global optimization processing on the global marker point data according to the preset second precision threshold and the topology to obtain target marker point data includes: Based on the topology in each of the first optimized marker point data, a first transformation relationship is determined between the first optimized marker point data corresponding to two adjacent scan areas; Based on the first transformation relationship, the first optimized marker point data corresponding to each scanned area are spliced ​​together to obtain the first spliced ​​marker point data; The first stitching accuracy between each scanning area is determined based on the distance between the same topological structure in the first stitching marker data. When the first splicing accuracy meets the preset second accuracy threshold, the splicing marker point data is used as the target marker point data.

3. The marker point optimization method according to claim 2, characterized in that, Also includes: When the first splicing accuracy does not meet the preset second accuracy threshold, each of the first conversion relationships is updated according to the first splicing accuracy and the second accuracy threshold to obtain the second conversion relationship; Based on the second transformation relationship, the first optimized marker point data corresponding to each scanned area are spliced ​​together to obtain the second spliced ​​marker point data; The second stitching accuracy between each scanning area is determined based on the distance between the same topology in the second stitching marker data. The second stitching mark data is used as the target mark data until the second stitching accuracy meets the preset second accuracy threshold.

4. The marker point optimization method according to claim 1, characterized in that, The step of performing global optimization processing on the global marker point data according to the preset second precision threshold and the topology to obtain target marker point data includes: The marker points in the global marker point data are reconstructed in three dimensions, and the three-dimensional reconstruction results are reprojected to obtain the second reprojected marker point data; The second optimization accuracy of the global marker point data is determined based on the distance difference between the same topology in the second reprojection marker point data and the global marker point data; When the second optimization accuracy meets the preset second accuracy threshold, the second reprojection marker data is used as the target marker data.

5. The marker point optimization method according to claim 1, characterized in that, The step of determining the first optimized marker point data corresponding to each scanned region includes: Based on the scanning area, the scanning object is scanned in sections to obtain a two-dimensional image corresponding to each scanning area; Based on the first precision threshold, the marker points in each group of two-dimensional images are optimized to determine the first optimized marker point data corresponding to each scanned area.

6. The marker point optimization method according to claim 5, characterized in that, The step of optimizing the marker points in each group of two-dimensional images based on the first precision threshold to determine the first optimized marker point data corresponding to each scanned region includes: The marker points in each set of two-dimensional images are reconstructed in three dimensions, and the three-dimensional reconstruction results are reprojected to obtain the first reprojected marker point data; The first optimization accuracy of each group of two-dimensional images is determined based on the distance difference between the first reprojection marker point data and the same topological structure in the two-dimensional image; When the first optimization accuracy meets the preset first accuracy threshold, the first reprojection marker point data is used as the first optimized marker point data corresponding to the scanning area; When the optimization accuracy does not meet the preset first accuracy threshold, the first reprojection marker data is optimized until the optimization accuracy meets the preset first accuracy threshold.

7. The marker point optimization method according to claim 1, characterized in that, After dividing the scanned object into multiple scan regions, the process also includes: When an open-loop marker structure is formed between multiple scanning regions, a two-dimensional image of the scanning aid is acquired as a two-dimensional image of the open-loop region; the scanning aid is disposed in the open-loop region, and the marker is disposed on the scanning aid.

8. The marker point optimization method according to any one of claims 1 to 7, characterized in that, Also includes: Based on the target marker point data, the scan data corresponding to each scan area is stitched together to complete the three-dimensional scan.

9. A marker point optimization device, characterized in that, include: The module includes a partitioning module, a local optimization module, a splicing module, and a global optimization module. The segmentation module is used to divide the scanned object into multiple scan regions; each scanned object in the scan region is provided with a marker point; The local optimization module is used to determine the first optimized marker point data corresponding to each of the multiple scanning regions when forming a closed-loop marker point structure between the multiple scanning regions; the optimization accuracy of the first optimized marker point data meets a preset first accuracy threshold; the closed-loop marker point structure refers to the fact that the first frame of two-dimensional image and the last frame of two-dimensional image obtained by scanning have the same topological structure. The stitching module is used to stitch together the first optimized marker point data corresponding to each scanning area based on the topological structure in each group of the first optimized marker point data to obtain global marker point data; the topological structure is formed by at least four marker points; The global optimization module is used to perform global optimization processing on the global marker point data according to the preset second precision threshold and the topology to obtain the target marker point data.

10. A marker point optimization system, characterized in that, include: A 3D scanning device and a processing device; wherein the 3D scanning device and the processing device are connected; The three-dimensional scanning device is used to scan the object being scanned; The processing device is used to execute the marker optimization method according to any one of claims 1 to 8.

11. The marker point optimization system according to claim 10, characterized in that, The system also includes a scanning aid with marked points; The scanning aid is set in the open-loop region of the scanned object and is used to combine multiple scanning regions to form a closed-loop marker point structure.

12. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the steps of the marker optimization method according to any one of claims 1 to 8.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the marker optimization method according to any one of claims 1 to 8.

Images