A method, device, medium, and system based on three-dimensional image segmented multi-mode optimization

Abstract

The present application provides a three-dimensional scanned image optimization method based on a segmented optimization strategy. The method includes: obtaining a pair of depth texture images in one-to-one correspondence, and using a strategy of gradual refinement to perform feature matching between the depth texture image corresponding to the current frame and the depth texture image corresponding to the sample frame; performing segmented processing on the data obtained through inter-frame motion estimation to obtain multiple data segments, and performing pose optimization on each of the data segments, selecting a key frame from each of the data segments, and combining the key frame and loop closure information to optimize between each of the data segments; respectively fixing the poses of the key frames in the corresponding data segments, and performing pose optimization on the poses of other image frames within the data segments. By adopting a segmented multi-mode optimization strategy, problems can be modeled at different abstraction levels, achieving fast and accurate optimization; by adopting a local-global-local multi-mode optimization strategy, the amount of data processing is reduced, and the efficiency and accuracy of inter-frame matching are improved.

Description

[0001] This application is a divisional application of the application with the application number 202110625611.5. The filing date of the parent case is June 4, 2021, and the invention title is an online matching optimization method and a three-dimensional scanning system combining geometry and texture. Technical Field

[0002] The present invention belongs to the field of image recognition, and more specifically, relates to a method, device, medium and system based on three-dimensional image segmentation multi-mode optimization. Background Art

[0003] In recent years, three-dimensional scanning, as a fast three-dimensional digitization technology, has been increasingly applied in various fields, including reverse engineering, industrial inspection, computer vision, CG production, etc. Especially in the current rapidly developing fields of 3D printing and intelligent manufacturing, three-dimensional scanning, as a front-end three-dimensional digitization and three-dimensional vision sensing technology, has become an important part of the industrial chain; at the same time, various applications have put forward higher requirements in many aspects such as the cost, practicability, accuracy and reliability of three-dimensional scanning.

[0004] In traditional three-dimensional scanning methods, such as the direct method, pixel gray levels are directly used for matching, resulting in poor adaptability of the system to optical changes, large interference from the environment, unstable registration results, and large pose estimation errors. At the same time, in traditional methods, when performing optimization, all frames are optimized together, with a large amount of data processing and low optimization efficiency. Summary of the Invention

[0005] This application provides a three-dimensional scanning image optimization method and a three-dimensional scanning system based on a segmentation optimization strategy. The technical solution adopted by the present invention to solve its technical problems is: a three-dimensional scanning image optimization method based on a segmentation optimization strategy, the method includes:

[0006] Obtain a corresponding pair of depth texture images, the pair of depth texture images includes a depth image collected by a depth sensor and a texture image collected by a camera device, and adopt a strategy of refinement to perform feature matching between the depth texture image corresponding to the current frame and the depth texture image corresponding to the sample frame to estimate the preliminary pose of the depth sensor, and then optimize the preliminary pose by combining geometric constraints and texture constraints to obtain a refined inter-frame motion estimation.

[0007] Perform segmentation processing on the data obtained through inter-frame motion estimation to obtain multiple data segments, and perform pose optimization on each segment of data;

[0008] Select a key frame from each segment of data, and combine the key frame and loop information to optimize between segments;

[0009] For each data segment, the poses of the key frames in the corresponding data segment are respectively fixed, and the poses of other image frames in the data segment are optimized to obtain a globally consistent and smoothly transitional motion trajectory map.

[0010] Further, the extraction of the key frames includes: if the current image frame can express global information, the current image frame is used as a preset key frame; or, if the current image frame can match the previous image frame, but the current image frame cannot match the preset reference key frame, the previous image frame of the current image frame is used as the preset key frame; or, if the current image frame can match the previous image frame and the preset reference key frame, but the overlap rate with the preset reference key frame is insufficient, the current image frame is used as the preset key frame.

[0011] The loop information includes: matching adjacent key frames, and when the matching is successful and the overlap rate reaches a set threshold, it is determined that a corresponding loop is formed, and the corresponding loop information is obtained.

[0012] Further, the present application also provides a three-dimensional scanning image optimization device based on a segmented optimization strategy. The device includes: an acquisition module for acquiring depth texture image pairs; an inter-frame motion estimation module for performing inter-frame motion estimation on the depth texture image pairs; a segmented multi-mode optimization module for selecting a key frame from each segment of data, combining the key frame and the loop information to optimize between segments; and for respectively fixing the poses of the key frames in the corresponding data segments for each data segment, and optimizing the poses of other image frames in the data segment to obtain a globally consistent and smoothly transitional motion trajectory map.

[0013] Further, the present application also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the method are implemented.

[0014] Further, the present application also provides a three-dimensional scanning system applied to an online matching optimization method, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the method are implemented.

[0015] Implementing the three-dimensional scanning image optimization method and system based on the segmented optimization strategy of the present invention, by adopting the segmented multi-mode optimization strategy, the data obtained by inter-frame motion estimation is segmented to obtain multiple data segments, and a key frame is selected from each segment of data. Combining the key frame and the loop information, the data between each segment is optimized, and the poses of the key frames of each segment of data are fixed, and the other image frames in each segment of data are optimized, which can model the problem at different levels of abstraction and achieve fast and accurate optimization. Brief Description of the Drawings

[0016] The present invention will be further described below in conjunction with the drawings and embodiments. In the drawings:

[0017] Figure 1 is a flowchart of a method for three-dimensional image segmentation multi-mode optimization in an embodiment of the present invention;

[0018] Figure 2 is a typical optical path schematic diagram of a three-dimensional scanning system;

[0019] Figure 3 is a schematic diagram of the process details of online matching optimization combining geometry and texture in an embodiment of the present invention;

[0020] Figure 4 is a schematic diagram of the effect after mesh fusion. Detailed Description of the Embodiment

[0021] For a clearer understanding of the technical features, objectives, and effects of the present invention, the specific embodiments of the present invention will now be described in detail with reference to the drawings.

[0022] Please refer to Figure 1 , the method includes:

[0023] Obtain a corresponding pair of depth texture images, where the pair of depth texture images includes a depth image collected by a depth sensor and a texture image collected by a camera device. Adopt a strategy of gradually refining, perform feature matching between the depth texture image corresponding to the current frame and the depth texture image corresponding to the sample frame to estimate the initial pose of the depth sensor, and then optimize the initial pose by combining geometric constraints and texture constraints to obtain a refined inter-frame motion estimate.

[0024] Perform segmentation processing on the data obtained through inter-frame motion estimation to obtain multiple data segments, and perform pose optimization on each segment of data.

[0025] Select a key frame from each segment of data, and optimize between segments in combination with the key frame and loop closure information.

[0026] For each data segment, fix the pose of the key frame in the corresponding data segment, and optimize the poses of other image frames in the data segment to obtain a globally consistent and smoothly transitioning motion trajectory map.

[0027] The solution of the present invention further includes:

[0028] S1. Obtain a corresponding pair of depth texture images, where the pair of depth texture images includes a depth image collected by a depth sensor and a texture image collected by a camera device.

[0029] Specifically, please refer to Figure 2 , which is a typical optical path schematic diagram of a three-dimensional scanning system. Currently, there are two optical paths. Among them, beam A is structured light, and after beam A penetrates a specific coding pattern with white light, it will be further projected onto the object to be measured. Beam B is texture illumination light, and beam B is directly projected onto the object to be measured with white light. In addition, while beam B is being projected, the imaging device will turn on the photographing function, and its exposure time is strictly synchronized with the time pulse of the beam projection. It should be noted that while a single projection of beam A is completed, the imaging device also completes a single photograph of the object onto which beam A is projected. Immediately afterwards, beam B starts to be projected, and the imaging device completes a single photograph of the object onto which beam B is projected. The above is a single cycle of the measurement process. When the above process is repeated at a certain repetition frequency, and at the same time the relative position and relative angle of the three-dimensional scanning device and the object to be measured change continuously, the continuous measurement of the structure of the three-dimensional scanning device can be completed.

[0030] Optionally, in one embodiment, the above three-dimensional scanning device will be applied in a continuous rapid measurement mode. In the current mode, beams A and B will be projected alternately to complete the measurement of the object to be measured. Among them, the beams emitted by the above three-dimensional scanning device will be output in the form of high-power short pulses, which also provides a good basis for subsequent high-precision measurement. It should be noted that in the current embodiment, the instantaneous power of beam A can reach the kilowatt level, and the pulse width is in the order of hundreds of microseconds; the instantaneous power of beam B is in the order of hundreds of watts, and the pulse width is in the order of hundreds of microseconds; the time difference between beams A and B and the camera exposure time of both are in the order of hundreds of microseconds.

[0031] S2. Adopt a strategy of successive refinement to perform feature matching between the depth texture image corresponding to the current frame and the depth texture image corresponding to the sample frame to estimate the preliminary pose of the depth sensor.

[0032] Specifically, in step S2, the estimation of the preliminary pose of the depth sensor includes:

[0033] S21. For each image frame that needs to be matched currently in the depth texture image, obtain a sample frame adapted to the image frame;

[0034] S22. For each of the image frames and sample frames, extract the corresponding image feature data, and perform image feature matching between the image frame and the corresponding sample frame to obtain a plurality of initial feature pairs;

[0035] S23. Screen out an initial transformation matrix from the plurality of initial feature pairs, and estimate the preliminary pose of the depth sensor according to the initial transformation matrix.

[0036] Specifically, this application considers extracting SIFT features from the captured RGB images, and based on the extracted SIFT features, performing feature matching between the current frame and the sample frame. It should be noted that SIFT is a widely used feature detector and descriptor, which is significantly superior to other features in terms of the fineness and stability of feature point description. During the SIFT matching process, by searching for the nearest neighbor in the image frame F i the best candidate match for each key point of the frame F j is obtained. This brute-force matching method can obtain N pairs of initial feature pairs between the frame F j and the frame F i , which are represented by the vector (U; V). These feature pairs contain both correct data (Inliers) and abnormal data (Outliers). In order to be able to screen out the correct data from these matched feature pairs, this application uses the RANSAC algorithm to screen effective sample data from the sample data set containing abnormal data. The idea of the RANSAC algorithm is: randomly select a set of RANSAC samples from N and calculate the transformation matrix (r; t). According to (r; t), calculate the number of the consistent point set that satisfies the preset error metric function (see the following formula (1)), that is, the number f of Inliers, see the following formula (2). Iterate in this way to obtain the consistent set with the largest f, and then calculate the optimal transformation matrix through the consistent set:

[0037]

[0038]

[0039] where I(U i , V i , r, t) represents that the i-th matching point pair (U i , V i ) can meet the preset condition thresholds d and θ under the constraint of the current (r; t). If it meets, I = 1, otherwise I = 0. N Pi , N Qi respectively represent the unit normal vectors of the three-dimensional points P i , Q i . N is the total number of matching point pairs. f(r, t) is the number of Inliers.

[0040] S3. Combine geometric constraints and texture constraints to optimize the preliminary pose estimated in step S2 to obtain a refined inter-frame motion estimation.

[0041] Specifically, in step S3, the combination of geometric constraints and texture constraints to optimize the preliminary pose estimated in step S2 to obtain a refined inter-frame motion estimation includes:

[0042] S31. With the initial transformation matrix as the optimization target, construct an initial optimization function E1 according to the following formula:

[0043]

[0044] where G is the geometric constraint, L is the texture constraint, ω is the confidence of the texture constraint, κi,j is the set of matching point pairs, p is the three-dimensional point of image frame i, q is the corresponding point of the three-dimensional point p in image frame j, and m is the total number of preset matching point pairs.

[0045] Specifically, combining geometric and optical constraints, the minimization objectives in the current embodiment include two parts: one is the distance between each target point and the tangent plane of its corresponding source point, and the other is the gradient error between each target point and its corresponding source point. Different weights w will be assigned to the two according to actual applications.

[0046] In the current frame F i and the sample frame F j corresponding matching point set κ i,j assuming that p = (p x , p y , p z , 1)T is the source point cloud, q = (q x , q y , q z , 1) T is the target point cloud corresponding to p, n = (n x , n y , n z , 1)T is the unit normal vector, g p is the gradient value of the source point cloud p, g q is the gradient value of the target point cloud q, and m is the number of matching point pairs. When iteratively optimizing the above formula (3), the goal of each iteration is to find the optimal (r opt ; t opt ), where (r opt ; t opt ) satisfies the following formula:

[0047]

[0048] S32. Use a non-linear optimization method to perform iterative optimization calculations on the optimization target, and when the preset iteration end condition is reached, obtain the refined inter-frame motion estimation based on the optimal transformation matrix output in the last iteration.

[0049] Specifically, to solve the constructed objective function, in this embodiment, the initial transformation matrix is defined as a vector of six parameters: ξ = (α, β, γ, a, b, c), then the initial transformation matrix can be linearly represented as:

[0050]

[0051] where T k is the transformation estimate of the last iteration, and currently the Gauss-Newton method J r T J r ξ = -J r T r Solve for the parameter ξ and apply the parameter ξ to T k to update T, where r is the residual and J r is the Jacobian matrix.

[0052] In one of the embodiments, the preset iteration end condition can be reaching the preset maximum number of iterations, etc., and in different embodiments, it can be flexibly adjusted according to the actual application scenario.

[0053] In the above embodiments, geometric and optical dual constraints are integrated, texture information is fully utilized, and it is proposed to perform computational solution on the texture image to obtain eigenvalues that are insensitive to light and have strong anti-interference ability to replace the unprocessed pixel intensities, making the system more adaptable to optical changes and the registration result more robust.

[0054] S4. Segment the data obtained through inter-frame motion estimation to obtain multiple data segments, and optimize the pose in each of the data segments; wherein, each of the data segments includes multiple image frames.

[0055] S5. For each data segment, select a key frame from the multiple image frames included in the data segment respectively, and perform joint optimization between segments in combination with each of the key frames and loop closure information.

[0056] Optionally, the extraction of the key frame needs to satisfy at least one of the following situations:

[0057] (1) There is at least one key frame in every N image frames to express the global information through the key frames.

[0058] (2) When the current image frame can be matched with the previous image frame, but the current image frame cannot be matched with the preset reference key frame, the previous image frame of the current image frame will be added to the preset key frame set to ensure the continuity of trajectory tracking.

[0059] (3) Although the current image frame can be matched with the previous image frame, and at the same time the current image frame can also be matched with the preset reference key frame, but the overlap rate between the current image frame and the preset reference key frame is not enough. At this time, the current image frame needs to be added to the preset key frame set to ensure that there is overlap between adjacent key frames.

[0060] In one embodiment, there will be a large pose error in the absolute pose estimation of the depth sensor with the accumulation of time. After the local optimization measures in step S4 are implemented, the pose information between segments does not have global consistency, and the cumulative error still exists. At this time, in order to overcome the above problems, in this embodiment, loop information and each of the key frames are used to perform joint optimization between segments. It should be noted that loop information is usually directly calculated based on images or features. In one embodiment, in order to obtain accurate loop information, the present application adopts an inter-frame matching method to match each pair of adjacent key frames. When the matching is successful and the overlap rate reaches a set threshold, a corresponding loop is formed.

[0061] In addition, since the key frames run through the entire tracking process and can fully reflect the global situation, in order to improve the efficiency of pose optimization, not all frames participate in the global optimization in this embodiment. Instead, one frame is selected from each data segment to represent the data segment. This image frame is collectively referred to as a key frame, and then global optimization is performed in combination with loop information. At this time, most of the cumulative errors can be quickly eliminated through global optimization.

[0062] In the above embodiment, based on the segmented multi-mode optimization strategy, problems can be modeled at different abstraction levels to achieve fast and accurate optimization.

[0063] S6. For each data segment, fix the poses of the key frames in the corresponding data segment respectively, and optimize the poses of other image frames in the data segment to obtain a globally consistent and smoothly transitioning motion trajectory map.

[0064] Specifically, through the optimization of the global pose graph, the update of the poses of the key frames has been completed. However, in order to obtain a globally consistent and smoothly transitioning motion trajectory, the poses within the current local range also need to be updated. Therefore, in this embodiment, the idea of layering is adopted, and not all image frames are optimized simultaneously. The poses of the key frames in each segment are fixed, and only the poses of other image frames within the segment are optimized.

[0065] S7. Combine the relative pose measured by the depth sensor and the absolute pose estimated from the motion trajectory map to construct a corresponding target optimization function.

[0066] S8. Incorporate a preset penalty factor into the target optimization function, and through iterative transformation estimation, eliminate the cumulative error generated with the accumulation of the number of scanned frames during inter-frame matching, and perform fusion and network construction.

[0067] E2 = ∑ i,j ρ(e 2 (p i ,p j ;∑ i,j,T i,j )); (6)

[0068] Among them, the estimated absolute pose is taken as a node, p i represents node i, p j represents node j; T i,j represents the relative pose between node i and node j, ∑ i,j represents the summation over all constraint pairs; e 2 (p i , p j ; ∑ i,j , T i,j ) = e(p i , p j ; T i,j ) T ∑ i,j -1 e(p i , p j ; T i,j ), e(p i , p j ; T i,j ) = T i,j - p i -1 p j ; ρ is the incorporated penalty factor.

[0069] In one embodiment,[[]] u = d 2 , where d is the surface diameter of the reconstructed object. Among them, considering that an appropriate penalty function can perform good calibration and screening without increasing additional computational cost, the Geman - mclure function in M - estimation is selected in the current embodiment, that is

[0070] Since the above formula (6) is difficult to optimize directly, the current hypothesis relationship l is assumed, and the target optimization function E2 is assumed to be:

[0071] E2 = ∑ i,j l(e 2 (p i , p j ; ∑ i,j , T i,j ) + ∑ i,j ψ(l); (7)

[0072] Among them, it is known that Minimizing the formula E2 and taking the partial derivative with respect to l can obtain In actual calculation, l is regarded as the confidence level. Since the constraints with smaller residuals have higher weights for the generated errors and are more credible, on the contrary, the constraints with larger residuals are less credible, so as to achieve verification and the purpose of elimination, and obtain a robust optimization effect. In addition, the selection of the parameter μ is also crucial. μ = d 2 , which represents the surface diameter of the reconstructed object and controls the range of the significant influence of the residual on the target. A larger μ makes the objective function smoother and allows more corresponding terms to participate in the optimization. As μ decreases, the objective function becomes sharper, more abnormal matches are eliminated, and the data participating in the optimization is more accurate.

[0073] To solve this non-linear squared error function problem, in the current embodiment, the transformation matrix is also transformed according to formula (5). Considering that in the pose graph, only a small number of nodes have direct edge connections, that is, the sparsity of the pose graph, and at the same time for numerical stability, the sparse BA algorithm is used to solve in the current embodiment. Sparse BA usually uses the LM method for optimization. LM adds a positive definite diagonal matrix on the basis of Gauss-Newton, that is, ξ is solved by (JrTJr + λI)ξ = -JrTr.

[0074] It should be noted that the effect after rapid optimization is as shown in Figure 4 (c), and then fusion and network construction are carried out, and the effect is as shown in Figure 4 (d).

[0075] In one embodiment, a three-dimensional scanning system applied to the online matching optimization method described above is also provided. The system includes:

[0076] An acquisition module, configured to acquire a pair of depth texture images in one-to-one correspondence, where the pair of depth texture images includes a depth image collected by a depth sensor and a texture image collected by a camera device;

[0077] An inter-frame motion estimation module, configured to adopt a step-by-step refinement strategy to perform feature matching on the pair of depth texture images corresponding to the current frame and the pair of depth texture images corresponding to the sample frame, so as to estimate the preliminary pose of the depth sensor, and optimize the estimated preliminary pose by combining geometric constraints and texture constraints to obtain a refined inter-frame motion estimation;

[0078] A multi-mode optimization module is used to segment the data obtained by inter-frame motion estimation to obtain multiple data segments, and optimize the poses in each of the data segments. Each of the data segments includes multiple image frames. For each data segment, a key frame is selected from the multiple image frames included in the data segment. Combining each of the key frames and loop closure information, joint optimization between segments is performed, the poses of the key frames in the corresponding data segments are fixed, and the poses of other image frames in the data segments are optimized to obtain a globally consistent and smoothly transitional motion trajectory map.

[0079] An accumulated error elimination module is used to construct a corresponding objective optimization function by combining the relative pose measured by the depth sensor and the absolute pose estimated from the motion trajectory map. It is also used to incorporate a preset penalty factor into the objective optimization function, and through iterative transformation estimation, eliminate the accumulated error generated as the number of scanned frames accumulates during inter-frame matching, and perform fusion and network construction.

[0080] In one embodiment, the depth sensor includes a projection module and a depth information acquisition module. The projection module is used to project white light or a structured light beam of a specific wavelength onto the surface of the object to be measured. The depth information acquisition module is used to acquire the depth information of the surface of the object to be measured when the projection module projects the structured light beam. The imaging device includes a texture information acquisition module, and the texture information acquisition module is used to acquire the texture information of the surface of the object to be measured when the imaging device projects a texture illumination beam onto the surface of the object to be measured.

[0081] Here, the structured light beam and the texture illumination beam are alternately projected. When a single projection of the structured light beam is completed, the depth information acquisition module completes the acquisition of the depth information of the surface of the object to be measured, and then the projection of the texture illumination beam is started, and the imaging device performs a single acquisition of the texture information of the surface of the object to be measured where the texture illumination beam is projected.

[0082] The above is a single cycle of the measurement process of the texture information and depth information of the surface of the object to be measured. When the above measurement process is repeated at a certain repetition frequency, the relative positions and relative angles between the imaging device, the depth sensor, and the object to be measured will change continuously, and thus the continuous measurement of the structure of the object to be measured can be completed.

[0083] In one embodiment, a computer-readable storage medium is also provided. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the steps of any one of the above online matching optimization methods are implemented.

[0084] In one embodiment, a three-dimensional scanning device for an online matching optimization method is further provided, which includes a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0085] Implementing an online matching optimization method and a three-dimensional scanning system combining geometry and texture according to the present invention, on the one hand, fuses double constraints of geometry and optics, makes full use of texture information, proposes to calculate and solve the texture image, and obtains eigenvalues that are insensitive to light and have strong anti-interference ability to replace the unprocessed pixel intensity, making the system more adaptable to optical changes and the registration result more robust. On the other hand, adopts a strategy of gradually refining, decomposes and simplifies complex problems, first estimates the pose through features, then refines the pose, and gradually obtains an accurate pose estimate. In addition, a penalty factor is added to the subsequently established optimization objective function, and without increasing additional computational costs, it can perform good verification and screening on different constraint pairs, ensuring the accuracy and stability of optimization, and adopting a segmented multi-mode optimization strategy, which can model problems at different levels of abstraction and achieve fast and accurate optimization.

[0086] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the purpose of the present invention and the scope protected by the claims, and these all belong to the protection scope of the present invention.

Claims

1. A method based on three-dimensional image segmented multi-mode optimization, characterized in that The method includes: Obtaining a corresponding pair of depth texture images, where the pair of depth texture images includes a depth image collected via a depth sensor and a texture image collected via a camera device. Adopting a strategy of successive refinement, performing feature matching between the depth texture image corresponding to the current frame and the depth texture image corresponding to the sample frame to estimate the initial pose of the depth sensor, and then optimizing the initial pose by combining geometric constraints and texture constraints to obtain a refined inter-frame motion estimation. Performing segmentation processing on the data obtained through inter-frame motion estimation to obtain multiple data segments, and performing pose optimization on each segment of data. Selecting a key frame from each of the data segments, and combining the key frame and loop closure information to optimize between segments. For each data segment, fixing the pose of the key frame in the corresponding data segment and optimizing the poses of other image frames within the data segment to obtain a globally consistent and smoothly transitioning motion trajectory map.

2. The method for three-dimensional image segmentation multi-mode optimization according to claim 1, wherein Selecting a key frame from each of the data segments includes: If the current image frame can represent global information, then the current image frame is used as a preset key frame. Or, If the current image frame can be matched with the previous image frame but cannot be matched with a preset reference key frame, then the previous image frame of the current image frame is used as the preset key frame. Or, If the current image frame can be matched with the previous image frame and can be matched with the preset reference key frame but the overlap rate is insufficient, then the current image frame is used as the preset key frame.

3. The method for three-dimensional image segmentation multi-mode optimization according to claim 1, characterized in that The loop closure information includes: Matching adjacent key frames. When the matching is successful and the overlap rate reaches a set threshold, a corresponding loop closure is determined, and the corresponding loop closure information is obtained.

4. A three-dimensional image scanning device, characterized in that, For implementing the method of three-dimensional image segmentation multi-mode optimization according to any one of claims 1 to 3, the device includes: An acquisition module for acquiring a pair of depth texture images. An inter-frame motion estimation module for performing inter-frame motion estimation on the pair of depth texture images. A segmentation multi-mode optimization module for: selecting a key frame from each of the data segments, and combining the key frame and loop closure information to optimize between segments. For each data segment, fixing the pose of the key frame in the corresponding data segment and optimizing the poses of other image frames within the data segment to obtain a globally consistent and smoothly transitioning motion trajectory map.

5. A computer-readable storage medium, characterized in that, Storing a computer program, which when executed by a processor, implements the steps of the method according to any one of claims 1 to 3.

6. A three-dimensional scanning system, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 3.

Images