3D Reconstruction Optimization Method, Device, Equipment and Storage Medium

Abstract

Embodiments of the present application disclose a three-dimensional reconstruction optimization method, apparatus, device, and storage medium. The technical solution provided by the embodiments of the present application performs a first reconstruction process on an image, including optimizing the internal parameters of a camera, to obtain first reconstruction information including a first camera internal parameter value when the number of key frame images in the image to be processed is greater than a first preset number and less than a second preset number, and continues to obtain the image to be processed for reconstruction when a first deviation value between the first camera internal parameter value and an initial camera internal parameter value is less than a first preset threshold, solving the problem of low three-dimensional reconstruction efficiency in the prior art, optimizing the three-dimensional reconstruction process, and improving the success rate of three-dimensional reconstruction.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of three-dimensional reconstruction technology, and particularly to a three-dimensional reconstruction optimization method, device, equipment and storage medium. Background Art

[0002] With the development of unmanned equipment and three-dimensional reconstruction technology, it has become increasingly common to perform three-dimensional reconstruction on the image information captured by unmanned equipment and apply the reconstructed data. During the three-dimensional reconstruction process, a three-dimensional reconstruction algorithm is usually used to calculate the corresponding three-dimensional information from a time series of two-dimensional images.

[0003] When using a three-dimensional reconstruction algorithm to perform three-dimensional reconstruction on the image information captured by unmanned equipment, the accuracy of the reconstructed data will be affected by various parameters involved in the operation, such as common camera internal parameters. Once the parameters involved in the operation are incorrect, it will lead to the failure of the final mapping in three-dimensional reconstruction. However, in the prior art, there is a lack of a reasonable and efficient optimization mechanism for three-dimensional reconstruction parameters, resulting in low three-dimensional reconstruction efficiency and the need for improvement. Summary of the Invention

[0004] The embodiments of the present application provide a three-dimensional reconstruction optimization method, device, equipment and storage medium, which solve the problem of low three-dimensional reconstruction efficiency in the prior art, optimize the three-dimensional reconstruction process, and improve the success rate of three-dimensional reconstruction.

[0005] In a first aspect, the embodiments of the present application provide a three-dimensional reconstruction optimization method, including:

[0006] Obtain the picture to be processed;

[0007] If the number of key frame pictures in the picture is greater than a first preset number and less than a second preset number, perform a first reconstruction process on the picture to obtain first reconstruction information, where the first reconstruction process includes an optimization process of camera internal parameters, and the first reconstruction information includes a first camera internal parameter value;

[0008] Determine a first deviation value between the first camera internal parameter value and an initial camera internal parameter value. If the first deviation value is less than a first preset threshold, continue to obtain the input picture to be processed.

[0009] In a second aspect, the embodiments of the present application provide a three-dimensional reconstruction optimization device, including a picture acquisition module, a first reconstruction module and a deviation processing module, where:

[0010] The picture acquisition module is used to obtain the picture to be processed;

[0011] The first reconstruction module is configured to perform a first reconstruction process on the picture to obtain first reconstruction information when the number of key-frame pictures in the picture is greater than a first preset number and less than a second preset number. The first reconstruction process includes optimizing the camera internal parameters, and the first reconstruction information includes a first camera internal parameter value.

[0012] The deviation processing module is configured to determine a first deviation value between the first camera internal parameter value and an initial camera internal parameter value. If the first deviation value is less than a first preset threshold, continue to obtain the input picture to be processed.

[0013] In a third aspect, an embodiment of the present application provides a three-dimensional reconstruction optimization device, including: a memory and one or more processors;

[0014] The memory is configured to store one or more programs;

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the three-dimensional reconstruction optimization method as described in the first aspect.

[0016] In a fourth aspect, an embodiment of the present application provides a storage medium containing computer-executable instructions, and the computer-executable instructions are used to execute the three-dimensional reconstruction optimization method as described in the first aspect when executed by a computer processor.

[0017] In the embodiment of the present application, when the number of key-frame pictures in the picture to be processed is greater than a first preset number and less than a second preset number, a first reconstruction process including optimizing the camera internal parameters is performed on the picture to obtain first reconstruction information including a first camera internal parameter value, and when the first deviation value between the first camera internal parameter value and the initial camera internal parameter value is less than a first preset threshold, continue to obtain the picture to be processed for reconstruction, solving the problem of low three-dimensional reconstruction efficiency in the prior art, optimizing the three-dimensional reconstruction process, and improving the success rate of three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flowchart of a three-dimensional reconstruction optimization method provided by an embodiment of the present application;

[0019] Figure 2 is a flowchart of another three-dimensional reconstruction optimization method provided by an embodiment of the present application;

[0020] Figure 3 is a flowchart of another three-dimensional reconstruction optimization method provided by an embodiment of the present application;

[0021] Figure 4 is a flowchart of another three-dimensional reconstruction optimization method provided by an embodiment of the present application;

[0022] Figure 5 is a flowchart of another 3D reconstruction optimization method provided by an embodiment of the present application;

[0023] Figure 6 is a flowchart of another 3D reconstruction optimization method provided by an embodiment of the present application;

[0024] Figure 7 is a schematic structural diagram of a pose solution device provided by an embodiment of the present application;

[0025] Figure 8 is a schematic structural diagram of a 3D reconstruction optimization device provided by an embodiment of the present application. Detailed implementation manners

[0026] In order to make the objectives, technical solutions, and advantages of the present application clearer, the following further describes the specific embodiments of the present application in detail with reference to the accompanying drawings. It can be understood that the specific embodiments described herein are only used to explain the present application, rather than limiting the present application. Additionally, it should be noted that, for the sake of description, only parts related to the present application are shown in the accompanying drawings rather than all the content. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. When the operations are completed, the process can be terminated, but there can also be additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0027] Figure 1 A flowchart of a 3D reconstruction optimization method provided by an embodiment of the present application is given. The 3D reconstruction optimization method provided by the embodiment of the present application can be executed by a pose solution device, and the pose solution device can be implemented in a hardware and / or software manner and integrated in a 3D reconstruction optimization device.

[0028] The following describes the case where the pose solution device executes the 3D reconstruction optimization method as an example. Refer to Figure 1 , and the 3D reconstruction optimization method includes:

[0029] S101: Obtain the picture to be processed.

[0030] Exemplarily, obtain the picture to be processed, and the picture to be processed is used for subsequent reconstruction processing. The picture to be processed provided in this embodiment is a picture after feature matching. For example, receive the original pictures taken by the unmanned device during movement, and perform feature matching on the received original pictures to obtain the feature matching results between the original pictures.

[0031] In a possible embodiment, before obtaining the picture to be processed, the 3D reconstruction optimization method provided in this embodiment further includes S1001 - S1003:

[0032] S1001: Extract features from the obtained original picture to obtain a feature extraction result.

[0033] Exemplarily, sequentially obtain the original pictures taken by an unmanned device (such as a drone) during movement, and extract features from the original pictures to obtain the feature extraction results of the original pictures. For example, based on the SIFT (Scale - invariant feature transform) algorithm, extract the feature descriptors in the original photo to obtain a feature extraction result including the feature descriptors.

[0034] S1002: According to the feature extraction result, perform feature matching on the original picture and the previously obtained original picture to obtain a feature matching result.

[0035] Exemplarily, according to the feature extraction result of the currently obtained original picture and the feature extraction result of the previously obtained original picture, perform feature matching on the currently obtained original picture and the previously obtained original picture to obtain the feature matching result of the currently obtained original picture. For example, determine the distance (such as the Euclidean distance, etc.) between the feature descriptors of the currently obtained original picture and the feature descriptors of the previously obtained original picture, and according to the distance between the feature descriptors, determine the original picture that matches the currently obtained original picture in the previously obtained original pictures to obtain the feature matching result.

[0036] In a possible embodiment, the previously obtained original picture used for feature matching with the currently obtained original picture can be an original picture that is geospatially or temporally close to the currently obtained original picture. For example, determine the geospatial location or time when the original picture was taken according to the GPS information or timestamp recorded in the original picture, and determine the original pictures taken previously within a set location range or time range as the original pictures for feature matching.

[0037] S1003: Determine the picture to be processed according to the feature matching result.

[0038] Exemplarily, after obtaining the feature matching result of the currently obtained original picture, use the original picture after feature matching as the picture to be processed (including the currently obtained original picture after feature matching and the previously obtained original picture after feature matching).

[0039] S102: If the number of key-frame pictures in the said picture is greater than the first preset number and less than the second preset number, then perform a first reconstruction process on the said picture to obtain first reconstruction information. The first reconstruction process includes optimizing the camera internal parameters, and the first reconstruction information includes the first camera internal parameter value.

[0040] Exemplarily, after obtaining the picture to be processed, determine the number of key-frame pictures in the obtained picture to be processed. Among them, the key positive picture is the picture in the picture to be processed that has undergone a reconstruction process (including the first to fourth reconstruction processes provided in this application). It can be understood that after the newly obtained picture to be processed undergoes a reconstruction process, it becomes a key-frame picture in the three-dimensional reconstruction optimization process of the next cycle.

[0041] Further, determine whether the number of key-frame pictures in the picture to be processed is greater than the first preset number and less than the second preset number. If so, perform a first reconstruction process on the picture to be processed to obtain first reconstruction information. Among them, the first reconstruction process includes optimizing the camera internal parameters, and the first reconstruction information includes the first camera internal parameter value.

[0042] In a possible embodiment, the first reconstruction process is performed based on the SFM (Structure From Motion) method. When performing the reconstruction process based on the SFM method, the parameters to be optimized can be set. For example, it is set to optimize the camera internal parameters. When using the picture to be processed to perform the reconstruction process and obtain the three-dimensional point cloud data, the camera internal parameters are optimized. Further, the first reconstruction process can be visual BA optimization, or visual BA (Bundle Adjustment) optimization with positioning parameter fusion processing (such as GPS constraint) added. This embodiment is described by taking the visual BA optimization with positioning parameter fusion processing as an example. It can be understood that after completing the first reconstruction process on the picture to be processed, the camera internal parameter value is optimized and updated to the first camera internal parameter value. If the camera internal parameter value is not initialized subsequently, the next reconstruction process will be performed based on the optimized and updated first camera internal parameter value.

[0043] S103: Determine the first deviation value between the first camera internal parameter value and the initial camera internal parameter value. If the first deviation value is less than the first preset threshold, then continue to obtain the input picture to be processed.

[0044] Exemplarily, after performing the first reconstruction process on the picture to be processed, determine the first deviation value of the optimized and updated first camera internal parameter value relative to the initial camera internal parameter value, and compare the first deviation value with the first preset threshold. When the first deviation value is less than the first preset threshold, then continue to obtain the input picture to be processed and perform the three-dimensional reconstruction optimization process of the next cycle.

[0045] If the first deviation value is not less than the first preset threshold, it is considered that the deviation degree of the optimized first camera internal parameter value relative to the initial camera internal parameter value is too large. If the first camera internal parameter value is maintained, there will be a situation of local optimum, which will affect the 3D reconstruction effect. Therefore, the camera internal parameters are initialized, that is, the camera internal parameter value is initialized to the initial camera internal parameter value. After completing the initialization process of the reconstruction processing thread, continue to obtain the pictures to be processed and perform the 3D reconstruction optimization process of the next cycle. In this embodiment, when the camera internal parameter value deviates from the normal value, the camera internal parameters are initialized to ensure the normal progress of subsequent 3D reconstruction and optimization.

[0046] As described above, when the number of key frame pictures in the pictures to be processed is greater than the first preset number and less than the second preset number, perform the first reconstruction process including optimizing the camera internal parameters on the pictures to obtain the first reconstruction information including the first camera internal parameter value. And when the first deviation value between the first camera internal parameter value and the initial camera internal parameter value is less than the first preset threshold, continue to obtain the pictures to be processed for reconstruction, solve the problem of low 3D reconstruction efficiency in the prior art, optimize the 3D reconstruction process, and improve the success rate of 3D reconstruction.

[0047] Based on the above embodiment, Figure 2 The flowchart of another 3D reconstruction optimization method provided by the embodiment of the present application is given. This 3D reconstruction optimization method is a concretization of the above 3D reconstruction optimization method. Refer to Figure 2 , this 3D reconstruction optimization method includes:

[0048] S201: Obtain the pictures to be processed.

[0049] S202: Determine whether the number of the pictures is greater than the third preset number. If so, initialize the reconstruction processing thread, and the reconstruction processing thread is used to perform the reconstruction process.

[0050] In this embodiment, the pictures to be processed are reconstructed by the reconstruction processing thread (such as the SFM thread). The reconstruction processing thread can optimize one or more of the spatial points, camera poses, and camera internal parameters according to requirements. It can be understood that before enabling the reconstruction processing thread, the reconstruction processing thread needs to be initialized, and in the initialized reconstruction processing thread, the camera internal parameter value is set to the preset initial camera internal parameter value.

[0051] Specifically, determine whether the number of the pictures to be processed is greater than the third set number, and when it is determined that the number of the pictures is greater than the third set number, initialize the reconstruction processing thread. After successfully initializing the reconstruction processing thread, when a new picture to be processed is received subsequently, if the reconstruction processing thread has been initialized previously, continue to the next step and determine the type of reconstruction process (the first to the first reconstruction process) according to the number of key frame pictures.

[0052] The third set quantity provided in this embodiment can be set as needed. For example, the third set quantity is set to 10, and the feature extraction thread is used to perform feature extraction on the received original image, and the feature extraction result is passed to the feature matching thread. The feature matching thread performs feature matching on the received original image with features, and passes the current original image and its feature matching result to the reconstruction processing thread. The reconstruction processing thread determines the images to be processed according to the received matching result, and when the number of images to be processed is greater than 10, initializes the reconstruction processing thread, and uses the initialized reconstruction processing thread to perform 3D reconstruction and parameter optimization.

[0053] S203: Determine whether the number of key frame images in the image is greater than a first preset quantity and less than a second preset quantity. If so, perform a first reconstruction process on the image to obtain first reconstruction information. The first reconstruction process includes an optimization process of the camera internal parameters. The first reconstruction information includes a first camera internal parameter value. The third preset quantity is less than the first preset quantity.

[0054] Specifically, when the reconstruction processing thread receives a new image to be processed, determine the number of images to be processed received in the reconstruction processing thread, and determine the number of key frame images in the images to be processed. Determine whether the number of key frame images in the image is greater than a first preset quantity and less than a second preset quantity. When the number of key frame images is greater than the first preset quantity and less than the second preset quantity, perform a first reconstruction process including an optimization process of the camera internal parameters on all images to be processed, and obtain first reconstruction information including the optimized first camera internal parameter value.

[0055] The third preset quantity provided in this embodiment is less than the first preset quantity, that is, the third preset quantity, the first preset quantity, and the second preset quantity increase in sequence. For example, if the third preset quantity, the first preset quantity, and the second preset quantity are set to 10, 30, and 50 respectively, then when the number of images to be processed received by the reconstruction processing thread is not greater than 10, the images to be processed are retained until the number of images to be processed is greater than 10, and then the reconstruction processing thread is initialized. Further, determine whether the number of key frame images in the images to be processed is greater than 30 and less than 50. If so, perform a first reconstruction process including an optimization process of the camera internal parameters on all received images to be processed, and obtain first reconstruction information including the first camera internal parameter value.

[0056] S204: Determine the first deviation value between the first camera internal parameter value and the initial camera internal parameter value. If the first deviation value is less than the first preset threshold, continue to obtain the input picture to be processed. If the first deviation value is not less than the first preset threshold, re-initialize the reconstruction processing thread.

[0057] Specifically, after completing the first reconstruction processing of the picture to be processed, determine the first deviation value between the optimized first camera internal parameter value and the initial camera internal parameter value, and compare the first deviation value with the set first preset threshold.

[0058] If the first deviation value is less than the first preset threshold, continue to obtain the input picture to be processed (by the feature matching thread) and perform the three-dimensional reconstruction optimization process for the next cycle. If the first deviation value is not less than the first preset value, at this time, the optimized first camera internal parameter value deviates too much from the initial camera internal parameter value, and there may be a situation where the local optimum affects the three-dimensional reconstruction optimization effect. Then, re-initialize the reconstruction processing thread, reset the camera internal parameter value to the initial camera internal parameter value, and ensure the three-dimensional reconstruction optimization effect. After completing the initialization processing of the reconstruction processing thread, continue to obtain the picture to be processed and perform the three-dimensional reconstruction optimization process for the next cycle.

[0059] In a possible embodiment, the first deviation value provided in this embodiment is determined based on the following formula:

[0060]

[0061] Where K' is the first camera internal parameter value, f' is the first camera focal length, cx' is the abscissa of the first camera inner axis, cy' is the ordinate of the first camera inner axis, K is the initial camera internal parameter value, f is the initial camera focal length, cx is the abscissa of the initial camera inner axis, and cy is the ordinate of the initial camera inner axis.

[0062] Furthermore, the first preset threshold provided in this embodiment is determined based on the following formula:

[0063]

[0064] Where Ke1 is the first preset threshold, Δf1 is the set first focal length deviation threshold, Δcx1 is the set first abscissa deviation threshold of the inner axis, and Δcy1 is the set first ordinate deviation threshold of the inner axis.

[0065] When |K' - K| < |Ke1|, it is determined that the first deviation value is less than the first preset threshold, and then continue to obtain the input picture to be processed. Otherwise, it is considered that the first camera internal parameter value deviates too much from the initial camera internal parameter value, and it is necessary to re-initialize the reconstruction processing thread.

[0066] As described above, when the number of key-frame images in the image to be processed is greater than the first preset number and less than the second preset number, a first reconstruction process including optimizing the camera internal parameters is performed on the image to obtain first reconstruction information including the first camera internal parameter value. And when the first deviation value between the first camera internal parameter value and the initial camera internal parameter value is less than the first preset threshold, continue to obtain the image to be processed for reconstruction, solving the problem of low 3D reconstruction efficiency in the prior art, optimizing the 3D reconstruction process, and improving the success rate of 3D reconstruction. And when the first deviation value is not less than the first preset threshold, re-initialize the reconstruction process thread. When it is detected that the camera internal parameters of the reconstruction process thread deviate from the normal value, initialize the reconstruction process thread in time, reducing the subsequent situation of failed reconstruction optimization and ensuring the effect of 3D reconstruction optimization.

[0067] Based on the above embodiments, Figure 3 The flowchart of another 3D reconstruction optimization method provided by the embodiment of the present application is given. This 3D reconstruction optimization method is a concretization of the above 3D reconstruction optimization method. Refer to Figure 3 and this 3D reconstruction optimization method includes:

[0068] S301: Obtain the image to be processed.

[0069] S302: Determine whether the number of the images is greater than a third preset number. If so, initialize a reconstruction process thread, and the reconstruction process thread is used to perform a reconstruction process, and the third preset number is less than the first preset number.

[0070] S303: Determine whether the number of key-frame images in the images is greater than the first preset number and less than the second preset number.

[0071] If the number of key-frame images in the image is not greater than the first preset number, jump to step S304 to perform a second reconstruction process on the image. If the number of key-frame images is greater than the first preset number and less than the second preset number, jump to step S305 to perform a first reconstruction process on the image.

[0072] S304: If the number of the key-frame images is not greater than the first preset number, perform a second reconstruction process on the image, and the second reconstruction process includes optimizing the camera pose.

[0073] Optionally, the second reconstruction process provided in this embodiment includes the optimization process of the camera pose and / or space points, but does not include the optimization process of the camera internal parameters. Exemplarily, the second reconstruction process provided in this embodiment is based on the SFM method. When performing the second reconstruction process, the parameters to be optimized can be set. For example, the parameter to be optimized is set as the camera pose. While reconstructing the processed pictures to obtain the three-dimensional point cloud data, the camera pose is optimized. It can be understood that since the optimization process of the camera internal parameters is not selected, the current camera internal parameter value will be maintained after reconstructing the processed pictures. Further, the first reconstruction process can be visual BA optimization, or visual BA optimization with the addition of positioning parameter fusion processing (such as GPS constraint). This embodiment will be described by taking the visual BA optimization with the addition of positioning parameter fusion processing as an example. It can be understood that after completing the second reconstruction process of the processed pictures, the current camera internal parameter value is not optimized and updated, and the next reconstruction process will be based on the current camera internal parameter value.

[0074] Specifically, when the number of key-frame pictures is not greater than the first preset number, the second reconstruction process is performed on all the processed pictures. The second reconstruction process includes the optimization process of the camera pose, but does not include the optimization process of the camera internal parameters. After completing the second reconstruction process of the pictures, continue to obtain the processed pictures and perform the three-dimensional reconstruction optimization process of the next cycle.

[0075] S305: If the number of key-frame pictures in the picture is greater than the first preset number and less than the second preset number, perform the first reconstruction process on the picture to obtain the first reconstruction information. The first reconstruction process includes the optimization process of the camera internal parameters, and the first reconstruction information includes the first camera internal parameter value.

[0076] S306: Determine the first deviation value between the first camera internal parameter value and the initial camera internal parameter value. If the first deviation value is less than the first preset threshold, continue to obtain the input processed pictures.

[0077] As described above, when the number of key-frame images in the image to be processed is greater than the first preset number and less than the second preset number, a first reconstruction process including optimizing the camera internal parameters is performed on the image to obtain first reconstruction information including a first camera internal parameter value. And when the first deviation value between the first camera internal parameter value and the initial camera internal parameter value is less than the first preset threshold, continue to obtain the image to be processed for reconstruction, solving the problem of low 3D reconstruction efficiency in the prior art, optimizing the 3D reconstruction process, and improving the success rate of 3D reconstruction. And when the number of key-frame images is not greater than the first preset number, since the current number of key-frame images is small and there is no need to optimize the camera internal parameters, a second reconstruction process that does not include optimizing the camera internal parameters is performed on the image to ensure the effect and efficiency of 3D reconstruction optimization.

[0078] Based on the above embodiments, Figure 4 a flowchart of another 3D reconstruction optimization method provided by an embodiment of the present application is given. This 3D reconstruction optimization method is a concretization of the above 3D reconstruction optimization method. Refer to Figure 4 and this 3D reconstruction optimization method includes:

[0079] S401: Obtain the image to be processed.

[0080] S402: Determine whether the number of the images is greater than a third preset number. If so, initialize a reconstruction processing thread for performing the reconstruction process, where the third preset number is less than the first preset number.

[0081] S403: Determine whether the number of key-frame images in the images is greater than the first preset number and less than the second preset number.

[0082] If the number of key-frame images in the images is greater than the first preset number and less than the second preset number, jump to step S404 to perform a first reconstruction process on the images. If the number of key-frame images is not less than the second preset number, jump to step S406 to perform a third reconstruction process on the images.

[0083] S404: If the number of key-frame images in the images is greater than the first preset number and less than the second preset number, perform a first reconstruction process on the images to obtain first reconstruction information. The first reconstruction process includes optimizing the camera internal parameters, and the first reconstruction information includes a first camera internal parameter value.

[0084] S405: Determine the first deviation value between the first camera internal parameter value and the initial camera internal parameter value. If the first deviation value is less than the first preset threshold, continue to obtain the input image to be processed.

[0085] S406: If the number of the key-frame pictures is not less than the second preset number, perform third reconstruction processing on the pictures to obtain third reconstruction information. The third reconstruction processing includes optimization processing of the camera internal parameters, and the third reconstruction information includes a third camera internal parameter value.

[0086] The third reconstruction processing provided in this embodiment includes optimization processing of the camera internal parameters. The third reconstruction information obtained by the third reconstruction processing includes a third camera internal parameter value. Optionally, the third reconstruction processing provided in this embodiment includes optimization processing of the camera pose and / or spatial points, and includes optimization processing of the camera internal parameters. Exemplarily, the third reconstruction processing provided in this embodiment is based on the SFM method. When performing the third reconstruction processing, parameters to be optimized can be set. For example, set the parameters to be optimized as the camera pose and the camera internal parameters. When using the pictures to be processed for reconstruction processing to obtain three-dimensional point cloud data, optimize the camera pose and the camera internal parameters to obtain optimized third reconstruction information. The third reconstruction information includes the optimized camera pose and the third camera internal parameter value. In the reconstruction processing of the next cycle, it will be based on the third camera internal parameter value. Further, the third reconstruction processing can be visual BA optimization, or visual BA optimization with positioning parameter fusion processing (such as GPS constraint) added. This embodiment takes visual BA optimization with positioning parameter fusion processing added as an example for description. It can be understood that after the third reconstruction processing of the pictures to be processed is completed, the camera internal parameter value is optimized and updated to the third camera internal parameter value. If the camera internal parameter value is not initialized subsequently, the next reconstruction processing will be based on the optimized and updated third camera internal parameter value.

[0087] Specifically, when the number of the key-frame pictures is not less than the second preset number, perform third reconstruction processing including optimization processing of the camera internal parameters on all the pictures to be processed to obtain third reconstruction information including a third camera internal parameter value. After the third reconstruction processing of the pictures is completed, continue to obtain the pictures to be processed and perform the three-dimensional reconstruction optimization process of the next cycle.

[0088] As described above, when the number of the key-frame pictures in the pictures to be processed is greater than the first preset number and less than the second preset number, perform first reconstruction processing including optimization processing of the camera internal parameters on the pictures to obtain first reconstruction information including a first camera internal parameter value. And when the first deviation value between the first camera internal parameter value and the initial camera internal parameter value is less than the first preset threshold, continue to obtain the pictures to be processed for reconstruction processing, solving the problem of low three-dimensional reconstruction efficiency in the prior art, optimizing the three-dimensional reconstruction process, and improving the success rate of three-dimensional reconstruction. At the same time, when the number of the key-frame pictures is not less than the second preset number, perform third reconstruction processing including optimization processing of the camera internal parameters on the pictures, and continue to optimize the camera internal parameters of the reconstruction processing thread to improve the quality of three-dimensional reconstruction optimization.

[0089] Based on the above embodiments, Figure 5 a flowchart of another 3D reconstruction optimization method provided by an embodiment of the present application is given. This 3D reconstruction optimization method is a concretization of the above 3D reconstruction optimization method. Refer to Figure 5 , this 3D reconstruction optimization method includes:

[0090] S501: Obtain the picture to be processed.

[0091] S502: Determine whether the number of the pictures is greater than a third preset number. If so, initialize a reconstruction processing thread for performing reconstruction processing, and the third preset number is less than the first preset number.

[0092] S503: Determine whether the number of key frame pictures in the pictures is greater than the first preset number and less than the second preset number.

[0093] If the number of key frame pictures in the pictures is greater than the first preset number and less than the second preset number, jump to step S504 to perform a first reconstruction process on the pictures. If the number of key frame pictures is not less than the second preset number, jump to step S506 to perform a third reconstruction process on the pictures.

[0094] S504: If the number of key frame pictures in the pictures is greater than the first preset number and less than the second preset number, perform a first reconstruction process on the pictures to obtain first reconstruction information. The first reconstruction process includes optimizing the camera internal parameters, and the first reconstruction information includes a first camera internal parameter value.

[0095] S505: Determine a first deviation value between the first camera internal parameter value and an initial camera internal parameter value. If the first deviation value is less than a first preset threshold, continue to obtain the input pictures to be processed.

[0096] S506: Obtain a preset identifier.

[0097] In this embodiment, the convergence situation of the camera internal parameters is recorded through a preset identifier, that is, the preset identifier is used to indicate the convergence situation of the camera internal parameters. Specifically, when the preset identifier is a first identifier (for example, the preset identifier is "1" or "true"), it is determined that the camera internal parameter value tends to be stable, that is, the camera internal parameters converge. When the preset identifier is a second identifier (for example, the preset identifier is "0" or "false"), it is determined that the camera internal parameter value does not tend to be stable, that is, the camera internal parameters do not converge. And the default initial result of the preset identifier is the second identifier.

[0098] Specifically, when the number of key-frame pictures is not less than the second preset number, obtain a preset identifier and judge the result of the preset identifier. If the preset identifier is the first identifier, jump to step S507 to perform a fourth reconstruction process on the pictures. If the preset identifier is the second identifier, jump to step S508 to perform a third reconstruction process on the pictures.

[0099] S507: If the number of the key-frame pictures is not less than the second preset number, and the preset identifier is the first identifier, then perform a fourth reconstruction process on the pictures. The fourth reconstruction process includes an optimization process of the camera pose and a positioning parameter fusion process.

[0100] The fourth reconstruction process provided in this embodiment does not include an optimization process of the camera internal parameters. Optionally, the fourth reconstruction process provided in this embodiment includes an optimization process of the camera pose and / or spatial points, and a positioning parameter fusion process, and does not include an optimization process of the camera internal parameters. Exemplarily, the fourth reconstruction process provided in this embodiment is based on the SFM method. When performing the fourth reconstruction process, the parameters to be optimized can be set. For example, the parameters to be optimized are set as the camera pose and spatial points. When using the pictures to be processed for reconstruction to obtain three-dimensional point cloud data, the camera pose and spatial points are optimized. Further, the fourth reconstruction process can be a visual BA optimization with a positioning parameter fusion process (such as GPS constraint) added. This embodiment describes it by taking the visual BA optimization with a positioning parameter fusion process added as an example. It can be understood that at this time, the preset identifier is the first identifier, and it can be considered that the current camera internal parameters are in a converged state and there is no need to optimize the camera internal parameters anymore. When obtaining new pictures to be processed next time, the reconstruction process can be performed based on the camera internal parameter values in the converged state.

[0101] Specifically, when the number of key-frame pictures is not less than the second preset number and the preset identifier is the first identifier, perform a fourth reconstruction process on all the pictures to be processed, including an optimization process of the camera pose and a positioning parameter fusion process. After completing the fourth reconstruction process of the pictures, continue to obtain the pictures to be processed and perform the three-dimensional reconstruction optimization process of the next cycle.

[0102] S508: If the number of the key-frame pictures is not less than the second preset number, and the preset identifier is the second identifier, then perform a third reconstruction process on the pictures to obtain third reconstruction information. The third reconstruction process includes an optimization process of the camera internal parameters, and the third reconstruction information includes a third camera internal parameter value.

[0103] Specifically, when the number of key-frame images is not less than the second preset number and the preset identifier is the second identifier, and at this time the camera internal parameters are not in a converged state, a third reconstruction process including optimizing the camera internal parameters is performed on all images to be processed, and third reconstruction information including a third camera internal parameter value is obtained. After completing the third reconstruction process of the images, continue to obtain the images to be processed and perform the three-dimensional reconstruction optimization process of the next cycle.

[0104] Optionally, the first to fourth reconstruction processes provided in this embodiment may be the same reconstruction process or different reconstruction processes. For example, the reconstruction process may be pure-vision BA optimization or vision BA optimization with positioning parameter fusion processing added, and the parameters to be optimized in the reconstruction process may be the same or different. The optimizable parameters include camera pose, spatial points, camera internal parameters, and camera external parameters. In this embodiment, the first reconstruction process and the third reconstruction process are the same reconstruction process, both of which are vision BA optimization with positioning parameter fusion processing added, and the optimized parameters both include camera pose and camera internal parameters. The second reconstruction process is pure-vision BA optimization and does not optimize the camera internal parameters. The fourth reconstruction process is vision BA optimization with positioning parameter fusion processing added and does not optimize the camera internal parameters.

[0105] S509: Determine a second deviation value between the third camera internal parameter value and the initial camera internal parameter value. If the second deviation value is less than a second preset threshold, set the preset identifier to the first identifier, where the initial result of the preset identifier is the second identifier, and the second preset threshold is less than the first preset threshold.

[0106] Specifically, after obtaining the third camera internal parameter value through the third reconstruction of the images, calculate the second deviation value of the third camera internal parameter value relative to the initial camera internal parameter value. Compare the second deviation value with the preset second preset threshold. If the second deviation value is less than the second preset threshold, it is considered that after multiple optimization processes, the camera internal parameter value has tended to be stable, the camera internal parameters have converged, and the preset identifier is set to the first identifier. When receiving new images to be processed next time, a fourth reconstruction process that does not include optimizing the camera internal parameters will be performed on the images. When the second deviation value is not less than the second preset threshold, it is considered that the camera internal parameters have not converged, and the preset identifier is kept as the second identifier. When receiving new images to be processed next time, a third reconstruction process including optimizing the camera internal parameters will be performed on the images.

[0107] The initial result of the preset identifier is the second identifier, and the second preset threshold is less than the first preset threshold. In a possible embodiment, the second deviation value provided in this embodiment is determined based on the following formula:

[0108]

[0109] Among them, K' is the internal parameter value of the third camera, f' is the focal length of the third camera, cx' is the abscissa of the internal axis of the third camera, cy' is the ordinate of the internal axis of the third camera, K is the initial camera internal parameter value, f is the initial camera focal length, cx is the abscissa of the internal axis of the initial camera, and cy is the ordinate of the internal axis of the initial camera.

[0110] Furthermore, the second preset threshold provided in this embodiment is determined based on the following formula:

[0111]

[0112] Among them, Ke2 is the second preset threshold, Δf2 is the set second focal length deviation threshold, Δcx2 is the set second abscissa deviation threshold of the internal axis, and Δcy2 is the set second ordinate deviation threshold of the internal axis.

[0113] When |K' - K| < |Ke2| and it is determined that the second deviation value is less than the second preset threshold, the preset identifier is set to the first identifier.

[0114] As described above, when the number of key frame pictures in the picture to be processed is greater than the first preset number and less than the second preset number, a first reconstruction process including optimizing the camera internal parameters is performed on the picture to obtain first reconstruction information including the first camera internal parameter value. And when the first deviation value between the first camera internal parameter value and the initial camera internal parameter value is less than the first preset threshold, the picture to be processed is continuously obtained for reconstruction processing, solving the problem of low 3D reconstruction efficiency in the prior art, optimizing the 3D reconstruction process, and improving the success rate of 3D reconstruction. And the convergence state of the camera internal parameters is recorded through a preset identifier. When the camera internal parameters converge, in subsequent reconstruction processing, the camera internal parameters do not need to be optimized again, improving the 3D reconstruction optimization efficiency. When the camera internal parameters do not converge, in subsequent reconstruction processing, the camera internal parameters are continuously optimized to improve the 3D reconstruction optimization quality.

[0115] Based on the above embodiment, Figure 6 A flowchart of another 3D reconstruction optimization method provided by the embodiment of the present application is given. This 3D reconstruction optimization method is a specific implementation of the above 3D reconstruction optimization method. Refer to Figure 6 , this 3D reconstruction optimization method includes:

[0116] S601: Extract features from the obtained original picture to obtain a feature extraction result.

[0117] S602: According to the feature extraction result, perform feature matching on the original picture and the previously obtained original pictures to obtain a feature matching result. According to the feature matching result, determine the picture to be processed.

[0118] The feature extraction thread obtains the original image captured by the unmanned device, extracts features from the original image, and the feature matching thread matches the original image according to the feature extraction result, determines the image to be processed according to the feature matching result, and sends the image to be processed to the reconstruction processing thread.

[0119] S603: Obtain the image to be processed.

[0120] The reconstruction processing thread obtains the image to be processed provided by the feature matching thread and performs subsequent 3D reconstruction optimization based on the image to be processed.

[0121] S604: Determine whether the number of the images is greater than a third preset number.

[0122] The reconstruction processing thread checks whether the number of images in the input queue is greater than a third preset number (for example, 10). If the number of images is greater than the third preset number, it jumps to step S605; otherwise, it returns to step S603 to continue obtaining the image to be processed provided by the feature matching thread.

[0123] S605: Determine whether the reconstruction processing thread has been initialized.

[0124] Judge whether the reconstruction processing thread has completed initialization. If it has completed initialization, it jumps to step S607; otherwise, it jumps to step S606.

[0125] S606: Initialize the reconstruction processing thread.

[0126] Initialize the reconstruction processing thread to set the camera internal parameter value in the reconstruction processing thread to the initial camera internal parameter value.

[0127] S607: Determine whether the number of key frame images in the images is greater than a first preset number.

[0128] Determine whether the number of key frames in the image to be processed is greater than a first preset number (for example, 30). If the number of key frame images is greater than the third preset number, it jumps to step S609; otherwise, it jumps to step S608.

[0129] S608: Perform a second reconstruction process on the images. The second reconstruction process includes optimizing the camera pose. After completing the second reconstruction process on all the images to be processed, return to step S603.

[0130] S609: Determine whether the number of key frame images in the images is greater than a second preset number.

[0131] Determine whether the number of key frames in the image to be processed is greater than a second preset number (for example, 50). If the number of key frame images is greater than the second preset number, jump to step S613; otherwise, jump to step S610.

[0132] S610: Perform a first reconstruction process on the image to obtain first reconstruction information. The first reconstruction process includes optimizing the camera internal parameters, and the first reconstruction information includes the first camera internal parameter value.

[0133] S611: Determine the first deviation value between the first camera internal parameter value and the initial camera internal parameter value, and determine whether the first deviation value is less than a first preset threshold.

[0134] After completing the first reconstruction process on all images to be processed, calculate the first deviation value between the first camera internal parameter value and the initial camera internal parameter value, and determine whether the first deviation value is less than the first preset threshold. If the first deviation value is less than the first preset threshold, return to step S603; otherwise, jump to step S612.

[0135] S612: Re-initialize the reconstruction processing thread.

[0136] When the first deviation value is not less than the first preset threshold, it is necessary to re-initialize the reconstruction processing thread, and after completing the initialization of the reconstruction thread, return to step S603.

[0137] S613: Determine whether the preset identifier is the first identifier.

[0138] Judge whether the preset identifier is the first identifier. If so, jump to step S617; otherwise, jump to step S614.

[0139] S614: Perform a third reconstruction process on the image to obtain third reconstruction information. The third reconstruction process includes optimizing the camera internal parameters, and the third reconstruction information includes the third camera internal parameter value.

[0140] S615: Determine the second deviation value between the third camera internal parameter value and the initial camera internal parameter value, and determine whether the second deviation value is less than a second preset threshold.

[0141] After completing the third reconstruction process on all images to be processed, calculate the second deviation value between the third camera internal parameter value and the initial camera internal parameter value, and determine whether the second deviation value is less than the second preset threshold. If the second deviation value is less than the first preset threshold, jump to step S616; otherwise, return to step S603.

[0142] S616: Set the preset identifier to the first identifier.

[0143] Change the preset identifier from the second identifier to the first identifier.

[0144] S617: Perform a fourth reconstruction process on the said picture. The fourth reconstruction process includes an optimization process for the camera pose and a positioning parameter fusion process. After completing the fourth reconstruction process on all pictures to be processed, return to step S603 until all original pictures have participated in the 3D reconstruction optimization.

[0145] As described above, when the number of pictures to be processed reaches the third preset number, initialize the reconstruction processing thread. If the number of key frame pictures in the pictures is less than or equal to the first preset number, perform a second reconstruction process on all pictures without optimizing the camera internal parameters. If the number of key frame pictures in the pictures is greater than the first preset number and less than or equal to the second preset number, perform a first reconstruction process on all pictures with optimized camera internal parameters, and determine whether the camera internal parameters are out of control according to the optimized first camera internal parameter value (at this time, the first deviation value is greater than or equal to the first preset threshold). If the camera internal parameters are out of control, re-initialize the reconstruction processing thread. If the number of key frame pictures in the pictures is greater than the second preset number and the preset flag is the second flag, perform a third reconstruction process on all pictures with optimized camera internal parameters, and determine whether the camera internal parameters converge according to the optimized third camera internal parameter value (at this time, the second deviation value is less than the second preset threshold). If the camera internal parameters converge, set the preset flag to the first flag. If the number of key frame pictures in the pictures is greater than the second preset number and the preset flag is the first flag, perform a third reconstruction process on all pictures with supplementary optimized camera internal parameters, which solves the problem of low 3D reconstruction efficiency in the prior art, optimizes the 3D reconstruction process, and improves the success rate of 3D reconstruction.

[0146] Figure 7 The structural schematic diagram of a pose calculation device provided by an embodiment of the present application is given. Refer to Figure 7 . The pose calculation device includes a picture acquisition module 31, a first reconstruction module 32, and a deviation processing module 33.

[0147] Among them, the picture acquisition module 31 is used to acquire pictures to be processed; the first reconstruction module 32 is used to perform a first reconstruction process on the pictures to obtain first reconstruction information when the number of key frame pictures in the pictures is greater than the first preset number and less than the second preset number. The first reconstruction process includes an optimization process for the camera internal parameters, and the first reconstruction information includes a first camera internal parameter value; the deviation processing module 33 is used to determine a first deviation value between the first camera internal parameter value and the initial camera internal parameter value. If the first deviation value is less than the first preset threshold, continue to acquire the input pictures to be processed.

[0148] As described above, when the number of key-frame images in the image to be processed is greater than the first preset number and less than the second preset number, a first reconstruction process including optimizing the camera internal parameters is performed on the image to obtain first reconstruction information including a first camera internal parameter value. And when the first deviation value between the first camera internal parameter value and the initial camera internal parameter value is less than the first preset threshold, the image to be processed is continuously obtained for reconstruction processing, solving the problem of low 3D reconstruction efficiency in the prior art, optimizing the 3D reconstruction process, and improving the success rate of 3D reconstruction.

[0149] In a possible embodiment, the device further includes an initialization management module, configured to determine whether the number of the images is greater than a third preset number after the image acquisition module 31 acquires the image to be processed. If so, initialize a reconstruction processing thread, and the reconstruction processing thread is used to perform reconstruction processing, and the third preset number is less than the first preset number.

[0150] In a possible embodiment, the initialization management module is further configured to re-initialize the reconstruction processing thread when the first deviation value is not less than the first preset threshold.

[0151] In a possible embodiment, the device further includes a second reconstruction module, configured to perform a second reconstruction process on the image when the number of the key-frame images is not greater than the first preset number, and the second reconstruction process includes optimizing the camera pose.

[0152] In a possible embodiment, the device further includes a third reconstruction module, configured to perform a third reconstruction process on the image to obtain third reconstruction information when the number of the key-frame images is not less than the second preset number, and the third reconstruction process includes optimizing the camera internal parameters, and the third reconstruction information includes a third camera internal parameter value.

[0153] In a possible embodiment, the device further includes an identification acquisition module, configured to acquire a preset identification. The device further includes a fourth reconstruction module, configured to perform a fourth reconstruction process on the image when the preset identification is the first identification, and the fourth reconstruction process includes optimizing the camera pose and fusing positioning parameters. The preset identification is used to indicate the convergence condition of the camera internal parameters. The device further includes an identification update module, configured to determine a second deviation value between the third camera internal parameter value and the initial camera internal parameter value. If the second deviation value is less than a second preset threshold, set the preset identification to the first identification, where the initial result of the preset identification is the second identification, and the second preset threshold is less than the first preset threshold.

[0154] In a possible embodiment, the second deviation value is determined based on the following formula:

[0155]

[0156] Among them, K' is the internal parameter value of the third camera, f' is the focal length of the third camera, cx' is the abscissa of the inner axis of the third camera, cy' is the ordinate of the inner axis of the third camera, K is the initial camera internal parameter value, f is the initial camera focal length, cx is the abscissa of the inner axis of the initial camera, and cy is the ordinate of the inner axis of the initial camera;

[0157] The second preset threshold is determined based on the following formula:

[0158]

[0159] Among them, Ke2 is the second preset threshold, Δf2 is the set second focal length deviation threshold, Δcx2 is the set second abscissa deviation threshold of the inner axis, and Δcy2 is the set second ordinate deviation threshold of the inner axis.

[0160] In a possible embodiment, the first deviation value is determined based on the following formula:

[0161]

[0162] Among them, K' is the internal parameter value of the first camera, f' is the focal length of the first camera, cx' is the abscissa of the inner axis of the first camera, cy' is the ordinate of the inner axis of the first camera, K is the initial camera internal parameter value, f is the initial camera focal length, cx is the abscissa of the inner axis of the initial camera, and cy is the ordinate of the inner axis of the initial camera;

[0163] The first preset threshold is determined based on the following formula:

[0164]

[0165] Among them, Ke1 is the first preset threshold, Δf1 is the set first focal length deviation threshold, Δcx1 is the set first abscissa deviation threshold of the inner axis, and Δcy1 is the set first ordinate deviation threshold of the inner axis.

[0166] The embodiment of the present application also provides a three-dimensional reconstruction optimization device, and this three-dimensional reconstruction optimization device can integrate the pose solution device provided by the embodiment of the present application. Figure 8 It is a schematic structural diagram of a three-dimensional reconstruction optimization device provided by the embodiment of the present application. Refer to Figure 8, the 3D reconstruction optimization device includes: an input device 43, an output device 44, a memory 42, and one or more processors 41; the memory 42 is used to store one or more programs; when the one or more programs are executed by the one or more processors 41, the one or more processors 41 implement the 3D reconstruction optimization method provided in the above embodiments. The input device 43, the output device 44, the memory 42, and the processor 41 may be connected through a bus or other means, Figure 8 Taking connection through a bus as an example.

[0167] The memory 42, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the 3D reconstruction optimization method described in any embodiment of the present application (for example, the picture acquisition module 31, the first reconstruction module 32, and the deviation processing module 33 in the pose calculation device). The memory 42 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the device. In addition, the memory 42 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 42 may further include a memory remotely set relative to the processor 41, and these remote memories may be connected to the device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and their combinations.

[0168] The input device 43 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function control of the device. The output device 44 may include a display device such as a display screen.

[0169] The processor 41 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 42, that is, implements the above 3D reconstruction optimization method.

[0170] The above-provided pose calculation device, device, and computer can be used to execute the 3D reconstruction optimization method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0171] An embodiment of the present application further provides a storage medium containing computer-executable instructions, and the computer-executable instructions are used to execute the three-dimensional reconstruction optimization method provided in the above embodiment when executed by a computer processor. The three-dimensional reconstruction optimization method includes: obtaining an image to be processed; if the number of key frame images in the image is greater than a first preset number and less than a second preset number, performing a first reconstruction process on the image to obtain first reconstruction information, the first reconstruction process includes optimizing the internal parameters of the camera, and the first reconstruction information includes a first internal parameter value of the camera; determining a first deviation value between the first internal parameter value of the camera and an initial internal parameter value of the camera, and if the first deviation value is less than a first preset threshold, continue to obtain the input image to be processed.

[0172] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media such as CD-ROMs, floppy disks or tape devices; computer system memories or random access memories such as DRAM, DDRRAM, SRAM, EDORAM, Rambus RAM, etc.; non-volatile memories such as flash memories, magnetic media (such as hard disks or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memories or combinations thereof. Additionally, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system that is connected to the first computer system via a network (such as the Internet). The second computer system may provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (such as in different computer systems connected via a network). The storage medium may store program instructions (such as specifically implemented as a computer program) executable by one or more processors.

[0173] Of course, for a storage medium containing computer-executable instructions provided in an embodiment of the present application, the computer-executable instructions are not limited to the three-dimensional reconstruction optimization method as described above, and may also execute related operations in the three-dimensional reconstruction optimization method provided in any embodiment of the present application.

[0174] The pose calculation device, device, and storage medium provided in the above embodiment can execute the three-dimensional reconstruction optimization method provided in any embodiment of the present application. For technical details not described in detail in the above embodiment, reference may be made to the three-dimensional reconstruction optimization method provided in any embodiment of the present application.

[0175] The above is only the preferred embodiment of the present application and the technical principles applied. The present application is not limited to the specific embodiments described herein. Various obvious changes, re-adjustments, and substitutions that can be made by those skilled in the art will not depart from the protection scope of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments. Without departing from the concept of the present application, it may also include more other equivalent embodiments, and the scope of the present application is determined by the scope of the claims.

Claims

1. A three-dimensional reconstruction optimization method, characterized in that, Including: Obtain the picture to be processed; If the number of key frame pictures in the picture is greater than the first preset number and less than the second preset number, perform a first reconstruction process on the picture to obtain first reconstruction information. The first reconstruction process includes optimizing the camera internal parameters. The first reconstruction information includes the first camera internal parameter value, and the key frame picture is a picture that has undergone a reconstruction process; Determine the first deviation value between the first camera internal parameter value and the initial camera internal parameter value. If the first deviation value is less than the first preset threshold, continue to obtain the input picture to be processed; If the number of key frame pictures is not less than the second preset number, obtain a preset identifier. If the preset identifier is the first identifier, perform a fourth reconstruction process on the picture; Perform a third reconstruction process on the picture to obtain third reconstruction information. The third reconstruction information includes the third camera internal parameter value; Determine the second deviation value between the third camera internal parameter value and the initial camera internal parameter value. If the second deviation value is less than the second preset threshold, set the preset identifier to the first identifier; wherein, the fourth reconstruction process includes optimizing the camera pose and fusing the positioning parameters. The preset identifier is used to indicate the convergence situation of the camera internal parameters. The third reconstruction process includes optimizing the camera internal parameters. The initial result of the preset identifier is the second identifier, and the second preset threshold is less than the first preset threshold.

2. The three-dimensional reconstruction optimization method according to claim 1, wherein After obtaining the picture to be processed, it further includes: Determine whether the number of the pictures is greater than the third preset number. If so, initialize the reconstruction processing thread, and the reconstruction processing thread is used to perform the reconstruction process. The third preset number is less than the first preset number.

3. The three-dimensional reconstruction optimization method according to claim 2, wherein If the first deviation value is not less than the first preset threshold, re-initialize the reconstruction processing thread.

4. The three-dimensional reconstruction optimization method according to claim 1, wherein If the number of key frame pictures is not greater than the first preset number, perform a second reconstruction process on the picture. The second reconstruction process includes optimizing the camera pose.

5. The three-dimensional reconstruction optimization method according to claim 1, characterized in that The second deviation value is determined based on the following formula: Among them, is the internal parameter value of the third camera, is the focal length of the third camera, is the abscissa of the internal axis of the third camera, is the ordinate of the internal axis of the third camera, is the initial internal parameter value of the camera, is the initial focal length of the camera, is the abscissa of the internal axis of the initial camera, is the ordinate of the internal axis of the initial camera; The second preset threshold is determined based on the following formula: Among them, is the second preset threshold, is the set second focal length deviation threshold, is the set second inner axis abscissa deviation threshold, is the set second inner axis ordinate deviation threshold.

6. The three-dimensional reconstruction optimization method according to claim 1, wherein The first deviation value is determined based on the following formula: Among them, is the first camera internal parameter value, is the first camera focal length, is the abscissa of the first camera internal axis, is the ordinate of the first camera internal axis, is the initial camera internal parameter value, is the initial camera focal length, is the abscissa of the initial camera internal axis, is the ordinate of the initial camera internal axis; The first preset threshold is determined based on the following formula: Among them, is the first preset threshold, is the set first focal length deviation threshold, is the set first inner axis abscissa deviation threshold, is the set first inner axis ordinate deviation threshold.

7. The three-dimensional reconstruction optimization method according to any one of claims 1-6, characterized in that Before obtaining the picture to be processed, it further includes: Extract features from the obtained original picture to obtain a feature extraction result; According to the feature extraction result, perform feature matching on the original picture and the previously obtained original picture to obtain a feature matching result; Determine the picture to be processed according to the feature matching result.

8. A three-dimensional reconstruction optimization device, characterized in that, Including a picture acquisition module, a first reconstruction module, a deviation processing module, a third reconstruction module, an identifier acquisition module, a fourth reconstruction module, and an identifier update module, where: The picture acquisition module is used to obtain the picture to be processed; The first reconstruction module is configured to perform a first reconstruction process on the picture to obtain first reconstruction information when the number of key-frame pictures in the picture is greater than a first preset number and less than a second preset number. The first reconstruction process includes optimizing the camera internal parameters. The first reconstruction information includes a first camera internal parameter value, and the key-frame picture is a picture that has undergone a reconstruction process; The deviation processing module is configured to determine a first deviation value between the first camera internal parameter value and an initial camera internal parameter value. If the first deviation value is less than a first preset threshold, continue to obtain the input picture to be processed; The third reconstruction module is configured to perform a third reconstruction process on the picture to obtain third reconstruction information when the number of key-frame pictures is not less than the second preset number. The third reconstruction process includes optimizing the camera internal parameters. The third reconstruction information includes a third camera internal parameter value; The identification acquisition module is configured to acquire a preset identification before performing the third reconstruction process on the picture to obtain third reconstruction information. The fourth reconstruction module is configured to perform a fourth reconstruction process on the picture when the preset identification is a first identification. The fourth reconstruction process includes optimizing the camera pose and fusing the positioning parameters. The preset identification is used to indicate the convergence situation of the camera internal parameters; The identification update module is configured to determine a second deviation value between the third camera internal parameter value and the initial camera internal parameter value after performing the third reconstruction process on the picture to obtain third reconstruction information. If the second deviation value is less than a second preset threshold, set the preset identification to the first identification, where the initial result of the preset identification is a second identification, and the second preset threshold is less than the first preset threshold.

9. A three-dimensional reconstruction optimization device, characterized in that, Comprising: A memory and one or more processors; The memory is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the three-dimensional reconstruction optimization method according to any one of claims 1-7.

10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions are used to execute the three-dimensional reconstruction optimization method according to any one of claims 1-7 when executed by a computer processor.

Images