Vision-based pose optimization method and apparatus

Abstract

The application provides a vision-based pose optimization method and device, which comprises the following steps: obtaining a to-be-processed image frame; obtaining a feature pixel point set in the to-be-processed image frame, which has a mapping relationship with a preset space point cloud set; performing reprojection error calculation based on a current pose to obtain a reprojection error value of each feature pixel point in the feature pixel point set; performing outlier processing on the feature pixel point set according to a preset 3Sigma principle and the reprojection error value to obtain a target feature point set; performing optimization processing on the target feature point set according to a preset Sigma principle and the reprojection error value to obtain a final feature point set; and performing vision-based pose optimization on the to-be-processed image frame according to the final feature point set to obtain a pose-optimized image. It can be seen that the method can realize pose optimization in a closed-loop checking manner, thereby reducing the use of computing power and improving optimization efficiency while ensuring optimization accuracy.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a vision-based pose optimization method and apparatus. Background Technology

[0002] Currently, with the development of technology, more and more smart terminals are emerging, such as virtual reality (VR) headsets, augmented reality (AR) glasses, and robots. Among these terminals, pose tracking is one of the key technologies. Existing visual SLAM schemes typically use nonlinear optimization algorithms for pose optimization, that is, they adopt an iterative approach, continuously searching for the increment of the error term to reduce the gradient of the overall sum of squared errors until the error increment is sufficiently small, at which point the objective function reaches its minimum value, thus obtaining the optimal pose solution for the target image. However, in practice, it has been found that existing methods, combined with empirical values, provide a given number of optimization attempts, but this often results in more attempts than needed, requiring optimization of a large number of images, leading to a waste of computational resources. Summary of the Invention

[0003] The purpose of this application is to provide a vision-based pose optimization method and apparatus that can achieve pose optimization by means of closed-loop verification, thereby reducing the use of computing power and improving optimization efficiency while ensuring optimization accuracy.

[0004] The first aspect of this application provides a vision-based pose optimization method, including:

[0005] Obtain the image frame to be processed;

[0006] Obtain the set of all feature pixels in the image frame to be processed that have a mapping relationship with a preset spatial point cloud set;

[0007] Based on the current pose, the reprojection error is calculated to obtain the reprojection error value of each feature pixel in the feature pixel set.

[0008] Based on the preset 3Sigma principle and the reprojection error value, the feature pixel set is processed to remove outliers to obtain the target feature point set;

[0009] Based on the preset Sigma principle and the reprojection error value, the target feature point set is optimized to obtain the final feature point set.

[0010] Based on the final set of feature points, the image frame to be processed is subjected to vision-based pose optimization to obtain a pose-optimized image.

[0011] In the above implementation process, this method can first acquire the image frame to be processed; then, acquire the set of all feature pixels in the image frame that have a mapping relationship with a preset spatial point cloud set; and calculate the reprojection error based on the current pose to obtain the reprojection error value of each feature pixel in the feature pixel set; then, according to the preset 3Sigma principle and the reprojection error value, remove outliers from the feature pixel set to obtain the target feature point set; and then, according to the preset Sigma principle and the reprojection error value, optimize the target feature point set to obtain the final feature point set; finally, perform vision-based pose optimization on the image frame to be processed based on the final feature point set to obtain the pose-optimized image. It can be seen that this method can achieve pose optimization using a closed-loop verification method, thereby reducing computational power and improving optimization efficiency while ensuring optimization accuracy.

[0012] Further, the step of removing outliers from the feature pixel set according to the preset 3Sigma principle and the reprojection error value to obtain the target feature point set includes:

[0013] Based on the preset 3Sigma principle and the reprojection error value, calculate the standard deviation corresponding to each feature pixel in the feature pixel set;

[0014] The feature pixel set is processed by removing outliers according to the 3Sigma principle and the standard deviation to obtain the target feature point set.

[0015] Further, the optimization process of the target feature point set based on the preset Sigma principle and the reprojection error value to obtain the final feature point set includes:

[0016] The target feature point set is optimized according to the preset Sigma principle, the reprojection error value, and the preset kernel function to obtain the overall optimized feature point set;

[0017] Verify the degree of overall error reduction of the overall optimized feature point set after this optimization;

[0018] Determine whether the overall error reduction is within a preset difference range;

[0019] If so, the overall optimized feature point set obtained in this optimization will be determined as the final feature point set.

[0020] Further, the optimization of the target feature point set based on the preset Sigma principle, the reprojection error value, and the preset kernel function to obtain the overall optimized feature point set includes:

[0021] Based on the preset Sigma principle and the reprojection error value, the target feature point set is divided into a feature point set to be optimized and other feature point sets; wherein, the reprojection error value of the feature points in the feature point set to be optimized is greater than the standard deviation corresponding to the feature point;

[0022] The set of feature points to be optimized is optimized by using a preset kernel function to obtain an optimized set of feature points;

[0023] The overall optimized feature point set is obtained by summing the optimized feature point set and the other feature point sets.

[0024] Further, the verification of the overall error reduction of the overall optimized feature point set includes:

[0025] Obtain the sum of the quadratic terms of the total error in this optimization and the sum of the quadratic terms of the total error in the previous optimization;

[0026] The difference between the sum of the quadratic terms of the overall error and the sum of the quadratic terms of the historical overall error is calculated to obtain the degree of reduction in the overall error after this optimization.

[0027] A second aspect of this application provides a vision-based pose optimization device, the vision-based pose optimization device comprising:

[0028] The first acquisition unit is used to acquire the image frame to be processed;

[0029] The second acquisition unit is used to acquire the set of all feature pixels in the image frame to be processed that have a mapping relationship with the preset spatial point cloud set;

[0030] The calculation unit is used to calculate the reprojection error based on the current pose and obtain the reprojection error value of each feature pixel in the set of feature pixels.

[0031] The first processing unit is used to perform outlier removal processing on the feature pixel set according to the preset 3Sigma principle and the reprojection error value to obtain the target feature point set.

[0032] The second processing unit is used to optimize the target feature point set according to the preset Sigma principle and the reprojection error value to obtain the final feature point set.

[0033] The pose optimization unit is used to perform vision-based pose optimization on the image frame to be processed based on the final feature point set to obtain a pose-optimized image.

[0034] In the above implementation process, the device can acquire the image frame to be processed through the first acquisition unit; acquire the set of feature pixels in the image frame to be processed that have a mapping relationship with the preset spatial point cloud set through the second acquisition unit; calculate the reprojection error based on the current pose through the calculation unit to obtain the reprojection error value of each feature pixel in the feature pixel set; perform outlier removal processing on the feature pixel set through the first processing unit according to the preset 3Sigma principle and the reprojection error value to obtain the target feature point set; optimize the target feature point set according to the preset Sigma principle and the reprojection error value through the second processing unit to obtain the final feature point set; and then perform vision-based pose optimization on the image frame to be processed based on the final feature point set to obtain the pose-optimized image. It can be seen that the device can achieve pose optimization by using a closed-loop verification method, thereby reducing computing power and improving optimization efficiency while ensuring optimization accuracy.

[0035] Further, the first processing unit includes:

[0036] The calculation subunit is used to calculate the standard deviation of each feature pixel in the feature pixel set according to the preset 3Sigma principle and the reprojection error value.

[0037] The elimination sub-unit is used to remove outliers from the feature pixel set according to the 3Sigma principle and the standard deviation to obtain the target feature point set.

[0038] Further, the second processing unit includes:

[0039] An optimization subunit is used to optimize the target feature point set according to the Sigma principle, the reprojection error value, and a preset kernel function to obtain an overall optimized feature point set.

[0040] The verification subunit is used to verify the degree of overall error reduction of the overall optimized feature point set after this optimization.

[0041] The judgment subunit is used to determine whether the overall error reduction is within a preset difference range;

[0042] A sub-unit is defined to determine the overall optimized feature point set obtained by the second optimization as the final feature point set when it is determined that the degree of reduction of the overall error is within a preset difference range.

[0043] Furthermore, the optimization subunit includes:

[0044] The partitioning module is used to partition the target feature point set into a set of feature points to be optimized and other feature point sets according to the preset Sigma principle and the reprojection error value; wherein, the reprojection error value of the feature points in the set of feature points to be optimized is greater than the standard deviation of the corresponding feature points;

[0045] The optimization module is used to optimize the set of feature points to be optimized using a preset kernel function to obtain an optimized set of feature points.

[0046] The aggregation module is used to aggregate the optimized feature point set and the other feature point sets to obtain the overall optimized feature point set.

[0047] Furthermore, the verification subunit includes:

[0048] The acquisition module is used to obtain the sum of the quadratic terms of the total error in this optimization and the sum of the quadratic terms of the total error in the previous optimization.

[0049] The verification module is used to calculate the difference between the sum of the quadratic terms of the overall error and the sum of the quadratic terms of the historical overall error, so as to obtain the degree of reduction in the overall error after this optimization.

[0050] A third aspect of this application provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to cause the electronic device to perform the vision-based pose optimization method described in any one of the first aspects of this application.

[0051] A fourth aspect of this application provides a computer-readable storage medium storing computer program instructions, which, when read and executed by a processor, perform the vision-based pose optimization method described in any one of the first aspects of this application. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A flowchart illustrating a vision-based pose optimization method provided in this application embodiment;

[0054] Figure 2 A flowchart illustrating another vision-based pose optimization method provided in this application embodiment;

[0055] Figure 3 A schematic diagram of a vision-based pose optimization device provided in an embodiment of this application;

[0056] Figure 4 This is a schematic diagram of another vision-based pose optimization device provided in an embodiment of this application. Detailed Implementation

[0057] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0058] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0059] Example 1

[0060] Please refer to Figure 1 , Figure 1 This embodiment provides a flowchart illustrating a vision-based pose optimization method. The vision-based pose optimization method includes:

[0061] S101. Obtain the image frame to be processed.

[0062] S102. Obtain the set of all feature pixels in the image frame to be processed that have a mapping relationship with the preset spatial point cloud set.

[0063] S103. Calculate the reprojection error based on the current pose to obtain the reprojection error value of each feature pixel in the feature pixel set.

[0064] In this embodiment, the method can obtain the error values ​​of all feature pixels in the image that have a mapping relationship with the spatial point cloud.

[0065] S104. Based on the preset 3Sigma principle and reprojection error value, the feature pixel set is processed to remove outliers to obtain the target feature point set.

[0066] In this embodiment, the method uses the "3Sigma principle" to achieve the effect of eliminating outliers.

[0067] In this embodiment, the 3Sigma principle refers to the 99.74% probability that the value in the Gaussian distribution falls within (μ-3σ, μ+3σ). In visual SLAM, the reprojection error follows a Gaussian distribution e ~ N(0, σ²), and the standard deviation σ is generally determined based on the pyramid level of the feature points. The scaling factor of each layer of the image pyramid is set to S. Specifically, the standard deviation of the 0th layer of the pyramid is set to P pixels, the second layer to P*S1 pixels, the third layer to P*S2 pixels, and so on.

[0068] As can be seen, since the standard deviation of each feature point is known, outliers can be directly eliminated according to the "3Sigma principle". That is, if the absolute value of the pixel error is greater than 3σ, it is determined to be an outlier and will not participate in the optimization process. This avoids the calculation process of chi-square test and improves execution efficiency.

[0069] S105. Based on the preset Sigma principle and reprojection error value, optimize the target feature point set to obtain the final feature point set.

[0070] S106. Based on the final feature point set, perform vision-based pose optimization on the image frame to be processed to obtain the pose-optimized image.

[0071] In this embodiment, the subject executing the method can be a computing device such as a computer or server, and no limitation is made in this embodiment.

[0072] In this embodiment, the subject executing the method can also be a smart device such as a smartphone or tablet, and no limitation is made in this embodiment.

[0073] As can be seen, the vision-based pose optimization method described in this embodiment can replace the chi-square test with the "3 Sigma principle," allowing more feature points to participate in the optimization process while ensuring outlier removal. Furthermore, it avoids the computational process of the chi-square test by directly comparing feature points. On the other hand, this method can perform coarse filtering based on the "3 Sigma principle" to filter out outliers during the optimization process, and then perform fine filtering based on the "Sigma principle." This enhances the robustness of the optimization results when using kernel functions for larger error terms, thus fully leveraging the role of the kernel function. Finally, compared to empirical values ​​in existing methods, this method can also employ closed-loop verification, reducing computational power and improving optimization efficiency while ensuring optimization accuracy.

[0074] Example 2

[0075] Please refer to Figure 2 , Figure 2This embodiment provides a flowchart illustrating a vision-based pose optimization method. The vision-based pose optimization method includes:

[0076] S201. Obtain the image frame to be processed.

[0077] S202. Obtain the set of all feature pixels in the image frame to be processed that have a mapping relationship with the preset spatial point cloud set.

[0078] S203. Calculate the reprojection error based on the current pose to obtain the reprojection error value of each feature pixel in the feature pixel set.

[0079] S204. Based on the preset 3Sigma principle and reprojection error value, calculate the standard deviation corresponding to each feature pixel in the feature pixel set.

[0080] S205. Based on the 3Sigma principle and standard deviation, outliers are removed from the feature pixel set to obtain the target feature point set.

[0081] S206. Based on the preset Sigma principle and reprojection error value, the target feature point set is divided into the feature point set to be optimized and other feature point sets.

[0082] In this embodiment, the reprojection error value of the feature points in the set of feature points to be optimized is greater than the standard deviation of the feature points.

[0083] S207. The set of feature points to be optimized is optimized by using a preset kernel function to obtain the optimized feature point set.

[0084] In this embodiment, the method can use the "Sigma principle" to determine the kernel function threshold.

[0085] In this embodiment, the Sigma principle refers to the fact that in a Gaussian distribution, the probability of a value falling within the range (μ-σ, μ+σ) is 68.26%. In visual SLAM, the reprojection error follows a Gaussian distribution e ~ N(0, σ²). Since the standard deviation of each feature point is known, according to the Sigma principle, when the absolute value of the pixel error is greater than σ, the result is considered to be a large error, and such feature points need to be marked. Based on this, it is easy to see that using a kernel function during the optimization process can improve the robustness of the optimization results.

[0086] S208. Summarize the optimized feature point set and other feature point sets to obtain the overall optimized feature point set.

[0087] S209. Obtain the sum of the quadratic terms of the total error in this optimization and the sum of the quadratic terms of the total error in the previous optimization.

[0088] S210. Calculate the difference between the sum of the quadratic terms of the overall error and the sum of the quadratic terms of the historical overall error to obtain the degree of reduction in the overall error after this optimization.

[0089] S211. Determine whether the overall error reduction is within the preset difference range. If yes, proceed to step S212; otherwise, proceed to steps S206 to S211.

[0090] In this embodiment, after each optimization, the method verifies the degree of overall error reduction to determine whether to terminate the optimization.

[0091] In this embodiment, the method can obtain the sum of the quadratic terms of the total error of this optimization after a single optimization. This result is then compared with the result of the previous optimization. If the difference is within an acceptable range, the optimization process can be stopped.

[0092] S212. Determine the overall optimized feature point set obtained from the second optimization as the final feature point set.

[0093] S213. Based on the final feature point set, perform vision-based pose optimization on the image frame to be processed to obtain the pose-optimized image.

[0094] In this embodiment, reprojection error is a means of acquiring error in visual SLAM, and pose optimization is one application scenario. Specifically, this method can also be extended to all scenarios that use nonlinear optimization methods for error adjustment, such as prior map matching and "visual-inertial navigation" fusion.

[0095] In this embodiment, the subject executing the method can be a computing device such as a computer or server, and no limitation is made in this embodiment.

[0096] In this embodiment, the subject executing the method can also be a smart device such as a smartphone or tablet, and no limitation is made in this embodiment.

[0097] As can be seen, the vision-based pose optimization method described in this embodiment can replace the chi-square test with the "3 Sigma principle," allowing more feature points to participate in the optimization process while ensuring outlier removal. Furthermore, it avoids the computational process of the chi-square test by directly comparing feature points. On the other hand, this method can perform coarse filtering based on the "3 Sigma principle" to filter out outliers during the optimization process, and then perform fine filtering based on the "Sigma principle." This enhances the robustness of the optimization results when using kernel functions for larger error terms, thus fully leveraging the role of the kernel function. Finally, compared to empirical values ​​in existing methods, this method can also employ closed-loop verification, reducing computational power and improving optimization efficiency while ensuring optimization accuracy.

[0098] Example 3

[0099] Please refer to Figure 3 , Figure 3 This is a schematic diagram of a vision-based pose optimization device provided in this embodiment. Figure 3 As shown, the vision-based pose optimization device includes:

[0100] The first acquisition unit 310 is used to acquire the image frame to be processed;

[0101] The second acquisition unit 320 is used to acquire the set of all feature pixels in the image frame to be processed that have a mapping relationship with the preset spatial point cloud set;

[0102] The calculation unit 330 is used to calculate the reprojection error based on the current pose and obtain the reprojection error value of each feature pixel in the feature pixel set.

[0103] The first processing unit 340 is used to perform outlier removal processing on the feature pixel set according to the preset 3Sigma principle and reprojection error value to obtain the target feature point set.

[0104] The second processing unit 350 is used to optimize the target feature point set according to the preset Sigma principle and reprojection error value to obtain the final feature point set.

[0105] The pose optimization unit 360 is used to perform vision-based pose optimization on the image frame to be processed based on the final feature point set, so as to obtain the pose-optimized image.

[0106] In this embodiment, the explanation of the vision-based pose optimization device can be referred to the description in Embodiment 1 or Embodiment 2, and will not be repeated here.

[0107] As can be seen, the vision-based pose optimization device described in this embodiment can use the "3Sigma principle" instead of the "chi-square test," allowing more feature points to participate in the optimization process while ensuring outlier removal. Furthermore, it avoids the computational process of the "chi-square test" by directly comparing the magnitudes. On the other hand, this method can perform coarse filtering based on the "3Sigma principle" to filter out outliers during the optimization process, and then perform fine filtering based on the "Sigma principle." This enhances the robustness of the optimization results when using kernel functions for larger error terms, thus fully leveraging the role of the kernel function. Finally, compared to empirical values ​​in existing methods, this method can also employ closed-loop verification, reducing computational power and improving optimization efficiency while ensuring optimization accuracy.

[0108] Example 4

[0109] Please refer to Figure 4 , Figure 4 This is a schematic diagram of a vision-based pose optimization device provided in this embodiment. Figure 4 As shown, the vision-based pose optimization device includes:

[0110] The first acquisition unit 310 is used to acquire the image frame to be processed;

[0111] The second acquisition unit 320 is used to acquire the set of all feature pixels in the image frame to be processed that have a mapping relationship with the preset spatial point cloud set;

[0112] The calculation unit 330 is used to calculate the reprojection error based on the current pose and obtain the reprojection error value of each feature pixel in the feature pixel set.

[0113] The first processing unit 340 is used to perform outlier removal processing on the feature pixel set according to the preset 3Sigma principle and reprojection error value to obtain the target feature point set.

[0114] The second processing unit 350 is used to optimize the target feature point set according to the preset 3Sigma principle and reprojection error value to obtain the final feature point set.

[0115] The pose optimization unit 360 is used to perform vision-based pose optimization on the image frame to be processed based on the final feature point set, so as to obtain the pose-optimized image.

[0116] As an optional implementation, the first processing unit 340 includes:

[0117] The calculation subunit 341 is used to calculate the standard deviation of each feature pixel in the feature pixel set according to the preset 3Sigma principle and the reprojection error value.

[0118] The removal subunit 342 is used to remove outliers from the feature pixel set according to the 3Sigma principle and standard deviation to obtain the target feature point set.

[0119] As an optional implementation, the second processing unit 350 includes:

[0120] The optimization subunit 351 is used to optimize the target feature point set according to the Sigma principle, the reprojection error value and the preset kernel function to obtain the overall optimized feature point set;

[0121] Verification subunit 352 is used to verify the degree of overall error reduction of the overall optimized feature point set after this optimization.

[0122] Judgment subunit 353 is used to determine whether the overall error reduction is within a preset difference range;

[0123] Subunit 354 is defined as the final set of feature points obtained by sub-optimization when the degree of reduction of the overall error is determined to be within a preset difference range.

[0124] As an optional implementation, the optimization subunit 351 includes:

[0125] The partitioning module is used to divide the target feature point set into a set of feature points to be optimized and other feature point sets according to the preset Sigma principle and reprojection error value; wherein, the reprojection error value of the feature points in the set of feature points to be optimized is greater than the standard deviation of the feature points;

[0126] The optimization module is used to optimize the set of feature points to be optimized using a preset kernel function to obtain an optimized set of feature points.

[0127] The summary module is used to summarize the optimized feature point set and other feature point sets to obtain the overall optimized feature point set.

[0128] As an optional implementation, the verification subunit 352 includes:

[0129] The acquisition module is used to obtain the sum of the quadratic terms of the total error in this optimization and the sum of the quadratic terms of the total error in the previous optimization.

[0130] The verification module is used to calculate the difference between the sum of the quadratic terms of the overall error and the sum of the quadratic terms of the historical overall error, so as to obtain the degree of reduction in the overall error after this optimization.

[0131] In this embodiment, the explanation of the vision-based pose optimization device can be referred to the description in Embodiment 1 or Embodiment 2, and will not be repeated here.

[0132] As can be seen, the vision-based pose optimization device described in this embodiment can use the "3Sigma principle" instead of the "chi-square test," allowing more feature points to participate in the optimization process while ensuring outlier removal. Furthermore, it avoids the computational process of the "chi-square test" by directly comparing the magnitudes. On the other hand, this method can perform coarse filtering based on the "3Sigma principle" to filter out outliers during the optimization process, and then perform fine filtering based on the "Sigma principle." This enhances the robustness of the optimization results when using kernel functions for larger error terms, thus fully leveraging the role of the kernel function. Finally, compared to empirical values ​​in existing methods, this method can also employ closed-loop verification, reducing computational power and improving optimization efficiency while ensuring optimization accuracy.

[0133] This application provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform the vision-based pose optimization method in embodiment 1 or embodiment 2 of this application.

[0134] This application provides a computer-readable storage medium storing computer program instructions. When these computer program instructions are read and executed by a processor, they perform the vision-based pose optimization method described in embodiment 1 or embodiment 2 of this application.

[0135] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0136] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0137] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0138] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0139] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0140] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A vision-based pose optimization method, characterized in that, include: Obtain the image frame to be processed; Obtain the set of all feature pixels in the image frame to be processed that have a mapping relationship with a preset spatial point cloud set; Based on the current pose, the reprojection error is calculated to obtain the reprojection error value of each feature pixel in the feature pixel set. Based on the preset 3Sigma principle and the reprojection error value, calculate the standard deviation corresponding to each feature pixel in the feature pixel set; The feature pixel set is processed by removing outliers according to the 3Sigma principle and the standard deviation to obtain the target feature point set. Based on the preset Sigma principle and the reprojection error value, the target feature point set is optimized to obtain the final feature point set. Based on the final set of feature points, the image frame to be processed is subjected to vision-based pose optimization to obtain a pose-optimized image. In visual SLAM, the standard deviation Sigma is determined based on the hierarchy of the image pyramid of feature points. The scaling between the hierarchy of the image pyramid is a preset scale. The standard deviation of the first layer of the image pyramid is a preset number of pixels, and the standard deviation of other layers is calculated based on the preset number of pixels and the preset scale.

2. The vision-based pose optimization method according to claim 1, characterized in that, The optimization process, based on the preset Sigma principle and the reprojection error value, of the target feature point set to obtain the final feature point set includes: The target feature point set is optimized according to the preset Sigma principle, the reprojection error value, and the preset kernel function to obtain the overall optimized feature point set; Verify the degree of overall error reduction of the overall optimized feature point set after this optimization; Determine whether the overall error reduction is within a preset difference range; If so, the overall optimized feature point set obtained in this optimization will be determined as the final feature point set.

3. The vision-based pose optimization method according to claim 2, characterized in that, The optimization process of the target feature point set based on the preset Sigma principle, the reprojection error value, and the preset kernel function to obtain the overall optimized feature point set includes: Based on the preset Sigma principle and the reprojection error value, the target feature point set is divided into a feature point set to be optimized and other feature point sets; wherein, the reprojection error value of the feature points in the feature point set to be optimized is greater than the standard deviation corresponding to the feature point; The set of feature points to be optimized is optimized by using a preset kernel function to obtain an optimized set of feature points; The overall optimized feature point set is obtained by summing the optimized feature point set and the other feature point sets.

4. The vision-based pose optimization method according to claim 2, characterized in that, The verification of the overall error reduction of the optimized feature point set after this optimization includes: Obtain the sum of the quadratic terms of the total error in this optimization and the sum of the quadratic terms of the total error in the previous optimization; The difference between the sum of the quadratic terms of the overall error and the sum of the quadratic terms of the historical overall error is calculated to obtain the degree of reduction in the overall error after this optimization.

5. A vision-based pose optimization device, characterized in that, The vision-based pose optimization device includes: The first acquisition unit is used to acquire the image frame to be processed; The second acquisition unit is used to acquire the set of all feature pixels in the image frame to be processed that have a mapping relationship with the preset spatial point cloud set; The calculation unit is used to calculate the reprojection error based on the current pose and obtain the reprojection error value of each feature pixel in the set of feature pixels. The first processing unit is used to perform outlier removal processing on the feature pixel set according to the preset 3Sigma principle and the reprojection error value to obtain the target feature point set. The second processing unit is used to optimize the target feature point set according to the preset Sigma principle and the reprojection error value to obtain the final feature point set. The pose optimization unit is used to perform vision-based pose optimization on the image frame to be processed based on the final feature point set to obtain a pose-optimized image. The first processing unit includes: The calculation subunit is used to calculate the standard deviation of each feature pixel in the feature pixel set according to the preset 3Sigma principle and the reprojection error value. The elimination sub-unit is used to remove outliers from the feature pixel set according to the 3Sigma principle and the standard deviation to obtain the target feature point set. In visual SLAM, the standard deviation Sigma is determined based on the hierarchy of the image pyramid of feature points. The scaling between the hierarchy of the image pyramid is a preset scale. The standard deviation of the first layer of the image pyramid is a preset number of pixels, and the standard deviation of other layers is calculated based on the preset number of pixels and the preset scale.

6. The vision-based pose optimization device according to claim 5, characterized in that, The second processing unit includes: An optimization subunit is used to optimize the target feature point set according to the Sigma principle, the reprojection error value, and a preset kernel function to obtain an overall optimized feature point set. The verification subunit is used to verify the degree of overall error reduction of the overall optimized feature point set after this optimization. The judgment subunit is used to determine whether the overall error reduction is within a preset difference range; A sub-unit is defined to determine the overall optimized feature point set obtained by the second optimization as the final feature point set when it is determined that the degree of reduction of the overall error is within a preset difference range.

7. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor running the computer program to enable the electronic device to perform the vision-based pose optimization method according to any one of claims 1 to 4.

8. A readable storage medium, characterized in that, The readable storage medium stores computer program instructions, which, when read and executed by a processor, perform the vision-based pose optimization method according to any one of claims 1 to 4.

Images