Feature point optimization method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention relates to the technical field of visual computing, and discloses a feature point optimization method and system, electronic equipment and a storage medium. The method comprises the following steps: acquiring spatial positions of spatial feature points in a plurality of image acquisition devices under coordinate systems of the image acquisition devices, and calculating first spatial position information of the spatial feature points; constructing a coordinate transformation equation according to the spatial positions of the spatial feature points in the plurality of image acquisition devices under the coordinate system of each image acquisition device, and calculating the depth information of the spatial feature points in the space; and fusing the first spatial position information and the depth information to obtain target spatial coordinates of the spatial feature points, thereby improving the precision of triangulation calculation, enabling simple triangulation calculation in a platform with limited calculation power to obtain a calculation result with relatively high precision, and enabling the calculation result to be more accurate when a large number of feature points are calculated. Meanwhile, the calculation precision can be guaranteed, and calculation resources and time are saved.

Description

[Technical field]

[0001] Embodiments of the present invention relate to the field of visual computing technology, and in particular to a feature point optimization method, system, electronic device and storage medium. [Background Technology]

[0002] Feature point triangulation has been widely used in scenes such as visual SLAM, 3D reconstruction, and triangulation. Feature point triangulation usually includes methods such as direct linear transformation DLT, linear triangulation, and inverse depth nonlinear optimization, each of which has its own advantages and disadvantages. The nonlinear optimization method calculates the result with high accuracy, but requires high computing power and consumes more resources; the direct linear transformation has high calculation accuracy, but is more sensitive to noise and has moderate computational complexity; the linear triangulation method has the lowest computational complexity, but is not as accurate as DLT and nonlinear optimization methods. In visual VIO algorithms, most of them use linear triangulation methods, and finally use Gauss-Newton method to optimize the results of linear triangulation. Although the Gauss-Newton method can improve the accuracy of linear triangulation, it also increases the complexity of calculation. Therefore, it is like using methods such as inverse depth, which requires multiple iterations to calculate accurate results. If computing resources are limited, this method is not optimal, so it is crucial to balance accuracy and computing resources.

[0003] However, not all platforms have the computing power to meet complex calculation methods. For example, when a household lawn mower uses methods such as nonlinear optimization, it will consume a lot of computing resources, resulting in insufficient computing resources for other modules. Therefore, it is necessary to use linear triangulation methods to simplify the calculation. As we can see from the in-depth analysis of the linear triangulation method, the linear triangulation method projects the feature points onto the normalized plane, and then forms a nonlinear equation of Ax=b based on the antisymmetric matrix constructed according to the points of the normalized plane, and then solves the optimal solution of the equation to obtain the coordinates. In this process, the step of projecting the feature points onto the normalized plane ignores the impact of three-dimensional depth on the entire system. Each observation actually solves only the coordinate position of the point on the normalized plane in the anchor image frame. Although the optimal solution is obtained by solving the equation by the least squares method, there will still be accuracy problems. Therefore, in order to ensure the accuracy of the linear triangulation results, it is necessary to add depth calculation to the linear triangulation method to improve the accuracy of triangulation calculations. [Summary of the invention]

[0004] The purpose of the embodiments of the present invention is to provide a feature point optimization method, system, electronic device and storage medium, which improves the accuracy of triangulation calculation by calculating the depth information of spatial feature points and integrating the depth information into the spatial coordinate calculation of the feature points.

[0005] To solve the above technical problems, an embodiment of the present invention provides a feature point optimization method, which is applied to a system including multiple image acquisition devices; wherein each image acquisition device corresponds to a coordinate system, including: acquiring the spatial position of a spatial feature point in the coordinate system corresponding to each image acquisition device, and calculating the first spatial position information of the spatial feature point; constructing a coordinate transformation equation according to the spatial position of the spatial feature point in the coordinate system of each image acquisition device, and calculating the depth information of the spatial feature point in space; fusing the first spatial position information and the depth information to obtain the target spatial coordinates of the spatial feature point.

[0006] An embodiment of the present invention also provides a feature point optimization system, including: a feature point spatial coordinate calculation module, used to obtain the spatial position of the spatial feature point in the coordinate system of each of the multiple image acquisition devices, and calculate the first spatial position information of the spatial feature point; a feature point depth information calculation module, used to construct a coordinate transformation equation according to the spatial position of the spatial feature point in the coordinate system of each of the multiple image acquisition devices, and calculate the depth information of the spatial feature point in space; a feature point coordinate fusion module, used to fuse the first spatial position information and the depth information to obtain the target spatial coordinates of the spatial feature point.

[0007] An embodiment of the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned feature point optimization method.

[0008] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, including: when the computer program is executed by a processor, the above-mentioned feature point optimization method is implemented.

[0009] In an embodiment of the present invention, first, the spatial position of a spatial feature point in a coordinate system of each of the multiple image acquisition devices is obtained, and the spatial first position information of the spatial feature point is calculated to obtain the coordinates of the spatial feature point in the coordinate system of each of the image acquisition devices; secondly, a coordinate transformation equation is constructed according to the spatial position of the spatial feature point in the coordinate system of each of the multiple image acquisition devices, and the depth information of the spatial feature point in space is calculated, so as to calculate the missing depth information; finally, the spatial first position information and the depth information are fused to obtain the target spatial coordinates of the spatial feature point, that is, the calculated depth information is fused with the first position information to obtain the accurate coordinates of the spatial feature point; by calculating the depth information of the spatial feature point and fusing the depth information into the spatial coordinate calculation of the feature point, the triangulation calculation of the feature point is optimized, thereby improving the accuracy of the triangulation calculation, so that a simple triangulation calculation performed in a platform with limited computing power can also obtain a calculation result with higher accuracy, and when a large number of feature points are calculated, the calculation accuracy can be guaranteed and computing resources and time can be saved at the same time.

[0010] In some embodiments, the image acquisition device includes a camera, and the calculation of the first spatial position information of the spatial feature point includes: selecting one camera from a plurality of cameras as an anchor camera, and the remaining cameras are non-anchor cameras; obtaining the spatial coordinates of the spatial feature point in the coordinate system of each camera; constructing a first coordinate transformation equation according to the spatial coordinates of the spatial feature point in the coordinate system of the anchor camera, and eliminating the coefficient related to the depth information of the spatial feature point, to obtain the first spatial position information of the spatial feature point in the xOy plane in the world coordinate system.

[0011] In some embodiments, calculating the depth information of the spatial feature point in space includes: constructing a second coordinate transformation equation based on the spatial coordinates of the spatial feature point in the coordinate system of the anchor camera, and eliminating the coefficients related to the x-axis and y-axis of the spatial feature point in the world coordinate system to obtain the depth information of the spatial feature point in the z-axis.

[0012] In some embodiments, the first spatial position information and the depth information are merged to obtain the target spatial coordinates of the spatial feature point, including: constructing a nonlinear equation about the depth information of the spatial feature point on the z-axis according to the first spatial position information of the spatial feature point in the xOy plane in the world coordinate system, and obtaining the z-axis spatial coordinates of the spatial feature point by calculating the least squares method; combining the z-axis spatial coordinates of the spatial feature point with the spatial coordinates of the spatial feature point in the xOy plane in the world coordinate system to obtain the target spatial coordinates of the spatial feature point.

[0013] In some embodiments, before calculating the z-axis spatial coordinates of the spatial feature point by the least squares method, the method further includes: obtaining results of multiple observations, decomposing the results of the multiple observations using the SVD method or the QR method, and constructing an overdetermined equation about x.

[0014] In some implementations, the spatial coordinates of the spatial feature points in the coordinate system of each camera are feature point coordinates of the spatial feature points in the normalized plane of each camera.

[0015] In some embodiments, before fusing the first spatial position information and the depth information, the method further includes: obtaining multiple observation results, calculating the scaling ratio of the xOy plane of the spatial feature point in the world coordinate system and the z-axis of the spatial feature point, and using the scaling ratio as a fusion coefficient of the first spatial position information and the depth information. [Drawings]

[0016] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0017] Figure 1 is a flow chart of a feature point triangulation optimization method provided by an embodiment of the present application;

[0018] Figure 2 is a schematic diagram of spatial feature points and camera positions provided in an embodiment of the present application;

[0019] Figure 3 is a structural schematic diagram of a feature point triangulation optimization system provided in one embodiment of the present application;

[0020] Figure 4 It is a schematic diagram of the structure of an electronic device provided in one embodiment of the present application. [Specific implementation method]

[0021] Since household lawn mowers consume a lot of computing resources when using methods such as nonlinear triangulation, resulting in insufficient computing resources for other modules, it is necessary to use linear triangulation methods to simplify calculations. However, the linear triangulation method projects the feature points onto the normalized plane, and then forms a nonlinear equation based on the antisymmetric matrix constructed according to the points of the normalized plane and finds the optimal solution to obtain the coordinates of the feature points. Among them, the step of projecting the feature points onto the normalized plane ignores the impact of three-dimensional depth on the entire system. Each observation actually only solves the coordinate position of the point on the normalized plane in the anchor image frame. Although the optimal solution is obtained by solving the equation by the least squares method, there are still depth accuracy issues. Therefore, in order to ensure the simplification of calculations while improving the accuracy of the results of linear triangulation, this application improves the accuracy of triangulation calculations by calculating the depth information of spatial feature points and integrating the depth information into the calculation of the spatial coordinates of the feature points.

[0022] To make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the embodiments of the present invention, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can be implemented. The division of the following embodiments is for the convenience of description, and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined and referenced with each other without contradiction.

[0023] One embodiment of the present invention relates to a feature point optimization method, which can be applied to household lawn mowers and other equipment that require visual VIO, visual SLAM, 3D reconstruction, and triangulation, and can be specifically applied to robot pose calculation, feature point triangulation, linear triangulation, inverse depth triangulation and other calculation methods as a further precision improvement optimization method for triangulation calculation. The feature point optimization method is applied to a system including multiple image acquisition devices; wherein each image acquisition device corresponds to a coordinate system, and comprises: obtaining the spatial position of the spatial feature point in the coordinate system corresponding to each image acquisition device, and calculating the spatial first position information of the spatial feature point; constructing a coordinate transformation equation according to the spatial position of the spatial feature point in the coordinate system corresponding to each image acquisition device, and calculating the depth information of the spatial feature point in space; fusing the spatial first position information and the depth information to obtain the target spatial coordinates of the spatial feature point. By calculating the depth information of the spatial feature points and integrating the depth information into the spatial coordinate calculation of the feature points, the triangulation calculation of the feature points is optimized, thereby improving the accuracy of the triangulation calculation, so that simple triangulation calculations in a platform with limited computing power can also produce high-precision calculation results. When performing a large number of feature point calculations, the calculation accuracy can be guaranteed and computing resources and time can be saved at the same time. The following is a specific description of the implementation details of the feature point optimization method of an embodiment of the present invention. The following content is only the implementation details provided for the convenience of understanding and is not necessary for the implementation of this solution.

[0024] like Figure 1 As shown, in step 101, the spatial position of the spatial feature point in the coordinate system corresponding to each image acquisition device is obtained, and the spatial first position information of the spatial feature point is calculated.

[0025] In some embodiments, the image acquisition device includes a camera, and the calculation of the spatial first position information of the spatial feature point includes: selecting one camera from multiple cameras as an anchor camera, and the remaining cameras are non-anchor cameras; obtaining the spatial coordinates of the spatial feature point in the coordinate system of each camera; constructing a first coordinate transformation equation according to the spatial coordinates of the spatial feature point in the coordinate system of the anchor camera, and eliminating the coefficient related to the depth information of the spatial feature point, to obtain the spatial first position information of the spatial feature point in the xOy plane of the world coordinate system. It should be noted that in the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0026] In some implementations, the spatial coordinates of the spatial feature points in the coordinate system of each camera are feature point coordinates of the spatial feature points in the normalized plane of each camera.

[0027] In step 102, a coordinate transformation equation is constructed according to the spatial position of the spatial feature point in the coordinate system corresponding to each image acquisition device, and the depth information of the spatial feature point in space is calculated.

[0028] In some embodiments, calculating the depth information of the spatial feature point in space includes: constructing a second coordinate transformation equation based on the spatial coordinates of the spatial feature point in the coordinate system of the anchor camera, and eliminating the coefficients related to the x-axis and y-axis of the world coordinate system of the spatial feature point, to obtain the depth information of the z-axis of the spatial feature point in the world coordinate system.

[0029] In step 103, the first spatial position information and the depth information are fused to obtain the target spatial coordinates of the spatial feature point.

[0030] In some embodiments, the first spatial position information and the depth information are merged to obtain the target spatial coordinates of the spatial feature point, including: constructing a nonlinear equation about the depth information of the spatial feature point on the z-axis in the world coordinate system according to the first spatial position information of the spatial feature point in the xOy plane of the world coordinate system, and obtaining the z-axis spatial coordinates of the spatial feature point by calculating the least squares method; combining the z-axis spatial coordinates of the spatial feature point with the spatial coordinates of the spatial feature point in the xOy plane of the world coordinate system to obtain the target spatial coordinates of the spatial feature point.

[0031] In some embodiments, before calculating the z-axis spatial coordinates of the spatial feature points by the least squares method, the method further includes: obtaining the results of multiple observations, decomposing the results of the multiple observations using the SVD method or the QR method, and constructing an overdetermined equation about x. "Multiple times" means more than twice, unless otherwise clearly and specifically defined.

[0032] In some embodiments, before fusing the first spatial position information with the depth information, the method further includes: obtaining multiple observation results, calculating the scaling ratio of the spatial feature point in the xOy plane of the world coordinate system and the spatial feature point in the z axis, and using the scaling ratio as a fusion coefficient of the first spatial position information and the depth information. "Multiple times" means more than twice, unless otherwise clearly and specifically defined.

[0033] In summary, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results.

[0034] In an embodiment of the present invention, by calculating the depth information of the spatial feature points and integrating the depth information into the spatial coordinate calculation of the feature points, the triangulation calculation of the feature points is optimized, thereby improving the accuracy of the triangulation calculation, so that simple triangulation calculations performed on a platform with limited computing power can also produce calculation results with higher accuracy. When performing calculations on a large number of feature points, the calculation accuracy can be guaranteed and computing resources and time can be saved at the same time.

[0035] In a more practical implementation, Figure 2 As shown, in this embodiment, there are multiple camera positions A, C1, C2, and C3, where camera A is an anchor camera, and the image captured at the position of anchor camera A is the anchor image frame; the rotation matrix R of anchor camera A, camera C1, camera C2, and camera C3 can be obtained through camera IMU integration. f The spatial feature point P can be observed at the four camera positions A, C1, C2, and C3. f , so the spatial feature point P f The position in each camera coordinate system, i.e., the pixel coordinate, can be obtained by (u i ,v i ), where i represents the corresponding camera position; thus, we can also know the spatial feature point P f In each camera C i The coordinates of the feature points on the normalized plane of the coordinate system are also known.

[0036] Further, the spatial first position information of the spatial feature point is calculated: the spatial feature point P is calculated according to the above known parameters f In the spatial coordinates of the anchor camera position A, that is A P f Specifically, we can use the spatial feature point P f The position and spatial feature point P in the coordinate system of the anchor camera A f In a certain camera C i The position in the coordinate system establishes the following equation:

[0037]

[0038] in, Represents the spatial feature point P f In Camera C i The coordinate position in the coordinate system of ; Represents the position from anchor camera A to camera C i The rotation matrix of the position; A P f Represents the spatial feature point P f The coordinate position in the coordinate system of the anchor camera A, Indicates camera C i The coordinate position of the center point in the coordinate system of the anchor camera A.

[0039] Expanding and transforming the above formula, we can get P f In the space coordinates of the anchor camera position A:

[0040]

[0041] in, for The transposed matrix of .

[0042] because Can be regarded as a spatial feature point P f The depth and normalized coordinates are multiplied, because the spatial feature point P f The depth of is unknown, so the following equation can be established:

[0043]

[0044] in, is the z-axis coordinate to be determined; for Extract the coordinates Coordinate expression of u n and v n Represents the spatial feature point P f In Camera C i The position in the coordinate system, that is, the pixel coordinates.

[0045] According to the above equation, we can get:

[0046]

[0047] in, is the transformation representation; the unknown parameter in the formula is A P f and Therefore, the present application multiplies both sides of the equation by The antisymmetric matrix of :

[0048]

[0049] Thus Eliminate and we get:

[0050]

[0051] The above equation can be viewed as A P f However, since only one equation is obtained A Pf The result cannot meet the accuracy requirements. Therefore, the matrix can be formed by expanding the matrix and combining the observations of other camera positions to establish the following set of equations:

[0052]

[0053] Through the above matrix operation, we can get A P f However, due to the above calculation A P f The accuracy cannot meet the production requirements, and other nonlinear optimization methods cannot be used for calculation under the condition of limited computing resources. Therefore, this application will A P f The depth of is extracted separately for calculation. The specific process is as follows:

[0054] According to the above equation:

[0055]

[0056] Rewrite the above equation by replacing the spatial feature point P f Depth information A z f Extract and get:

[0057]

[0058] In the above equation, A P f Divided into and The spatial feature point P f The depth information of the anchor image frame is extracted; an antisymmetric matrix is ​​constructed to transform the spatial feature point P f Depth information at the camera Ci position:

[0059]

[0060] The spatial feature point P f The depth information at the camera Ci position is eliminated to obtain:

[0061]

[0062] In the above equation, A N i and A b f combine to form a vector, and since A b fIt can be calculated according to the optical flow tracing method and is a known parameter. Therefore, the above equation forms a nonlinear equation Ax=b containing one-dimensional depth, where x is the one-dimensional depth.

[0063] In summary, by combining the results of multiple observations, the optimal solution of the one-dimensional depth x can be obtained by using SVD decomposition or QR decomposition. In the calculation process, this application considers the anchor coordinate system to normalize the plane coordinates and add the depth information a z f , thereby improving the accuracy of calculation.

[0064] In a more practical implementation, based on the known information, the spatial coordinates P(x1, y1, z1) are obtained by linear triangulation of the spatial feature points; by introducing the observation of the anchor image frame A, the normalized plane coordinates at the position of A are used to construct a nonlinear equation about the depth z, and then the overdetermined equation Ax=b is constructed by combining multiple observations, and the least squares method is used to solve the more accurate depth z. ’ Since z1 and z ’ There is a certain scaling factor; finally, the depth scaling factor r is determined as the fusion coefficient of the first spatial position information and the depth information, that is, x1 and y1 are synchronously scaled according to r to obtain the accurate feature point P(x, y, z) = P(rx1, ry1, z ’ ).

[0065] In an embodiment of the present invention, the depth information is integrated into the original spatial position information of the spatial feature point. There is a certain scaling ratio between the depth information of the spatial feature point on the z-axis and the z-axis coordinate of the feature point. The values ​​of the x-axis and y-axis are synchronously enlarged or reduced according to the scaling ratio of the z-axis, that is, the information of the two dimensions outside the z-axis is also scaled in proportion, so the obtained coordinate result has a higher depth accuracy. In addition, the evaluation of the example in this application only involves two QR decompositions or SVD decompositions to integrate multiple observations and calculate the optimal solution. Unlike nonlinear optimization methods that require multiple iterations, it saves iterative calculation time, thereby achieving a time reduction in the calculation of a feature point. Therefore, in scenarios involving the calculation of hundreds of feature points, the calculation efficiency can be greatly improved.

[0066] The steps of the above method are divided only for the purpose of clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0067] In addition, the examples mentioned in the above embodiments can be freely combined, and any combination can be understood as an embodiment. The "embodiment" or "example" appearing in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It can be understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0068] Another embodiment of the present invention relates to a feature point triangulation optimization system, such as Figure 3 As shown, including:

[0069] The feature point spatial coordinate calculation module is used to obtain the spatial position of the spatial feature point in the coordinate system of each of the multiple image acquisition devices, and calculate the spatial first position information of the spatial feature point. Wherein, the image acquisition device includes a camera, and the spatial coordinates of the spatial feature point in the coordinate system of each camera are the feature point coordinates of the spatial feature point in the normalized plane of each camera.

[0070] The feature point depth information calculation module is used to construct a coordinate transformation equation according to the spatial position of the spatial feature point in the coordinate system of multiple cameras, and calculate the depth information of the spatial feature point in space.

[0071] The feature point coordinate fusion module is used to fuse the first spatial position information and the depth information to obtain the target spatial coordinates of the spatial feature point.

[0072] In some embodiments, the feature point spatial coordinate calculation module is specifically used to select one camera from multiple cameras as an anchor camera, and the remaining cameras are non-anchor cameras; obtain the spatial coordinates of the spatial feature points in the coordinate system of each camera; construct a first coordinate transformation equation according to the spatial coordinates of the spatial feature points in the coordinate system of the anchor camera, and after eliminating the coefficients related to the depth information of the spatial feature points, obtain the spatial first position information of the spatial feature points in the xOy plane of the world coordinate system.

[0073] In some embodiments, the feature point depth information calculation module is specifically used to construct a second coordinate transformation equation based on the spatial coordinates of the spatial feature point in the coordinate system of the anchor camera, and eliminate the coefficients related to the x-axis and y-axis of the spatial feature point to obtain the depth information of the spatial feature point on the z-axis.

[0074] In some embodiments, the feature point coordinate fusion module is specifically used to construct a nonlinear equation about the depth information of the spatial feature point in the z-axis based on the spatial first position information of the spatial feature point in the xOy plane, and obtain the z-axis spatial coordinates of the spatial feature point by calculating the least squares method; the z-axis spatial coordinates of the spatial feature point are combined with the spatial coordinates of the spatial feature point in the xOy plane to obtain the target spatial coordinates of the spatial feature point.

[0075] Furthermore, the feature point coordinate fusion module can also obtain the results of multiple observations, decompose the results of the multiple observations using the SVD method or the QR method, and construct an overdetermined equation about x. In addition, the feature point coordinate fusion module calculates the scaling ratio of the spatial feature point in the xOy plane and the spatial feature point in the z axis by obtaining the results of multiple observations, and uses the scaling ratio as the fusion coefficient of the first spatial position information and the depth information.

[0076] In an embodiment of the present invention, by calculating the depth information of the spatial feature points and integrating the depth information into the spatial coordinate calculation of the feature points, the triangulation calculation of the feature points is optimized, thereby improving the accuracy of the triangulation calculation, so that simple triangulation calculations performed on a platform with limited computing power can also produce calculation results with higher accuracy. When performing calculations on a large number of feature points, the calculation accuracy can be guaranteed and computing resources and time can be saved at the same time.

[0077] It is not difficult to find that this embodiment is a device embodiment corresponding to the above method embodiment, and this embodiment can be implemented in conjunction with the above method embodiment. The relevant technical details mentioned in the above method embodiment are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above method embodiment.

[0078] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of the present invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by the present invention, but this does not mean that there are no other units in this embodiment.

[0079] Another embodiment of the present invention relates to an electronic device, such as Figure 4 As shown, it includes at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the feature point triangulation optimization method as described above.

[0080] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0081] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0082] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0083] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0084] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present invention, and that in actual applications, various changes may be made in form and detail without departing from the spirit and scope of the present invention.

Claims

1. A feature point optimization method, characterized in that: The optimization method is applied to a system including a plurality of image acquisition devices, wherein each image acquisition device corresponds to a coordinate system, and the method comprises: Acquire the spatial position of the spatial feature point in the coordinate system corresponding to each image acquisition device, and calculate the first spatial position information of the spatial feature point; Constructing a coordinate transformation equation according to the spatial position of the spatial feature point in the coordinate system corresponding to each image acquisition device, and calculating the depth information of the spatial feature point in space; The first spatial position information and the depth information are fused to obtain the target spatial coordinates of the spatial feature point.

2. The feature point optimization method according to claim 1, characterized in that: The step of calculating the first spatial position information mark of the spatial feature point by the image acquisition device includes: Selecting one image acquisition device from a plurality of image acquisition devices as an anchor image acquisition device, and the remaining image acquisition devices as non-anchor image acquisition devices; Acquire the spatial coordinates of the spatial feature points in the coordinate system of each image acquisition device; According to the spatial coordinates of the spatial feature point in the coordinate system of the anchor image acquisition device, a first coordinate transformation equation is constructed, and after eliminating the coefficient related to the depth information of the spatial feature point, the first spatial position information of the spatial feature point on the xOy plane in the world coordinate system is obtained.

3. The feature point optimization method according to claim 2, characterized in that: The calculating the depth information of the spatial feature point in space includes: According to the spatial coordinates of the spatial feature point in the coordinate system of the anchor image acquisition device, a second coordinate transformation equation is constructed, and after eliminating the coefficients related to the x-axis and y-axis of the spatial feature point in the world coordinate system, the depth information of the z-axis of the spatial feature point in the world coordinate system is obtained.

4. The feature point optimization method according to claim 3, characterized in that: The fusing the first spatial position information and the depth information to obtain the target spatial coordinates of the spatial feature point includes: Constructing a nonlinear equation for the depth information of the spatial feature point on the z-axis in the world coordinate system according to the spatial coordinate expression of the spatial feature point on the xOy plane in the world coordinate system, and calculating the z-axis spatial coordinate of the spatial feature point by the least squares method; The z-axis spatial coordinate of the spatial feature point is combined with the first spatial position information of the spatial feature point in the xOy plane in the world coordinate system to obtain the target spatial coordinate of the spatial feature point.

5. The feature point optimization method according to claim 4, characterized in that: Before calculating the z-axis spatial coordinates of the spatial feature point by the least squares method, the method further includes: obtaining results of multiple observations, decomposing the results of the multiple observations using the SVD method or the QR method, and constructing an overdetermined equation about x.

6. The feature point optimization method according to claim 2, characterized in that: The spatial coordinates of the spatial feature points in the coordinate system of each image acquisition device are the feature point coordinates of the spatial feature points in the normalized plane of each image acquisition device.

7. The feature point optimization method according to any one of claims 1 to 6, characterized in that: Before fusing the first spatial position information and the depth information, the method further includes: Acquire multiple observation results, calculate the scaling ratio of the spatial feature point in the world coordinate system about the xOy plane and the spatial feature point in the z axis, and use the scaling ratio as a fusion coefficient of the first spatial position information and the depth information.

8. A feature point optimization system, characterized in that: include: A feature point spatial coordinate calculation module, used to obtain the spatial position of the spatial feature point in the coordinate system of each of the multiple image acquisition devices, and calculate the first spatial position information of the spatial feature point; A feature point depth information calculation module, used to construct a coordinate transformation equation according to the spatial position of the spatial feature point in the coordinate system of each of the multiple image acquisition devices, and calculate the depth information of the spatial feature point in space; The feature point coordinate fusion module is used to fuse the first spatial position information and the depth information to obtain the target spatial coordinates of the spatial feature point.

9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the feature point optimization method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the feature point optimization method according to any one of claims 1 to 7 is implemented.

Images