Method, device and electronic equipment for determining line feature observation error

Abstract

The application provides a line feature observation error determination method and device and electronic equipment, and relates to the technical field of instant positioning and map construction, and comprises the following steps: determining a visual observation error of a line feature by comparing the difference between the back projection plane of an observed line segment on an acquired image and the Plucker coordinate description of the line feature in a space corresponding to the observed line segment; establishing an observation information matrix of the line feature based on the observation noise of the observed line segment on the acquired image in terms of angle and position; and combining the visual observation error and the observation information matrix to determine the observation error of the line feature, so as to solve the technical problem of improving the reliability of SLAM application under the influence of noise and other performances of the line feature.

Description

Technical Field

[0001] This invention relates to the technical field of real-time positioning and map building, and in particular to a method, apparatus and electronic device for determining line feature observation error. Background Technology

[0002] In problems involving Simultaneous Localization and Mapping (SLAM) and Structure from Motion (SFM), line features are often used as landmarks to describe the environment.

[0003] However, due to noise, occlusion, and the performance of the algorithm itself, the endpoints of the line segments extracted from the acquired images are inaccurate, and the length of the observed line segments changes in different frames, which in turn affects the effectiveness of line features in SLAM problems. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method, apparatus and electronic device for determining the observation error of line features, and to solve the technical problem of improving the reliability of SLAM applications under the influence of noise and other performance factors on line features.

[0005] In a first aspect, an embodiment provides a method for determining the observation error of a line feature, the method comprising:

[0006] The visual observation error of the line feature is determined by comparing the difference between the back projection plane of the observed line segment on the acquired image and the Plück coordinate description of the line feature in space corresponding to the observed line segment.

[0007] Based on the observation noise of the observed line segments in terms of angle and position in the acquired image, an observation information matrix of the line features is established.

[0008] The observation error of the line feature is determined by combining the visual observation error with the observation information matrix.

[0009] In an optional implementation, the step of determining the visual observation error of the line feature by comparing the difference between the back-projection plane of the observed line segment on the acquired image and the Plück coordinate description of the line feature in space corresponding to the observed line segment includes:

[0010] Line features in the world coordinate system are expressed using Plück coordinates, and the line features expressed in the world coordinate system are transformed to the camera coordinate system through the transformation relationship between the camera coordinate system and the world coordinate system, generating unitized vector moments.

[0011] Based on the image intrinsic parameters, the observed line segments on the acquired image are back-projected into the space corresponding to the camera coordinate system to generate a back-projection plane. The back-projection plane passes through the origin of the camera coordinate system and is uniquely determined by a three-dimensional vector, which is the non-homogeneous unit normal vector of the back-projection plane.

[0012] The visual observation error of the line features corresponding to each observed line segment in the acquired image is obtained by subtracting the non-homogeneous unit normal vector of the back projection plane from the normalized vector moment expressed in the Plück coordinates of the line feature in the camera coordinate system.

[0013] In an optional implementation, the step of determining the visual observation error of the line feature by comparing the difference between the back-projection plane of the observed line segment on the acquired image and the Plück coordinate description of the line feature in space corresponding to the observed line segment further includes:

[0014] The visual observation error of the line feature is determined by the following formula:

[0015]

[0016]

[0017] n b =K T (a×b)

[0018] Where ξ represents the camera pose. The rotation matrix R and ρ in the corresponding transformation matrix correspond to the translation vector t and L in the corresponding transformation matrix. w and L c The line feature is expressed in Plück coordinates in the world coordinate system and the camera coordinate system, respectively, n. c For L c The vector moments, n b For L w The vector moments are K, where K is the camera intrinsic parameter matrix, and a and b are the two-dimensional homogeneous coordinates corresponding to the two endpoints of the line segment in the acquired image.

[0019] In an optional implementation, the step of establishing an observation information matrix for the line feature based on the observation noise in terms of angle and position includes:

[0020] Based on the two-dimensional homogeneous coordinates of the two endpoints of the line segment in the acquired image, the angular uncertainty and positional uncertainty of the line feature in the camera coordinate system under the influence of noise are determined.

[0021] Based on the angular uncertainty and the position uncertainty, an observation information matrix for the line feature is established.

[0022] In an optional implementation, the step of determining the angular and positional uncertainties of the line feature in the camera coordinate system under noise influence, based on the two-dimensional homogeneous coordinates of the two endpoints of the line segment in the acquired image, includes:

[0023] The angular and positional uncertainties of the offline features in the camera coordinate system under noise influence are determined using the following formulas:

[0024]

[0025] σ t =arcsin||K -1 σ pix q||≈||K -1 σ pix q||

[0026] Where, σ r Let σ be the angular uncertainty. t The positional uncertainty is given by σ, where a and b are the two-dimensional homogeneous coordinates corresponding to the two endpoints of the line segment in the acquired image, q is the homogeneous unit vector perpendicular to the observed line segment ab, K is the camera's intrinsic parameter matrix, and σ is the positional uncertainty. pix To capture image noise.

[0027] In an optional implementation, the step of establishing the observation information matrix of the line feature based on the angular uncertainty and the position uncertainty includes:

[0028] The observation information matrix of the line feature is established using the following formula:

[0029] ∑=I 33 (σ r 2 +σ t 2 )

[0030] Where ∑ is the observation information matrix of the line feature, σ r Let σ be the angular uncertainty. t Let represent the positional uncertainty.

[0031] In an optional implementation, the step of determining the observation error of the line feature based on the combination of the visual observation error and the observation information matrix includes:

[0032] The observation error of the line feature is determined by the following formula:

[0033]

[0034] Where E is the observation error of the line feature, r is the visual observation error of the line feature, and ∑ is the observation information matrix of the line feature.

[0035] Secondly, an embodiment provides a device for determining line feature observation error, the device comprising:

[0036] The first determining module extracts the vector moment information of the line feature expressed in Plück coordinates under the camera coordinate system, and extracts the non-homogeneous unit normal vector of the back projection plane of the line segment on the acquired image. By comparing the difference between the vector moment information and the non-homogeneous unit normal vector, the visual observation error of the line feature is determined. The establishing module establishes the observation information matrix of the line feature based on the observation noise of the line feature in terms of angle and position.

[0037] The second determining module combines the visual observation error with the observation information matrix to determine the observation error of the line feature.

[0038] Thirdly, an embodiment provides an electronic device including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the method described in any of the foregoing embodiments.

[0039] Fourthly, an embodiment provides a machine-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to perform the steps of the method described in any of the foregoing embodiments.

[0040] This invention provides a method, apparatus, and electronic device for determining line feature observation error. The line feature is expressed using Plück coordinates, and this expression is transformed from the world coordinate system to the camera coordinate system. Using the vector moment information in the Plück coordinate expression, the back-projection plane information of the line segment in the acquired image is compared with the unit normal vector moment. The visual observation error is initially determined by subtracting the unit normal vector of the back-projection plane from the unit vector normal vector of the unit normal vector. Based on this, considering the impact of performance factors such as noise on the observation error, the uncertainty index of the observation error is determined from the perspectives of the angle and position of the line segment in the acquired image, and an observation information matrix of the line feature is constructed. Based on this observation information matrix and the visual observation error, a relatively accurate line feature observation error can be finally determined, which can ignore the uncertainty caused by performance factors such as noise.

[0041] Other features and advantages of this disclosure will be set forth in the following description, or some features and advantages may be inferred from the description or determined without doubt, or may be learned by practicing the techniques described above.

[0042] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0043] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram illustrating the extraction of line segment information from acquired images, provided as an embodiment of the present invention.

[0045] Figure 2 A schematic diagram of an existing line feature error estimation provided for an embodiment of the present invention;

[0046] Figure 3 A flowchart illustrating a method for determining line feature observation error according to an embodiment of the present invention;

[0047] Figure 4 A schematic diagram of a new line feature observation error provided in an embodiment of the present invention;

[0048] Figure 5 This is a schematic diagram of uncertainty decomposition of line feature observation error provided in an embodiment of the present invention;

[0049] Figure 6 A schematic diagram of the uncertainty of line feature observation error provided in an embodiment of the present invention;

[0050] Figure 7 A functional block diagram of a device for determining line feature observation error provided in an embodiment of the present invention;

[0051] Figure 8 This is a schematic diagram of the hardware architecture of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Compared to point features, using line features as landmarks can describe the environment with less data and highlight structural information about the environment; for example... Figure 1As shown, there is abundant line segment information within the building. By utilizing the line segment information in the acquired images, it is possible to achieve camera motion estimation and 3D environment construction.

[0054] Traditional methods use specific points (such as endpoints) on a line segment and treat the distance from the point to the line as the observation error of the line in space to optimize the map, such as... Figure 2 As shown, in the camera coordinate system C, there is a straight line Lc in space, the portion of which observed by the camera is line segment ab. The observation error is defined as the sum of the distances from the two endpoints of line segment ab to the straight line l on the acquired image, where l is the projection of the straight line Lc in space onto the acquired image plane. When estimating the current camera pose through line features, or estimating the spatial pose of line features from the camera state, due to estimation errors, there is a difference between the projection l of the straight line Lc and the actual observed line segment ab.

[0055] These methods lack a complete description of straight lines due to observation errors, and are affected by occlusion, resulting in varying lengths of observed line segments. If the observation uncertainties of line segments in each acquired image are not considered separately, the accuracy of camera positioning results and line feature mapping results will be affected.

[0056] Based on this, the present invention provides a method, apparatus and electronic device for determining line feature observation error, which combines visual observation error and the influence of uncertainties such as noise to determine a more accurate line feature observation error.

[0057] To facilitate understanding of this embodiment, a method for determining line feature observation error disclosed in this embodiment of the invention will be described in detail first. This method can be applied to intelligent control devices such as vehicle-mounted systems, controllers, and cloud servers.

[0058] Figure 3 A flowchart illustrating a method for determining line feature observation error according to an embodiment of the present invention.

[0059] like Figure 3 As shown, the method includes the following steps:

[0060] Step S102: By comparing the difference between the back projection plane of the observed line segment on the acquired image and the Plück coordinate description of the corresponding spatial line feature, the visual observation error of the line feature is determined.

[0061] Specifically, the normalized vector moments in the Plück coordinate description of the line features in the camera coordinate system are extracted, and the unit normal vectors of the back projection plane of the line segments on the acquired image are extracted. The visual observation error of the line features is determined by subtraction.

[0062] Step S104: Based on the observation noise of line segments in the acquired image in terms of angle and position, establish the observation information matrix of line features.

[0063] Step S106: Based on the combination of visual observation error and observation information matrix, determine the observation error of line features.

[0064] In a preferred embodiment of practical application, line features in the world coordinate system are expressed using Plück coordinates. The line features expressed in the world coordinate system are then transformed into the camera coordinate system using the transformation relationship between the camera coordinate system and the world coordinate system. Based on the intrinsic parameters of the acquired image, line segments on the acquired image are back-projected into the space corresponding to the camera coordinate system, generating a back-projection plane. This back-projection plane passes through the origin of the camera coordinate system and can be uniquely determined by a first three-dimensional vector, which is the non-homogeneous unit normal vector of the back-projection plane. Based on the Plück coordinate expression of the line features in the camera coordinate system, it can be seen that the line features and the origin of the camera coordinate system constitute an estimation plane. This estimation plane is uniquely determined by another second three-dimensional vector, which is the non-homogeneous unit normal vector of the estimation plane formed by the line features and the origin in the camera coordinate system, and is also the vector moment in the Plück parameters. The difference between the second three-dimensional neighbor of this estimation plane and the first three-dimensional vector of the back-projection plane is considered as a visual observation error.

[0065] In some embodiments, step S102 in the foregoing embodiments includes:

[0066] Step 1.1) Express the line features in the world coordinate system using Plück coordinates, and transform the line features expressed in the world coordinate system to the camera coordinate system using the transformation relationship between the camera coordinate system and the world coordinate system.

[0067] Specifically, by expressing straight lines in space using Plück coordinates, and based on the camera pose in Lie algebra form, the line features expressed by Plück in the world coordinate system are transformed to the camera coordinate system to generate normalized vector moments.

[0068] Step 1.2): Based on the image intrinsic parameters, back-project the line segments on the acquired image to the space corresponding to the camera coordinate system to generate a back-projection plane.

[0069] The back-projection plane passes through the origin of the camera coordinate system and can be uniquely determined by a three-dimensional vector, which is the non-homogeneous unit normal vector of the back-projection plane.

[0070] Step 1.3) Subtract the non-homogeneous unit normal vector of the back-projection plane from the normalized moment vector expressed in the Plück coordinates of the line feature in the camera coordinate system to obtain the visual observation error of the line feature corresponding to each observed line segment in the acquired image.

[0071] like Figure 4 As shown, a straight line L in space under the camera coordinate system C c Based on L cThe Plück coordinates can be used to obtain the vector moments n of the line. c At the same time n c It is a non-homogeneous three-dimensional vector, and the line L in space c The plane formed by the camera origin C and the non-homogeneous three-dimensional vector n c Perpendicular. Based on the observed line segment ab, the back projection plane Cab of the observed line segment ab in the camera coordinate system can be calculated, with the normal direction of the back projection plane being n. b Since both camera and line feature pose estimation errors may exist, n c With n b The difference in vector direction can be reduced by using a nonlinear least squares method, which can optimize the pose accuracy of the camera and lines in space.

[0072] For example, the visual observation error of a line feature can be determined by the following formula:

[0073]

[0074]

[0075] n b =K T (a×b)

[0076] Where r is the visual observation error and ξ is the camera pose. The rotation matrix R and ρ in the corresponding transformation matrix correspond to the translation vector t and L in the corresponding transformation matrix. w and L c The poses of the line features in the world coordinate system and the camera coordinate system are respectively, n c For L c The moment, n b For L w The moment is K, which is the camera intrinsic parameter matrix, and a and b are the two-dimensional homogeneous coordinates corresponding to the two endpoints of the line feature.

[0077] In the world coordinate system, the transformation from the world coordinate system to the camera coordinate system is expressed using the Lie algebraic form ξ, and the line is expressed using Plück coordinates L. w =(n w ,v w ), L c =(n c ,v c By transforming the coordinate system in the formula, the straight line L in the world coordinate system can be obtained. w Obtain L in the camera coordinate system c In the moment n c The sign() function is used to ensure that vector n is a sign vector. c With n bThe directions are consistent. The general expression for the straight line of the observed line segment is obtained through the a×b operation; changes in the length of line segment ab do not affect the result of the straight line expression.

[0078] In some embodiments, the uncertainty of line observation can also be analyzed. For example, line extraction algorithms on acquired images typically select a portion of the region of interest, called the line-support region, as the pixels containing the line. Then, the line direction and position are calculated based on this support region. Due to the influence of image resolution, noise, and other performance factors, this region of interest often has uncertainty. Figure 5 As shown;

[0079] Specifically, uncertainty decomposition is performed on the straight-line observation information. Figure 5 In the left part, line segment ab is the line segment extracted from the acquired image. The error of line segment ab in the acquired image can be decomposed into the direction along the straight line and the direction perpendicular to the straight line. Specifically, when the endpoints of the line segment are disturbed by noise, the change in the position of the endpoints along the straight line does not affect the observation error of the straight line; however, when the endpoints are disturbed by noise and change in the direction perpendicular to the straight line, it will cause the line segment to translate or rotate in the acquired image. For example... Figure 5 As shown on the right, the observed line segment ab can be considered as the actual straight line a0b0 floating in the corresponding support area due to noise.

[0080] For example, step S104 can be implemented by the following steps, including:

[0081] Step 2.1) Based on the two-dimensional homogeneous coordinates of the two endpoints of the line feature, determine the angular uncertainty and positional uncertainty of the line feature in the camera coordinate system under the influence of noise.

[0082] Among them, reference Figure 6 The uncertainty of the observed line segment can be considered in two states: rotation or translation of the observed line segment relative to the actual straight line position. Projecting the line segment from the acquired image back onto the camera coordinate system, it can be expressed as two types of uncertainty: 1. ... Figure 6 As shown in the left part, the observation uncertainty of the angle of a line segment on the image acquisition plane corresponds to the spatial angle uncertainty σ in the camera coordinate system. r , σ r The axis of rotation is perpendicular to the image plane. 2. As... Figure 6 As shown in the side section, the observation uncertainty of the line segment's position corresponds to the spatial position uncertainty σ in the camera coordinate system. t , σ t The axis of rotation is parallel to the image acquisition plane. σ t and σ r The two corresponding axes of rotation are orthogonal.

[0083] The angular and positional uncertainties of the camera coordinate system under noise influence can be determined using the following formulas:

[0084]

[0085] σ t =arcsin||K -1 σ pix q||≈||K -1 σ pix q||

[0086] Where, σ r Let σ be the angular uncertainty. t Let be the positional uncertainty, a and b be the two-dimensional homogeneous coordinates corresponding to the two endpoints of the line feature, and q be a second-order unit vector perpendicular to the observed line segment ab. K is the camera's intrinsic parameter matrix, and σ is the positional uncertainty. pix To capture image noise, set it to a constant of 1.

[0087] Due to σ pix Much smaller than the line length ||ab||, and smaller than the camera intrinsic parameter f. x f y f x and f y These are the parameters of the camera's intrinsic parameter matrix K. Therefore, we approximate it using the arcsine function to obtain the final approximate result.

[0088] Step 2.2): Based on the angular uncertainty and positional uncertainty, establish the observation information matrix of the line features.

[0089] For example, the observation information matrix of the line feature is established using the following formula:

[0090] ∑=I 33 (σ r 2 +σ t 2 )

[0091] Where ∑ is the observation information matrix of the line feature, σ r Let σ be the angular uncertainty. t Let represent the positional uncertainty.

[0092] In some embodiments, the aforementioned step S016 specifically includes:

[0093] Step 3.1), determine the observation error of the line feature using the following formula:

[0094]

[0095] Where E is the observation error of the line feature, r is the visual observation error of the line feature, and ∑ is the observation information matrix of the line feature.

[0096] This invention provides a method for determining the visual observation residual of line features, which eliminates the interference of line occlusion and noise on the endpoints or positions of lines. By balancing the observation errors of different lines through weighting, it is beneficial to improve the accuracy of the optimized feature positions.

[0097] like Figure 7 As shown, this embodiment of the invention also provides a device 200 for determining line feature observation error, the device comprising:

[0098] The first determining module 201 determines the visual observation error of the line feature by comparing the difference between the back projection plane of the observed line segment on the acquired image and the Plück coordinate description of the corresponding spatial line feature; the establishing module 202 establishes the observation information matrix of the line feature based on the observation noise of the observed line segment on the acquired image in terms of angle and position; the second determining module 203 determines the observation error of the line feature by combining the visual observation error with the observation information matrix.

[0099] In some embodiments, the first determining module 201 is further specifically used to: express line features in the world coordinate system using Plück coordinates, and transform the line features expressed in the world coordinate system to the camera coordinate system using the transformation relationship between the camera coordinate system and the world coordinate system, generating a unitized vector moment; back-project the observed line segments on the acquired image to the space corresponding to the camera coordinate system according to image intrinsic parameters, generating a back-projection plane, wherein the back-projection plane passes through the origin of the camera coordinate system and is uniquely determined by a three-dimensional vector, the three-dimensional vector being the non-homogeneous unit normal vector of the back-projection plane; subtract the non-homogeneous unit normal vector of the back-projection plane from the unitized vector moment expressed in the Plück coordinates of the line features in the camera coordinate system to obtain the visual observation error of the line features corresponding to each observed line segment in the acquired image. In some embodiments, the first determining module 201 is further specifically used to: determine the visual observation error of the line features using the following formula:

[0100]

[0101]

[0102] n b =K T (a×b)

[0103] Where ξ represents the camera pose. The rotation matrix R and ρ in the corresponding transformation matrix correspond to the translation vector t and L in the corresponding transformation matrix. w and L c The line feature is expressed in Plück coordinates in the world coordinate system and the camera coordinate system, respectively, n. c For L c The vector moments, n b For L w The vector moments are K, where K is the camera intrinsic parameter matrix, and a and b are the two-dimensional homogeneous coordinates corresponding to the two endpoints of the line segment in the acquired image.

[0104] In some embodiments, the establishment module 202 is further specifically used to determine the angular uncertainty and positional uncertainty of the line feature in the camera coordinate system under the influence of noise, based on the two-dimensional homogeneous coordinates of the two endpoints of the line segment in the acquired image; and to establish the observation information matrix of the line feature according to the angular uncertainty and the positional uncertainty.

[0105] In some embodiments, the establishment module 202 is further specifically configured to determine the angular uncertainty and positional uncertainty of the offline features of the camera coordinate system under the influence of noise using the following formula:

[0106]

[0107] σ t =arcsin||K -1 σ pix q||≈||K -1 σ pix q||

[0108] Where, σ r Let σ be the angular uncertainty. t The positional uncertainty is given by σ, where a and b are the two-dimensional homogeneous coordinates corresponding to the two endpoints of the line segment in the acquired image, q is the homogeneous unit vector perpendicular to the observed line segment ab, K is the camera's intrinsic parameter matrix, and σ is the positional uncertainty. pix To capture image noise.

[0109] In some embodiments, the establishment module 202 is further specifically used to establish the observation information matrix of the line feature using the following formula:

[0110] ∑=I 33 (σ r 2 +σ t 2 )

[0111] Where ∑ is the observation information matrix of the line feature, σ r Let σ be the angular uncertainty. t Let represent the positional uncertainty.

[0112] In some embodiments, the second determining module 203 is further specifically configured to determine the observation error of the line feature using the following formula:

[0113]

[0114] Where E is the observation error of the line feature, r is the visual observation error of the line feature, and ∑ is the observation information matrix of the line feature.

[0115] Figure 8 This is a schematic diagram of the hardware architecture of the electronic device 300 provided in an embodiment of the present invention. See also... Figure 8 As shown, the electronic device 300 includes a machine-readable storage medium 301 and a processor 302, and may also include a non-volatile storage medium 303, a communication interface 304, and a bus 305; wherein the machine-readable storage medium 301, the processor 302, the non-volatile storage medium 303, and the communication interface 304 communicate with each other through the bus 305. The processor 302 can execute the method for determining the line feature observation error described in the above embodiment by reading and executing the machine-executable instructions for determining the line feature observation error in the machine-readable storage medium 301.

[0116] The machine-readable storage medium mentioned in this article can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0117] Non-volatile media can be non-volatile memory, flash memory, storage drives (such as hard disk drives), any type of storage disk (such as optical discs, DVDs, etc.), or similar non-volatile storage media, or combinations thereof.

[0118] It is understood that the specific operation methods of each functional module in this embodiment can be referred to the detailed description of the corresponding steps in the above method embodiment, and will not be repeated here.

[0119] The computer-readable storage medium provided in this embodiment of the invention stores a computer program. When the computer program code is executed, it can implement the method for determining the line feature observation error described in any of the above embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0120] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and apparatus described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0121] Furthermore, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0122] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0123] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the scope of the technology disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention.

Claims

1. A method for determining the observation error of line features, characterized in that, The method includes: The visual observation error of the line feature is determined by comparing the difference between the back projection plane of the observed line segment on the acquired image and the Plück coordinate description of the line feature in space corresponding to the observed line segment. Based on the observation noise of the observed line segments in terms of angle and position in the acquired image, an observation information matrix of the line features is established. The observation error of the line feature is determined by combining the visual observation error with the observation information matrix. The step of determining the visual observation error of a line feature by comparing the difference between the back-projection plane of the observed line segment on the acquired image and the Plück coordinate description of the corresponding spatial line feature includes: Line features in the world coordinate system are expressed using Plück coordinates, and the line features expressed in the world coordinate system are transformed to the camera coordinate system through the transformation relationship between the camera coordinate system and the world coordinate system, generating unitized vector moments. Based on the image intrinsic parameters, the observed line segments on the acquired image are back-projected into the space corresponding to the camera coordinate system to generate a back-projection plane. The back-projection plane passes through the origin of the camera coordinate system and is uniquely determined by a three-dimensional vector, which is the non-homogeneous unit normal vector of the back-projection plane. The visual observation error of the line features corresponding to each observed line segment in the acquired image is obtained by subtracting the non-homogeneous unit normal vector of the back projection plane from the normalized vector moment expressed in the Plück coordinates of the line features in the camera coordinate system. The step of establishing the observation information matrix of the line feature based on the observation noise in terms of angle and position includes: Based on the two-dimensional homogeneous coordinates of the two endpoints of the line segment in the acquired image, the angular uncertainty and positional uncertainty of the line feature in the camera coordinate system under the influence of noise are determined. Based on the angular uncertainty and the position uncertainty, establish the observation information matrix of the line feature; The step of determining the observation error of line features by combining the visual observation error with the observation information matrix includes: The observation error of the line feature is determined by the following formula: Where E is the observation error of the line feature, and r is the visual observation error of the line feature. is the observation information matrix of the line feature.

2. The method according to claim 1, characterized in that, The step of determining the visual observation error of the line feature by comparing the difference between the back projection plane of the observed line segment on the acquired image and the Plück coordinate description of the line feature in space corresponding to the observed line segment further includes: The visual observation error of the line feature is determined by the following formula: in, ξ For camera pose, ξ=(φ,ρ) , φ It is a rotation vector. exp(φ^) Rotation matrix in the corresponding transformation matrix R , ρ Translation vector in the corresponding transformation matrix t , L w and L c These are the Plück coordinates of the line feature in the world coordinate system and the camera coordinate system, respectively. L w =(n w ,v w ) , v The vector is along the direction of the line. n The direction is perpendicular to the plane formed by the origin and the straight line. n c for L c The vector moments, n b for L w vector moments ，K For the camera intrinsic parameter matrix, a and b Let be the two-dimensional homogeneous coordinates corresponding to the two endpoints of the observed line segment on the acquired image.

3. The method according to claim 1, characterized in that, The steps for determining the angular and positional uncertainties of the line feature in the camera coordinate system under noise influence, based on the two-dimensional homogeneous coordinates of the two endpoints of the line segment in the acquired image, include: The angular and positional uncertainties of the offline features in the camera coordinate system under noise influence are determined using the following formulas: in, σ r For angular uncertainty, σ t For positional uncertainty, a and b Let be the two-dimensional homogeneous coordinates corresponding to the two endpoints of the line segment in the acquired image, q be the homogeneous unit vector perpendicular to the observed line segment ab, and K be the camera's intrinsic parameter matrix. σ pix To capture image noise.

4. The method according to claim 1, characterized in that, The step of establishing the observation information matrix of the line feature based on the angular uncertainty and the position uncertainty includes: The observation information matrix of the line feature is established using the following formula: in, The observation information matrix of the line feature. σ r For angular uncertainty, σ t For positional uncertainty, It is a unit identity matrix.

5. A device for determining the observation error of line features, characterized in that, The device includes: The first determining module determines the visual observation error of the line feature by comparing the difference between the back projection plane of the observed line segment on the acquired image and the Plück coordinate description of the line feature in space corresponding to the observed line segment. A module is established to build an observation information matrix of the line features based on the observation noise of the observed line segments in terms of angle and position in the acquired image. The second determining module combines the visual observation error with the observation information matrix to determine the observation error of the line feature; The first determining module is further configured to express line features in the world coordinate system using Plück coordinates, and to transform the line features expressed in the world coordinate system to the camera coordinate system using the transformation relationship between the camera coordinate system and the world coordinate system, generating a unitized vector moment; based on image intrinsic parameters, to back-project the observed line segments on the acquired image to the space corresponding to the camera coordinate system, generating a back-projection plane, wherein the back-projection plane passes through the origin of the camera coordinate system and is uniquely determined by a three-dimensional vector, the three-dimensional vector being the non-homogeneous unit normal vector of the back-projection plane; to subtract the non-homogeneous unit normal vector of the back-projection plane from the unitized vector moment expressed in the Plück coordinates of the line features in the camera coordinate system, obtaining the visual observation error of the line features corresponding to each observed line segment in the acquired image; The module is also used to determine the angular uncertainty and positional uncertainty of the line feature in the camera coordinate system under the influence of noise, based on the two-dimensional homogeneous coordinates of the two endpoints of the line segment in the acquired image; and to establish the observation information matrix of the line feature based on the angular uncertainty and the positional uncertainty. The second determination module is also used to determine the observation error of the line feature using the following formula: Where E is the observation error of the line feature, and r is the visual observation error of the line feature. is the observation information matrix of the line feature.

6. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 5.

7. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-executable instructions that, when invoked and executed by a processor, cause the processor to perform the steps of the method according to any one of claims 1 to 5.

Images