A method for determining posture and electronic equipment

Abstract

The embodiment of the present application is applicable to the field of synchronous positioning and mapping technology, and provides a method for determining a posture and an electronic device, the method comprising: obtaining measurement data; the measurement data comprises positioning information, IMU information, image information and point cloud information; generating initial data based on the positioning information and IMU information; obtaining positioning observation information and motion observation information; constructing a first error function; optimizing the initial data using the positioning observation information, motion observation information and the first error function to generate first posture data; generating second posture data based on the positioning information, IMU information, image information and first posture data; generating relative posture based on point cloud information, IMU information and second posture data; determining the target posture based on the positioning information, IMU information, image information, second posture data and relative posture. The embodiment of the present application can achieve the optimization convergence direction of guiding the posture from coarse to fine, and improve the accuracy of posture estimation.

Description

Technical Field

[0001] The embodiments of the present application belong to the field of synchronous positioning and mapping technology, and in particular, relate to a posture determination method and electronic device. Background Art

[0002] Simultaneous Localization and Mapping (SLAM) technology is a technology used for intelligent devices such as robots or self-driving vehicles to navigate autonomously in unknown environments. SLAM technology usually requires the use of laser and visual sensors to obtain the relative position changes of the same static object in the environment in different observations to perform pose estimation, but observation degradation (for example: laser observation and visual observation) often occurs in tunnel scenes, resulting in large errors in pose estimation. Laser observation degradation refers to insufficient observation of static objects in a certain direction, resulting in a lack of constraints on pose estimation. Visual observation degradation refers to the low accuracy of extracting and tracking feature points of static objects, resulting in a lack of constraints on pose estimation.

[0003] To solve the mapping problem in observation-degraded scenes, related technologies use LIO (Lidar Inertial Odometry) / LVIO (Lidar Visual Inertial Odometry) algorithms to recursively infer the pose, and fuse LIO / LVIO pose with GPS observations through pose graph optimization to reduce the cumulative drift of pose. However, in related technologies, GPS is not effective enough in tunnel scenes, and LIO / LVIO lacks effective observation information, resulting in large errors in the calculated pose. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a posture determination method and an electronic device to improve the accuracy of posture calculation when laser observation is degraded and visual observation is degraded.

[0005] A first aspect of an embodiment of the present application provides a method for determining a posture, including:

[0006] Acquire measurement data; the measurement data includes positioning information, IMU information, image information and point cloud information;

[0007] Based on the positioning information and the IMU information, generating initial data;

[0008] Obtain positioning observation information and motion observation information;

[0009] Constructing a first error function;

[0010] The initial data is optimized using the positioning observation information, the motion observation information and the first error function to generate the first pose data.

[0011] Generate second pose data based on the positioning information, the IMU information, the image information and the first pose data;

[0012] Generate a relative pose based on the point cloud information, the IMU information, and the second pose data;

[0013] The target posture is determined based on the positioning information, the IMU information, the image information, the second posture data and the relative posture.

[0014] A second aspect of an embodiment of the present application provides a posture determination device, including:

[0015] A data acquisition module is used to acquire measurement data; the measurement data includes positioning information, IMU information, image information and point cloud information;

[0016] An initial data generation module, used to generate initial data based on the positioning information and IMU information;

[0017] The first observation information acquisition module is used to acquire positioning observation information and motion observation information;

[0018] A first error function construction module, used to construct a first error function;

[0019] A first pose data generating module, configured to optimize the initial data using the positioning observation information, the motion observation information and the first error function to generate the first pose data;

[0020] A second posture data generating module, used for generating second posture data based on the positioning information, the IMU information, the image information and the first posture data;

[0021] A relative posture generation module, used to generate a relative posture based on the point cloud information, the IMU information, and the second posture data;

[0022] A target posture generation module is used to determine the target posture based on the positioning information, the IMU information, the image information, the second posture data and the relative posture.

[0023] A third aspect of an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the posture determination method as described in the first aspect above.

[0024] A fourth aspect of an embodiment of the present application provides a computer program product, including a computer program, which, when executed, enables the posture determination method described in the first aspect to be executed.

[0025] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the posture determination method described in the first aspect above.

[0026] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0027] In an embodiment of the present application, measurement data is obtained; the measurement data includes positioning information, IMU information, image information and point cloud information; initial data is generated based on the positioning information and IMU information; positioning observation information and motion observation information are obtained; a first error function is constructed; the initial data is optimized using the positioning observation information, motion observation information and the first error function to generate first pose data; second pose data is generated based on the positioning information, the IMU information, the image information and the first pose data; relative pose is generated based on the point cloud information, the IMU information and the second pose data; target pose is determined based on the positioning information, the IMU information, the image information, the second pose data and the relative pose; thereby achieving the optimization convergence direction of the guiding pose from coarse to fine through multi-stage data fusion, thereby improving the pose estimation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 is a schematic diagram of a posture determination method provided in an embodiment of the present application;

[0030] Figure 2 is a schematic diagram of a GI posture optimization factor provided in an embodiment of the present application;

[0031] Figure 3 is a schematic diagram of a GVI posture optimization factor provided in an embodiment of the present application;

[0032] Figure 4 is a schematic diagram of a GLVI posture optimization factor provided in an embodiment of the present application;

[0033] Figure 5 is a schematic diagram of a posture determination device provided in an embodiment of the present application;

[0034] Figure 6 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0036] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0037] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0038] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0039] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0040] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0041] In the simultaneous positioning and mapping technology, the construction of a global map requires the use of laser sensors to scan the surrounding environment in real time, obtain the geometric information of the surrounding environment, and determine the position and posture of the laser touch sensor. The estimated position and posture are used to splice the laser point cloud data to construct a globally consistent environmental map. Therefore, the accuracy of the laser sensor's position and posture is critical to the consistency of the global map. The embodiment itself discloses a method for determining a position and posture, which includes a multi-stage data fusion process to gradually converge the position and posture of the carrier, and the relevant information obtained in the above-mentioned multiple stages can be used to reduce the measurement errors generated by LIO / LVIO in the above-mentioned scenarios where laser observation degradation or visual observation degradation occurs, thereby improving the accuracy of the target position and posture output in the end. The technical solution of the present application is illustrated below through specific embodiments.

[0042] Reference Figure 1 , shows a schematic diagram of a posture method provided in an embodiment of the present application, which may specifically include the following steps:

[0043] Step 101, obtaining measurement data; the measurement data includes positioning information, IMU information, image information and point cloud information;

[0044] The measurement data can be obtained by setting a GNSS (Global Navigation Satellite System) signal processing module, an IMU (Inertial Measurement Unit) module, a vision module, and a laser module in the same carrier.

[0045] The GNSS signal processing module can receive and process satellite signals (for example, GPS signals and Beidou satellite signals); the IMU module is provided with one or more IMUs for obtaining IMU information (for example, angular velocity and acceleration); the vision module is provided with one or more vision sensors (for example, cameras), which are used to collect visual information of the current environment and output image information; the laser module is used to send laser signals and generate point cloud information based on the received reflected laser signals.

[0046] Since there is a certain distance between different modules of the carrier, multiple coordinate systems can be constructed for each module, such as the laser coordinate system of the laser module, the visual coordinate system of the vision module, the IMU coordinate system of the IMU module, and the GNSSS coordinate system of the GNSS signal processing module.

[0047] One or more geographic coordinate systems may be constructed for the actual environment, such as a geocentric coordinate system, a station-centric coordinate system (eg, an east-north-sky coordinate system), and a world coordinate system constructed for LIO / LVIO.

[0048] It can be understood that since the above-mentioned coordinate system is foreseeably constructed and determined, the coordinate position in any of the above-mentioned coordinate systems can be used to convert to the coordinate position in any other coordinate system in any step or process of the embodiment of the present application. The specific conversion process and principle will not be repeated below.

[0049] The posture includes position and attitude, and the position and attitude are respectively represented by B-spline curves in the embodiment of the present application.

[0050] For the d-order uniform B-spline curve (the first B-spline curve), the translation , , by the control point Determine that the above control points correspond to the time , and the relationship between each control point can be expressed as follows, that is, the expression of the B-spline curve is:

[0051] (1)

[0052] In the formula, , , express The jth column of It is only related to the order of the uniform B-spline curve and is a pre-determined value, that is, the matrix of B-spline curves with the same order p(t) is the position at any time t, which is represented by several control points.

[0053] Formula (1) is the equation passing through the control point pi+j Calculate the position of point p(t) on the curve, is a vector containing different powers of u, used to represent the current time t in the current control point time interval [t i ,t i+1 ), t i Indicates the starting time of the current control point, t i+1 It is expressed as the starting time of the next control point. , normalize the time t.

[0054] In the embodiment of the present application, a 3rd order B-spline curve is used to represent the position in the posture. For a 3rd order (d=3) B-spline curve, Specifically:

[0055] (2)

[0056] Formula (1) is transformed into formula (3), which calculates the position of the point on the curve by accumulating the difference between adjacent control points, that is, it can be expressed by formula (3): The relationship between each control point can be accumulated as follows:

[0057] (3)

[0058] The corresponding third-order B-spline matrix is:

[0059] (4)

[0060] The speed is the distance traveled per unit time, so the speed can be determined according to formula (3): The expression is:

[0061] (5)

[0062] In formula (5), represents the time interval between control points, and acceleration is the speed change per unit time. Based on the definition of acceleration and the speed expression, i.e., formula (5), acceleration The representation is:

[0063] (6)

[0064] For rotation , , can be expressed by the following uniform B-spline curve (second B-spline curve):

[0065] (7)

[0066] The posture at any time t is R(t), and the posture is represented by several control points. In formula (7), R(t) is the rotation matrix at time t, R i is the initial rotation matrix, and Respectively The exponential mapping and logarithmic mapping of are used to convert between rotation matrices and rotation vectors. , , then the angular velocity is the distance rotated per unit time in the direction of rotation. In mathematical form, the angular velocity is the derivative of formula (7). The derivative of formula (7) can be calculated by the chain rule and the properties of the exponential map, and the angular velocity is obtained. The representation is:

[0067] (8)

[0068] Among them, the expression of the local rotation matrix is:

[0069] (9)

[0070] The time derivative of the local rotation matrix is:

[0071] (10)

[0072] Based on the characteristics of the above-mentioned B-spline curve, it can be obtained that the position at any moment is determined by a first control point before any moment and a first target number of consecutive first control points after any moment; the posture at any moment is determined by a second control point before any moment and a second target number of consecutive second control points after any moment; the first target number is determined based on the order of the first B-spline curve where the first control point is located, and the second target number is determined based on the order of the second B-spline curve where the second control point is located.

[0073] As an example, the first target quantity is the first B-spline curve order, and the second target quantity is the second B-spline curve order. Taking the first B-spline curve as order 3 as an example, the position at any moment is determined by a first control point before any moment and three consecutive first control points after any moment.

[0074] As an example, two 3rd order rational B-spline curves are used to represent the translation of the IMU coordinate system relative to the station center coordinate system. and rotation For the moment , and their corresponding translation and rotation control points are and The corresponding translation, velocity, acceleration, rotation and angular velocity at this moment are interpolated by the above equations (3), (5), (6), (7) and (8) respectively.

[0075] Since IMU information is only related to the motion of the carrier, it is not affected by the environment and there is no observation degradation. Therefore, it can be used as the key information for pose estimation, and the pose of each fusion stage can be calculated based on IMU information.

[0076] Due to occlusion, GPS positioning information cannot be obtained inside the tunnel. Therefore, for the observation degradation scenario of the tunnel environment, the first posture data can be obtained based on the fusion of positioning information and IMU information at both ends of the tunnel. The first posture data may include an initial fused posture.

[0077] The first pose data can be determined through steps 102-105, as follows:

[0078] Step 102, generating initial data based on the positioning information and IMU information;

[0079] Initial data may be generated based on the positioning information and the IMU information. The initial data may include an initial position and posture, and the initial position and posture is determined by a position curve and an attitude curve.

[0080] Step 103, obtaining positioning observation information and motion observation information;

[0081] By introducing IMU factors and GNSS factors, the initial pose is optimized.

[0082] IMU factors include: acceleration observation model, angular velocity observation model, acceleration zero bias observation model and angular velocity zero bias observation model.

[0083] Construct an acceleration observation model based on the acceleration in the IMU information:

[0084] (11)

[0085] Construct an angular velocity observation model based on the angular velocity in the IMU information:

[0086] (12)

[0087] in, and It is obtained by spline interpolation. is the local gravitational acceleration, and Respectively The acceleration and angular velocity in the IMU information at this moment, and express IMU acceleration bias and angular velocity bias at this moment, The corresponding covariance is , determined by the acceleration measurement noise, The corresponding covariance is , which is determined by the measurement noise of the angular velocity.

[0088] Since the motion of the carrier conforms to the random walk characteristics, the embodiment of the present application can also construct constraints (13) and (14):

[0089] (13)

[0090] (14)

[0091] The acceleration bias observation model is:

[0092] (15)

[0093] The angular velocity zero bias observation model is:

[0094] (16)

[0095] in, The corresponding covariance is , determined by the zero bias stability of acceleration; The corresponding covariance is , determined by the bias stability of the angular velocity.

[0096] The GNSS factor includes the GNSS position observation model.

[0097] The GNSS position observation model is:

[0098] (17)

[0099] In formula (17), Represents the time delay of position information relative to the IMU, represents the arm of the GNSS, Indicates the GNSS timestamp, and Obtained by spline interpolation.

[0100] The covariance matrix of , determined by the solution result obtained through GNSS signals.

[0101] The motion observation information is determined by the above-mentioned IMU factors, including: acceleration observation A, angular velocity observation W, acceleration zero bias observation BA, angular velocity zero bias observation BW, and position observation information, including: GNSS position observation P.

[0102] The stage of optimizing the initial pose is the GI pose optimization stage, G represents GNSS, and I represents IMU. Since the initial pose can be represented by a spline curve, it can be determined that in the GI pose optimization stage, the variables to be optimized are:

[0103] (18)

[0104] In formula (18), represents the rotation of the spline, Represents the translation control points of the spline. represents the acceleration zero bias, Indicates the angular velocity zero bias.

[0105] Step 104, constructing a first error function;

[0106] Step 105, optimizing the initial data using the positioning observation information, the motion observation information and the first error function to generate the first pose data;

[0107] Based on the above observation information, the first error function can be constructed:

[0108] (19)

[0109] Reference Figure 2 , shows a schematic diagram of a GI pose optimization factor provided in an embodiment of the present application, and the factor diagram corresponding to the first error function is as follows Figure 2 As shown, the first error function is optimized by using a preset algorithm to obtain the first pose data. The first pose data includes: an initial fusion pose.

[0110] Step 105, generating second pose data based on the positioning information, the IMU information, the image information and the first pose data;

[0111] In a tunnel environment, visual degradation does not necessarily exist constantly. If the initial fusion pose obtained above is optimized based on the information that no observation degradation occurs in the image information, the pose is constrained and optimized, and this process is determined as the visual fusion stage.

[0112] In the visual fusion stage, the initial fusion pose can be optimized by combining positioning information, IMU information and image information to obtain the second pose data containing the visual fusion pose.

[0113] Step 106, generating a relative pose based on the point cloud information, the IMU information, and the second pose data;

[0114] There is observation degradation in most areas of the tunnel, resulting in a lack of longitudinal translation constraints, but there is no observation degradation in other directions, and high-precision posture constraints in the lateral, azimuth, and rotation directions can still be provided. High-precision posture cannot be obtained by relying solely on point cloud data, so the point cloud information collected by the laser module is fused with the IMU information, and the LIO algorithm is used to obtain high-precision relative posture.

[0115] Step 107: determine the target posture according to the positioning information, the IMU information, the image information, the second posture data and the relative posture.

[0116] The second posture data and relative posture including the visual optimized posture obtained through the above steps are used to construct a posture output model based on positioning information, IMU information, image information and point cloud information, so as to further constrain the posture and obtain the target posture.

[0117] In a specific implementation, each target posture can be determined as a global posture that can be used for global mapping. Based on the global posture and the corresponding point cloud information, a global map is constructed, thereby improving the accuracy of the global map.

[0118] In an embodiment of the present application, measurement data is acquired; the measurement data includes positioning information, IMU information, image information and point cloud information; based on the positioning information and the IMU information, the first pose data is generated; based on the positioning information, the IMU information, the image information and the first pose data, the second pose data is generated; based on the point cloud information, the IMU information, and the second pose data, a relative pose is generated; according to the positioning information, the IMU information, the image information, the second pose data and the relative pose, the target pose is determined; thereby achieving the optimization convergence direction of the guiding pose from coarse to fine through multi-stage data fusion, thereby improving the pose estimation accuracy.

[0119] In one implementation of the present application, generating initial data based on positioning information and IMU information includes:

[0120] For the first area, determine a first control point and a second control point based on the positioning information;

[0121] For the second area, determining a first initial point and a second initial point based on the positioning information and the IMU information;

[0122] determining a position curve based on the first control point and the first initial point;

[0123] determining a posture curve based on the second control point and the second initial point;

[0124] generating initial data based on the position curve and the posture curve;

[0125] The second area is located between the two first areas, the first area is an area where the GNSS signal meets the preset signal status condition, the second area is an area where the GNSS signal does not meet the preset signal status condition, and the positioning information is generated based on the GNSS signal.

[0126] In a real environment, the GNSS signals received in the tunnel area are easily lost, resulting in a large difference between the solution result and the actual result. Therefore, a preset signal state condition can be set to evaluate the signal quality of the GNSS signal. For signals that meet the preset signal state conditions, they are determined to be GNSS signals of good quality, based on which the position and attitude of the carrier can be calculated; for signals that do not meet the preset signal state conditions, they are determined to be GNSS signals of poor quality, and the position and attitude of the carrier cannot be directly calculated based on them.

[0127] Therefore, the actual environment can be divided into a first area and a second area, the first area is outside the tunnel, and the second area is the area where the tunnel is located.

[0128] A first control point and a second control point located in the first area are determined based on the positioning information, and a first initial point and a second initial point located in the second area are determined based on the positioning information and the IMU information.

[0129] The first control point, the first initial point and the first difference control point are used as the spline curve control points to generate a position curve, and the second control point and the second initial point are used to generate a posture curve. The initial data is determined by the position curve and the posture curve, and the initial data includes an initial posture determined by the position curve and the posture curve.

[0130] As an example, the initial positions and poses of the position curve and the pose curve may be determined by:

[0131] 1. Transform the GPS latitude and longitude measurements into the local ENU coordinate system to obtain the ENU measurement values;

[0132] 2. In areas with good GPS signals, use the ENU measurement values ​​of GPS to initialize the control points of the position spline curve, and use the heading angle measurement values ​​of GPS to initialize the control points of the attitude spline curve;

[0133] 3. In areas with poor GPS signals, for the position spline curve, the ENU measurement values ​​of the two previous and subsequent GPS signal areas with good GPS signals are used for linear interpolation to obtain the initial value of the control point in the area (i.e., p in the above formula (3) i); For the attitude spline curve, the angular velocity of the IMU is recursively calculated based on the heading angle measurement values ​​of the two GPS signals before and after, and the average of the two recursive results is taken to obtain the initial value of the control point in the area (that is, R in the above formula (7) i ).

[0134] In one implementation of the present application, the positioning information includes spatial three-dimensional information and heading angle information;

[0135] The determining of the first control point and the second control point based on the positioning information includes: determining a station center measurement value based on the spatial three-dimensional information and a preset station center coordinate system; determining the first control point based on the station center measurement value, and determining the second control point based on the heading angle information;

[0136] The method of determining the first initial point and the second initial point based on the positioning information and the IMU information includes: determining the first initial point based on the target station center measurement value; determining the second initial point based on the target heading angle information and the IMU information; the target station center measurement value is determined by the station center measurement value of the first area located at both ends of the second area, and the target heading angle information is determined by the heading angle information of the first area station located at both ends of the second area.

[0137] The spatial three-dimensional information (longitude, latitude, and altitude) in the positioning information can be transformed to obtain the station center measurement value, and the first control point is determined based on the station center measurement value, and the second control point is determined based on the heading angle information in the positioning information.

[0138] A linear difference may be performed on the area between the first initial point and the first control point to obtain a first difference control point. The target heading angle information may be recursively deduced according to the angular velocity in the IMU information, and the second initial point may be determined based on the mean of the recursive results.

[0139] In one implementation of the present application, based on the positioning information, the IMU information, the image information and the first pose data, second pose data is generated; positioning observation information, motion observation information, and visual observation information are obtained; a second error function is constructed; and the first pose data is optimized using the positioning observation information, the motion observation information, the visual observation information and the second error function to generate second pose data.

[0140] The initial fusion pose can be optimized by introducing a visual factor, and the visual factor can be determined based on visual observation information.

[0141] The visual observation information includes visual residuals for visual feature points; the visual feature points are extracted from the image information; and the visual residuals are determined by the following steps:

[0142] Determine a position observation value of the visual feature point in the image information;

[0143] Determine that the visual feature point is located at a feature point station center position in a preset station center coordinate system;

[0144] Determining a position prediction value of the visual feature point according to the center position of the feature point and preset camera projection information;

[0145] Determining the visual residual according to the position observation value and the position prediction value;

[0146] Wherein, determining the position prediction value of the visual feature point according to the center position of the feature point and preset camera projection information includes:

[0147] Determine the feature point distance from the visual feature point to the camera origin according to the feature point center position;

[0148] Determine the feature point angle according to the feature point distance; the feature point angle is the angle between the position of the visual feature point in the station center coordinate system and the camera specified direction axis;

[0149] Determining depth distortion information according to the angle of the feature points;

[0150] The position prediction value is determined according to the depth distortion information, the center position of the feature point and the camera projection information.

[0151] As an example, the above-mentioned camera designated direction axis is the camera horizontal direction axis.

[0152] Visual factors include: visual observation model.

[0153] The embodiment of the present application tracks the K visual feature points extracted from the current image information. If the tracking error is greater than a certain threshold, it means that the tracking has failed. If a feature point fails to be tracked, the feature point is extracted again to keep the total number of feature points in each frame image as K. and its series of visual observations , we can get the following equation:

[0154] (20)

[0155] (twenty one)

[0156] (twenty two)

[0157] In the above formula (20), the matrix The initial fusion pose is interpolated as above, indicating the first Perform SVD decomposition on matrix A and take the singular vector corresponding to the minimum singular value ,but .

[0158] The visual observation model is:

[0159] (twenty three)

[0160] (twenty four)

[0161] in, Indicates The frame image The pixel coordinates of feature points (position observations), is the predicted value of the position. and It is obtained by B-spline interpolation. Represents the camera projection equation (camera projection information).

[0162] This application embodiment takes a four-parameter fisheye camera as an example to further illustrate the camera projection equation. For a four-parameter fisheye camera, there are:

[0163] (25)

[0164] (26)

[0165] (27)

[0166] (28)

[0167] (29)

[0168] (30)

[0169] in, Indicates the timestamp corresponding to the feature point in the image information, , , is the coordinate of the visual feature point in the station center coordinate system, and , respectively expressed as , , Normalized coordinates.

[0170] For the global exposure model:

[0171] (31)

[0172] in, Indicates the starting exposure time of the i-th frame image, Indicates the exposure time of the image. For the rolling shutter exposure model, there are:

[0173] (32)

[0174] in, Indicates the row number of the feature point. represents the line delay of the rolling camera, The covariance matrix of , which is determined by the extraction of corner points and tracking error.

[0175] The stage of optimizing the initial fusion pose is the GVI pose optimization stage, where G stands for GNSS, V stands for visual, and I stands for IMU. Since the initial fusion pose can be represented by a spline curve, it can be determined that in the GVI pose optimization stage, the variables to be optimized are:

[0176] (33)

[0177] in Indicates the coordinates of the visual feature points in the station center coordinate system.

[0178] The motion observation information determined by the above IMU factors includes: acceleration observation A, angular velocity observation W, acceleration zero bias observation BA, and angular velocity zero bias observation BW. The position observation information determined by the above GNSS factors includes: GNSS position observation P. The visual observation information determined by the above visual factors includes: visual feature observation C.

[0179] Based on the above observation information, the second error function can be constructed:

[0180] (34)

[0181] Reference Figure 3 , shows a schematic diagram of a GVI posture optimization factor provided in an embodiment of the present application, and the factor diagram corresponding to the second error function is as follows Figure 3 As shown, the second error function is optimized by a preset algorithm to obtain second posture data. The second posture data includes: visual fusion posture, speed information, and zero bias observation information.

[0182] In one implementation of the present application, determining the target pose based on the positioning information, the IMU information, the image information, the second pose data and the relative pose includes:

[0183] Obtain positioning observation information, motion observation information, and visual observation information;

[0184] Determining laser observation information based on the relative position and posture;

[0185] Constructing a third error function;

[0186] The second posture data is optimized using the positioning observation information, the motion observation information, the visual observation information, the laser observation information and the third error function to determine the target posture.

[0187] The visual fusion pose can be optimized by introducing a laser factor, and the laser factor can be determined based on laser observation information.

[0188] Laser factors include: relative posture observation model.

[0189] (35)

[0190] In formula (35), and Represents the relative pose observation information, and its covariance matrix is .

[0191] The stage of optimizing the initial fusion pose is the GLVI pose optimization stage, G represents GNSS, L represents lidar, V represents visual, and I represents IMU. Since the visual fusion pose can be represented by a spline curve, it can be determined that in the GLVI pose optimization stage, the variables to be optimized are:

[0192] (36)

[0193] The motion observation information is determined by the above IMU factors, including: acceleration observation A, angular velocity observation W, acceleration zero bias observation BA, angular velocity zero bias observation BW. The position observation information is determined by the above GNSS factors, including: GNSS position observation P. The visual observation information is determined by the above visual factors, including: visual feature observation C, and the laser observation information is determined by the laser factors, including: relative posture observation LP.

[0194] Based on the above observation information, the third error function can be constructed:

[0195] (37)

[0196] Reference Figure 4 , shows a schematic diagram of a GLVI pose optimization factor provided in an embodiment of the present application, and the factor diagram corresponding to the second error function is as follows Figure 4 As shown, the preset algorithm is used to optimize the third error function to obtain the target posture.

[0197] In one implementation of the present application, generating a relative pose based on the point cloud information, the IMU information, and the second pose data includes:

[0198] Constructing an initial model for state prediction based on the IMU information;

[0199] Generate a laser observation model based on the point cloud information, the IMU information and the second pose data;

[0200] Determine the degree of laser observation degradation at the current location;

[0201] Determining a velocity observation model based on the second posture data and the laser observation degradation degree;

[0202] Generate a laser inertial odometer update model based on the state prediction initial model, the laser observation model, and the speed observation model;

[0203] The laser inertial odometer is used to update the model and output relative pose.

[0204] The embodiment of the present application provides an improved LIO algorithm model. Since the visual fusion posture, velocity information, and zero-bias observation information can be obtained in the above-mentioned GVI stage, the improved LIO algorithm provided in the embodiment of the present application can be dynamically initialized and adaptively adjust the observation weights of each information in the second posture data generated in the GVI stage according to the degree of scene degradation, thereby reducing the posture estimation error of the observation-degraded scene.

[0205] The state prediction initial model is obtained based on the IMU information and the laser inertial odometer model based on iterative Kalman filtering; the laser inertial odometer model based on iterative Kalman filtering is provided with a state vector, and the state vector includes: the rotation of the IMU coordinates and the preset world coordinate system, the translation of the IMU coordinates and the world coordinate system, the speed of the IMU coordinate system in the world coordinate system, the angular velocity zero bias of the IMU, and the acceleration zero bias of the IMU; wherein, the IMU coordinate system is constructed based on the IMU module, the IMU coordinates are the position of the IMU module in the IMU coordinate system, and the IMU module is a device for measuring IMU information. Specifically as follows: The improved LIO algorithm model provided in the embodiment of the present application is a LIO algorithm model based on iterative Kalman filtering, and its state vector is:

[0206] (38)

[0207] In formula (38), Indicates the rotation of the IMU relative to the LIO world coordinate system, represents the translation of the IMU relative to the LIO world coordinate system, Indicates the velocity of the IMU coordinate system in the LIO world coordinate system, Indicates the angular velocity zero bias of the IMU, Indicates the acceleration bias of the IMU.

[0208] Construct an initial state prediction model based on IMU information:

[0209] (39)

[0210] in, (40)

[0211] In formula (40), Represents the angular velocity measurement value of the IMU, Represents the acceleration measurement value of the IMU, represents the IMU angle measurement noise, represents the IMU acceleration measurement noise, Gaussian white noise representing the zero bias of the IMU angular velocity, represents the Gaussian white noise of the IMU acceleration zero bias, represents a single recursion time, Represents the gravitational acceleration in the LIO world coordinate system.

[0212] According to the point cloud information and laser observation model, the state of the point-to-surface error is updated. The laser observation model is:

[0213] (41)

[0214] In formula (41), is the source point in the current laser point cloud, is the matching point in the map, is the normal vector of the plane where the matching point is located, represents the current position, Represents the external parameters of the laser relative to the IMU.

[0215] In an embodiment of the present application, a speed observation model is determined based on the second posture data and the degree of laser observation degradation, including: determining the rotation of the IMU coordinate system in the second posture data relative to the station center coordinate system and the speed of the IMU coordinate system relative to the station center coordinate system; constructing a station center velocity vector using the rotation of the IMU coordinate system relative to the station center coordinate system and the speed of the IMU coordinate system relative to the station center coordinate system; determining the rotation of the IMU coordinate system relative to a preset world coordinate system and the speed of the IMU coordinate system relative to the world coordinate system; constructing a world velocity vector using the rotation of the IMU coordinate system relative to the preset world coordinate system and the speed of the IMU coordinate system relative to the world coordinate system; determining a velocity observation vector based on the station center velocity vector and the world velocity vector; determining a degradation weighted value corresponding to the degree of laser observation degradation; and constructing a velocity observation model based on the product of the degradation weighted value and the velocity observation vector.

[0216] Based on the velocity information output in the GVI stage (including the rotation of the IMU coordinate system relative to the station center coordinate system and the velocity of the IMU coordinate system relative to the station center coordinate system), a velocity observation model is constructed:

[0217] (42)

[0218] in,

[0219] In formula (42), and They represent the rotation and velocity of the IMU coordinate system relative to the station center coordinate system at time k, respectively, which are obtained through the above GVI stage. is the rotation of the IMU coordinate system relative to the world coordinate system at time k, is the velocity of the IMU coordinate system relative to the world coordinate system at time k.

[0220] is the station center velocity vector, is the world velocity vector.

[0221] Indicates the laser degradation degree of the current environment. For the matrix The minimum eigenvalue of . Through the laser degradation function 1 ( ), determine the degradation weighted values ​​corresponding to different degradation degrees, and the degradation weighted values ​​are 1 or 0. The matrix H is:

[0222] (43)

[0223] In formula (43), Indicates the total number of matching points.

[0224] In an implementation of the embodiment of the present application, the generating of the laser inertial odometer update model based on the state prediction initial model, the laser observation model, and the speed observation model includes:

[0225] The state prediction initial model is used to output the state difference matrix between the predicted state vector and the current state vector of the IMU, as well as the state weight value;

[0226] The laser observation model is used to output the distance error matrix from the source end to the preset map in the current point cloud information;

[0227] The velocity observation model is used to output a velocity difference matrix between a velocity vector in the current station center coordinate system and a velocity vector in the current world coordinate system;

[0228] The laser inertial odometer update model is used to determine the target state vector when the sum of the first calculated value, the second calculated value and the third calculated value is the minimum value;

[0229] Among them, the first calculated value is the product of the state difference matrix and its transpose multiplied by the state weight value, the product of the distance error matrix and its transpose multiplied by the preset distance observation weight, and the product of the speed difference matrix and its transpose multiplied by the preset speed observation weight.

[0230] Combining the above initial model, laser observation model, and speed observation model, the laser inertial odometer update model (that is, the above improved LIO algorithm model) is generated. The laser inertial odometer update model is:

[0231] (44)

[0232] In formula (44), represents the prior state (predicted state vector), Expressed as a covariance matrix (state weight value), Represents the covariance matrix of laser observation error (distance observation weight value), The covariance matrix (speed observation weight value) representing the velocity observation is solved iteratively by updating the laser inertial odometer model to obtain the target state vector, and the relative position and posture are obtained by combining formula (31).

[0233] Reference Figure 5 , shows a schematic diagram of a posture determination device provided in an embodiment of the present application, which may specifically include a data acquisition module 501, an initial data generation module 502, a first observation information acquisition module 503, a first error function construction module 504, a first posture data generation module 505, a second posture data generation module 506, a relative posture generation module 507, and a target posture generation module 508, wherein:

[0234] The data acquisition module 501 is used to acquire measurement data; the measurement data includes positioning information, IMU information, image information and point cloud information;

[0235] An initial data generating module 502 is used to generate initial data based on the positioning information and IMU information;

[0236] The first observation information acquisition module 503 is used to acquire positioning observation information and motion observation information;

[0237] A first error function construction module 504, used to construct a first error function;

[0238] A first pose data generating module 505, configured to optimize the initial data using the positioning observation information, the motion observation information and the first error function to generate the first pose data;

[0239] A second posture data generating module 506, configured to generate second posture data based on the positioning information, the IMU information, the image information and the first posture data;

[0240] A relative pose generation module 507, configured to generate a relative pose based on the point cloud information, the IMU information, and the second pose data;

[0241] The target posture generation module 508 is used to determine the target posture according to the positioning information, the IMU information, the image information, the second posture data and the relative posture.

[0242] In one implementation of the embodiment of the present application, the posture includes position and posture;

[0243] The position at any moment is determined by a first control point before any moment and a first target number of consecutive first control points after any moment; the posture at any moment is determined by a second control point before any moment and a second target number of consecutive second control points after any moment; the first target number is determined based on the order of the first B-spline curve where the first control point is located, and the second target number is determined based on the order of the second B-spline curve where the second control point is located.

[0244] In one implementation of the embodiment of the present application, the initial data generation submodule includes:

[0245] A control point determination unit, configured to determine, for the first area, a first control point and a second control point based on the positioning information;

[0246] an initial point determination unit, configured to determine, for a second area, a first initial point and a second initial point based on the positioning information and the IMU information;

[0247] a position curve determining unit, configured to determine a position curve based on the first control point and the first initial point;

[0248] a posture curve determining unit, configured to determine a posture curve based on the second control point and the second initial point;

[0249] An initial data generating unit, used for generating initial data based on the position curve and the posture curve;

[0250] The second area is located between the two first areas, the first area is an area where the GNSS signal meets the preset signal status condition, the second area is an area where the GNSS signal does not meet the preset signal status condition, and the positioning information is generated based on the GNSS signal.

[0251] In one implementation of the embodiment of the present application, the positioning information includes spatial three-dimensional information and heading angle information;

[0252] The control point determination unit comprises:

[0253] A station center measurement value determination subunit, used to determine the station center measurement value based on the spatial three-dimensional information and a preset station center coordinate system;

[0254] A control point determination subunit, configured to determine a first control point based on the station center measurement value, and determine a second control point based on the heading angle information;

[0255] The initial point determination unit comprises:

[0256] A first initial point determination unit, configured to determine a first initial point based on a target station center measurement value;

[0257] A second initial point determination unit is used to determine a second initial point based on the target heading angle information and the IMU information; the target station center measurement value is determined by the station center measurement value of the first area located at both ends of the second area, and the target heading angle information is determined by the heading angle information of the first area station located at both ends of the second area.

[0258] In one implementation of the embodiment of the present application, the second posture data generating module 506 includes:

[0259] The second acquisition submodule of observation information is used to acquire positioning observation information, motion observation information, and visual observation information;

[0260] A second error function construction submodule, used for constructing a second error function;

[0261] The second posture data generating submodule is used to optimize the first posture data by using the positioning observation information, the motion observation information, the visual observation information and the second error function to generate second posture data.

[0262] In one implementation of the embodiment of the present application, the target pose generation module 508 includes:

[0263] The third acquisition submodule of observation information is used to acquire positioning observation information, motion observation information, and visual observation information;

[0264] A laser observation information determination submodule, used to determine the laser observation information based on the relative position and posture;

[0265] A third error function construction submodule, used for constructing a third error function;

[0266] The target posture determination submodule is used to optimize the second posture data using the positioning observation information, the motion observation information, the visual observation information, the laser observation information and the third error function to determine the target posture.

[0267] In some implementations of the embodiments of the present application, the visual observation information includes visual residuals for visual feature points; the visual feature points are extracted from the image information; and the visual residuals are determined by the following submodules:

[0268] A position observation value determination submodule, used to determine the position observation value of the visual feature point in the image information;

[0269] A feature point center position determination submodule is used to determine the feature point center position of the visual feature point in a preset center coordinate system;

[0270] A position prediction value determination submodule, used to determine the position prediction value of the visual feature point according to the center position of the feature point and preset camera projection information;

[0271] A visual residual determination submodule, configured to determine the visual residual according to the position observation value and the position prediction value;

[0272] Wherein, the position observation value determination submodule includes:

[0273] A feature point distance determination unit, used to determine the feature point distance from the visual feature point to the camera origin according to the feature point station center position;

[0274] A feature point angle determination unit, used to determine the feature point angle according to the feature point distance; the feature point angle is the angle between the position of the visual feature point in the station center coordinate system and the camera specified direction axis;

[0275] A depth distortion information determining unit, configured to determine the depth distortion information according to the feature point angle;

[0276] A position prediction value determination unit is used to determine the position prediction value based on the depth distortion information, the center position of the feature point and the camera projection information.

[0277] In one implementation of the embodiment of the present application, the relative posture generation module 507 includes:

[0278] An initial model building submodule, used to build an initial model for state prediction based on the IMU information;

[0279] A laser observation model generation submodule, used to generate a laser observation model based on the point cloud information, the IMU information and the second pose data;

[0280] A laser observation degradation degree determination submodule is used to determine the laser observation degradation degree at the current location;

[0281] A speed observation model determination submodule, used to determine a speed observation model based on the second posture data and the laser observation degradation degree;

[0282] A laser inertial odometer update model generation submodule, used to generate a laser inertial odometer update model based on the state prediction initial model, the laser observation model, and the speed observation model;

[0283] The relative posture output submodule is used to update the model and output the relative posture using the laser inertial odometer.

[0284] In some implementations of the embodiments of the present application, the speed observation model determination submodule includes:

[0285] A second posture data extraction unit, used to determine the rotation of the IMU coordinate system relative to the station center coordinate system in the second posture data, and the speed of the IMU coordinate system relative to the station center coordinate system;

[0286] A station center velocity vector determination unit, used for constructing a station center velocity vector by using the rotation of the IMU coordinate system relative to the station center coordinate system and the velocity of the IMU coordinate system relative to the station center coordinate system;

[0287] A world coordinate system related information determination unit, used to determine the rotation of the IMU coordinate system relative to a preset world coordinate system and the speed of the IMU coordinate system relative to the world coordinate system;

[0288] A world speed vector determination unit, configured to construct a world speed vector using the rotation of the IMU coordinate system relative to a preset world coordinate system and the speed of the IMU coordinate system relative to the world coordinate system;

[0289] A speed observation vector determining unit, configured to determine a speed observation vector according to the station center speed vector and the world speed vector;

[0290] A degradation weighted value determining unit, used to determine a degradation weighted value corresponding to the laser observation degradation degree;

[0291] The speed observation model construction unit is used to construct a speed observation model according to the product of the degradation weighted value and the speed observation vector.

[0292] In one implementation of the embodiment of the present application, the state prediction initial model is used to output a state difference matrix between a predicted state vector of the IMU and a current state vector, as well as a state weight value;

[0293] The laser observation model is used to output the distance error matrix from the source end to the preset map in the current point cloud information;

[0294] The velocity observation model is used to output a velocity difference matrix between a velocity vector in the current station center coordinate system and a velocity vector in the current world coordinate system;

[0295] The laser inertial odometer update model is used to determine the target state vector when the sum of the first calculated value, the second calculated value and the third calculated value is the minimum value;

[0296] Among them, the first calculated value is the product of the state difference matrix and its transpose multiplied by the state weight value, the product of the distance error matrix and its transpose multiplied by the preset distance observation weight, and the product of the speed difference matrix and its transpose multiplied by the preset speed observation weight.

[0297] In an implementation of the embodiment of the present application, the state prediction initial model is obtained based on the IMU information and a laser inertial odometer model based on iterative Kalman filtering; the laser inertial odometer model based on iterative Kalman filtering is provided with a state vector, and the state vector includes:

[0298] The rotation of the IMU coordinates and the preset world coordinate system, the translation of the IMU coordinates and the world coordinate system, the speed of the IMU coordinate system in the world coordinate system, the angular velocity zero bias of the IMU, and the acceleration zero bias of the IMU;

[0299] Among them, the IMU coordinate system is constructed based on the IMU module, the IMU coordinates are the positions of the IMU module in the IMU coordinate system, and the IMU module is a device for measuring IMU information.

[0300] An embodiment of the present application provides a posture determination device, and by using this device, each step in the aforementioned method embodiments can be implemented.

[0301] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment part.

[0302] Reference Figure 6 , shows a schematic diagram of an electronic device provided by an embodiment of the present application. Figure 6 As shown, the electronic device 600 in the embodiment of the present application includes: a processor 610, a memory 620, and a computer program 621 stored in the memory 620 and executable on the processor 610. When the processor 610 executes the computer program 621, the steps in each embodiment of the above-mentioned posture method are implemented, such as Figure 1 Alternatively, when the processor 610 executes the computer program 621, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 5 Functions of modules 501 to 505 are shown.

[0303] Exemplarily, the computer program 621 may be divided into one or more modules / units, which are stored in the memory 620 and executed by the processor 610 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which may be used to describe the execution process of the computer program 621 in the electronic device 600. For example, the computer program 621 may be divided into a data acquisition module, a first pose data generation module, a second pose data generation module, a relative pose generation module, and a target pose generation module, and the specific functions of each module are as follows:

[0304] A data acquisition module is used to acquire measurement data; the measurement data includes positioning information, IMU information, image information and point cloud information;

[0305] A first pose data generating module, used to generate a first pose data based on the positioning information and the IMU information;

[0306] A second posture data generating module, used for generating second posture data based on the positioning information, the IMU information, the image information and the first posture data;

[0307] A relative posture generation module, used to generate a relative posture based on the point cloud information, the IMU information, and the second posture data;

[0308] A target posture generation module is used to determine the target posture based on the positioning information, the IMU information, the image information, the second posture data and the relative posture.

[0309] The electronic device 600 may be a computing device such as a desktop computer or a cloud server. The electronic device 600 may include, but is not limited to, a processor 610 and a memory 620. Those skilled in the art will appreciate that Figure 6 It is only an example of the electronic device 600 and does not constitute a limitation of the electronic device 600. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 600 may also include input and output devices, network access devices, buses, etc.

[0310] The processor 610 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0311] The memory 620 may be an internal storage unit of the electronic device 600, such as a hard disk or memory of the electronic device 600. The memory 620 may also be an external storage device of the electronic device 600, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 600. Further, the memory 620 may also include both an internal storage unit of the electronic device 600 and an external storage device. The memory 620 is used to store the computer program 621 and other programs and data required by the electronic device 600. The memory 620 may also be used to temporarily store data that has been output or is to be output.

[0312] An embodiment of the present application further discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the posture determination method as described in the aforementioned embodiments is implemented.

[0313] An embodiment of the present application also discloses a computer program product, including a computer program. When the computer program is run, the posture determination method described in the above embodiments is executed.

[0314] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application is described in detail with reference to the above-mentioned embodiments, a person skilled in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for determining a posture, characterized in that: include: Acquire measurement data; the measurement data includes positioning information, IMU information, image information and point cloud information; Based on the positioning information and the IMU information, generating initial data; Obtain positioning observation information and motion observation information; Constructing a first error function; The initial data is optimized using the positioning observation information, the motion observation information and the first error function to generate the first pose data. Generate second pose data based on the positioning information, the IMU information, the image information and the first pose data; Generate a relative pose based on the point cloud information, the IMU information, and the second pose data; The target posture is determined based on the positioning information, the IMU information, the image information, the second posture data and the relative posture.

2. The method according to claim 1, characterized in that Posture includes position and posture; The position at any moment is determined by a first control point before the any moment and a first target number of consecutive first control points after the any moment; The posture at any moment is determined by a second control point before said any moment and a second target number of consecutive second control points after said any moment; The first target quantity is determined based on the order of a first B-spline curve where the first control point is located, and the second target quantity is determined based on the order of a second B-spline curve where the second control point is located.

3. The method according to claim 1, characterized in that The generating initial data based on the positioning information and the IMU information includes: For the first area, determine a first control point and a second control point based on the positioning information; For the second area, determining a first initial point and a second initial point based on the positioning information and the IMU information; determining a position curve based on the first control point and the first initial point; determining a posture curve based on the second control point and the second initial point; generating initial data based on the position curve and the posture curve; The second area is located between the two first areas, the first area is an area where the GNSS signal meets the preset signal status conditions, the second area is an area where the GNSS signal does not meet the preset signal status conditions, and the positioning information is generated based on the GNSS signal of the first area.

4. The method according to claim 3, characterized in that The positioning information includes spatial three-dimensional information and heading angle information; The determining of the first control point and the second control point based on the positioning information includes: determining a station center measurement value based on the spatial three-dimensional information and a preset station center coordinate system; determining the first control point based on the station center measurement value, and determining the second control point based on the heading angle information; The method of determining the first initial point and the second initial point based on the positioning information and the IMU information includes: determining the first initial point based on the target station center measurement value; determining the second initial point based on the target heading angle information and the IMU information; the target station center measurement value is determined by the station center measurement value of the first area located at both ends of the second area, and the target heading angle information is determined by the heading angle information of the first area located at both ends of the second area.

5. The method according to claim 1, characterized in that The generating second pose data based on the positioning information, the IMU information, the image information and the first pose data comprises: Obtain positioning observation information, motion observation information, and visual observation information; constructing a second error function; The first posture data is optimized by using the positioning observation information, the motion observation information, the visual observation information and the second error function to generate second posture data; The determining the target posture according to the positioning information, the IMU information, the image information, the second posture data and the relative posture comprises: Obtain positioning observation information, motion observation information, and visual observation information; Determining laser observation information based on the relative position and posture; Constructing a third error function; The second posture data is optimized using the positioning observation information, the motion observation information, the visual observation information, the laser observation information and the third error function to determine the target posture.

6. The method according to claim 5, characterized in that The visual observation information includes visual residuals for visual feature points; the visual feature points are extracted from the image information; and the visual residuals are determined by the following steps: Determine a position observation value of the visual feature point in the image information; Determine that the visual feature point is located at a feature point station center position in a preset station center coordinate system; Determining a position prediction value of the visual feature point according to the center position of the feature point and preset camera projection information; Determining the visual residual according to the position observation value and the position prediction value; Wherein, determining the position prediction value of the visual feature point according to the center position of the feature point and preset camera projection information includes: Determine the feature point distance from the visual feature point to the camera origin according to the feature point center position; Determine the feature point angle according to the feature point distance; the feature point angle is the angle between the position of the visual feature point in the station center coordinate system and the camera specified direction axis; Determining depth distortion information according to the angle of the feature points; The position prediction value is determined according to the depth distortion information, the center position of the feature point and the camera projection information.

7. The method according to claim 1, characterized in that The generating a relative pose based on the point cloud information, the IMU information, and the second pose data comprises: Constructing an initial model for state prediction based on the IMU information; Generate a laser observation model based on the point cloud information, the IMU information and the second pose data; Determine the degree of laser observation degradation at the current location; Determining a velocity observation model based on the second posture data and the laser observation degradation degree; Generate a laser inertial odometer update model based on the state prediction initial model, the laser observation model, and the speed observation model; The laser inertial odometer is used to update the model and output relative pose.

8. The method according to claim 7, characterized in that The determining of the velocity observation model based on the laser observation degradation degree comprises: Determine the rotation of the IMU coordinate system in the second pose data relative to the station center coordinate system and the speed of the IMU coordinate system relative to the station center coordinate system; Constructing a station center velocity vector using the rotation of the IMU coordinate system relative to the station center coordinate system and the velocity of the IMU coordinate system relative to the station center coordinate system; Determine the rotation of the IMU coordinate system relative to a preset world coordinate system and the speed of the IMU coordinate system relative to the world coordinate system; Constructing a world velocity vector using the rotation of the IMU coordinate system relative to a preset world coordinate system and the velocity of the IMU coordinate system relative to the world coordinate system; Determining a velocity observation vector according to the station center velocity vector and the world velocity vector; Determining a degradation weighted value corresponding to the laser observation degradation degree; A speed observation model is constructed according to the product of the degradation weighted value and the speed observation vector.

9. The method according to claim 7, characterized in that: The state prediction initial model is used to output the state difference matrix between the predicted state vector and the current state vector of the IMU, as well as the state weight value; The laser observation model is used to output the distance error matrix from the source end to the preset map in the current point cloud information; The velocity observation model is used to output a velocity difference matrix between a velocity vector in the current station center coordinate system and a velocity vector in the current world coordinate system; The laser inertial odometer update model is used to determine the target state vector when the sum of the first calculated value, the second calculated value and the third calculated value is the minimum value; Among them, the first calculated value is the product of the state difference matrix and its transpose multiplied by the state weight value, the product of the distance error matrix and its transpose multiplied by the preset distance observation weight, and the product of the speed difference matrix and its transpose multiplied by the preset speed observation weight.

10. The method according to claim 7, characterized in that The state prediction initial model is obtained based on the IMU information and the laser inertial odometer model based on iterative Kalman filtering; the laser inertial odometer model based on iterative Kalman filtering is provided with a state vector, and the state vector includes: The rotation of the IMU coordinates and the preset world coordinate system, the translation of the IMU coordinates and the world coordinate system, the speed of the IMU coordinate system in the world coordinate system, the angular velocity zero bias of the IMU, and the acceleration zero bias of the IMU; Among them, the IMU coordinate system is constructed based on the IMU module, the IMU coordinates are the positions of the IMU module in the IMU coordinate system, and the IMU module is a device for measuring IMU information.

11. An electronic device, characterized in that: The electronic device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the method as claimed in any one of claims 1 to 10.

Images