Pose determination method and device for autonomous vehicle and electronic equipment

Abstract

The embodiment of the invention provides a pose determination method and device for an autonomous vehicle and electronic equipment. The method comprises the following steps: determining a visual pose increment based on a current image and a historical image collected by a camera, and determining an inertial pose increment based on inertial data; fusing the visual pose increment and the inertial pose increment to obtain a target pose increment; and finally, determining the current pose based on the initial pose and the target pose increment. The method is used for achieving the effects of combining the advantages of the visual pose and the inertial pose in automatic driving and improving the automatic driving positioning precision.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving, and in particular to a method, device and electronic equipment for determining the posture of an autonomous driving vehicle. Background Art

[0002] As driverless technology is gradually applied to vehicles, the navigation and positioning technologies required for driverless driving are also facing higher technical requirements.

[0003] As relatively mature navigation and positioning technologies, visual odometry and inertial odometry are more widely used in the field of autonomous driving. Among them, visual odometry uses visual information to estimate position, while inertial odometry uses inertial sensors to estimate position.

[0004] However, visual odometry has scene limitations. When there are many dynamic scenes, the positioning effect is poor. Inertial odometry has high requirements for components. Low-cost components will lead to increased errors. Therefore, existing navigation and positioning technologies are costly and have low accuracy. Summary of the invention

[0005] The embodiments of the present application provide a method, device and electronic device for determining the posture of an autonomous driving vehicle, so as to improve the positioning accuracy of the autonomous driving.

[0006] In a first aspect, an embodiment of the present application provides a method for determining a posture of an autonomous driving vehicle, comprising:

[0007] Obtaining a visual pose increment and an inertial pose increment, wherein the visual pose increment is determined based on a current image and a historical image captured by a camera, and the inertial pose increment is determined by inertial data;

[0008] Fusing the visual pose increment and the inertial pose increment to obtain a target pose increment;

[0009] A current pose is determined based on the initial pose and the target pose increment.

[0010] In a possible implementation, obtaining a visual pose increment includes:

[0011] The current image captured by the camera is input into a visual odometer, and the visual odometer determines a visual pose increment according to the current image and historical images.

[0012] In a possible implementation, the visual odometer determines the visual pose increment based on the following steps:

[0013] Extracting a first feature point from the current image;

[0014] extracting a second feature point from the historical image;

[0015] Determine a feature point matching pair between a second feature point of the historical image and a first feature point of the current image;

[0016] Based on the feature point relationship between the feature point matching pair and the camera intrinsic parameter matrix, an essential matrix is ​​calculated;

[0017] The visual pose increment corresponding to the current image is parsed from the essential matrix.

[0018] In a possible implementation, the visual odometer determines the visual pose increment based on the following steps, further comprising:

[0019] Based on bundle adjustment, the temporal pose increments corresponding to multiple frames of images, including the visual pose increment corresponding to the current image, are optimized with the goal of minimizing the reprojection error.

[0020] The optimized visual pose increment corresponding to the current image is used as the final visual pose increment of the current image.

[0021] In a possible implementation, obtaining an inertial posture increment includes:

[0022] Get the current inertial data from the inertial measurement unit;

[0023] An inertial posture increment is determined according to the current inertial data and the historical inertial data.

[0024] In a possible implementation, the inertial measurement unit calculates the inertial pose increment based on the following steps:

[0025] The acceleration in the inertial data is integrated within the period corresponding to the current moment to obtain the position increment, and the angular velocity is integrated to obtain the attitude increment.

[0026] In a possible implementation, the fusing the visual pose increment and the inertial pose increment to obtain a target pose increment includes:

[0027] The visual pose increment is used as an observation quantity, and the inertial pose increment is used as a prediction quantity, which are input into a filter, and the filter outputs a target pose increment.

[0028] In a possible implementation, the filter fuses the visual pose increment and the inertial pose increment based on the following steps:

[0029] Predicting the current moment's posture increment based on the historical inertial posture increment and the historical state to obtain the current moment's inertial posture increment prediction;

[0030] Calculate the difference between the visual pose increment and the predicted inertial pose increment at the current moment as a result of the preset observation model acting on the visual pose increment;

[0031] A fusion result is determined according to the preset gain of the filter, the difference and the historical state.

[0032] In a possible implementation, the filter is a Kalman filter.

[0033] In a second aspect, an embodiment of the present application provides a posture determination device for an autonomous driving vehicle, comprising:

[0034] An acquisition unit, used to acquire a visual pose increment and an inertial pose increment, wherein the visual pose increment is determined based on a current image and a historical image captured by a camera, and the inertial pose increment is determined by inertial data;

[0035] A fusion unit, used for fusing the visual pose increment and the inertial pose increment to obtain a target pose increment;

[0036] A determination unit is used to determine a current posture based on an initial posture and the target posture increment.

[0037] In a possible implementation, the acquiring unit includes:

[0038] The first determination module is used to input the current image captured by the camera into the visual odometer, and the visual odometer determines the visual pose increment according to the current image and historical images.

[0039] In a possible implementation, the visual odometer determines the visual pose increment based on the following steps:

[0040] Extracting a first feature point from the current image;

[0041] extracting a second feature point from the historical image;

[0042] Determine a feature point matching pair between a second feature point of the historical image and a first feature point of the current image;

[0043] Based on the feature point relationship between the feature point matching pair and the camera intrinsic parameter matrix, an essential matrix is ​​calculated;

[0044] The visual pose increment corresponding to the current image is parsed from the essential matrix.

[0045] In a possible implementation, the visual odometer determines the visual pose increment based on the following steps, further comprising:

[0046] Based on bundle adjustment, the temporal pose increments corresponding to multiple frames of images, including the visual pose increment corresponding to the current image, are optimized with the goal of minimizing the reprojection error.

[0047] The optimized visual pose increment corresponding to the current image is used as the final visual pose increment of the current image.

[0048] In a possible implementation, the acquiring unit includes:

[0049] An acquisition module is used to obtain current inertial data from an inertial measurement unit;

[0050] The second determination module is used to determine the inertial posture increment according to the current inertial data and the historical inertial data.

[0051] In a possible implementation, the inertial measurement unit calculates the inertial pose increment based on the following steps:

[0052] The acceleration in the inertial data is integrated within the period corresponding to the current moment to obtain the position increment, and the angular velocity is integrated to obtain the attitude increment.

[0053] In a possible implementation, the fusion unit includes:

[0054] A processing module is used to input the visual pose increment as an observed quantity and the inertial pose increment as a predicted quantity into a filter, and the filter outputs a target pose increment.

[0055] In a possible implementation, the filter fuses the visual pose increment and the inertial pose increment based on the following steps:

[0056] Predicting the current moment's posture increment based on the historical inertial posture increment and the historical state to obtain the current moment's inertial posture increment prediction;

[0057] Calculate the difference between the visual pose increment and the predicted inertial pose increment at the current moment as a result of the preset observation model acting on the visual pose increment;

[0058] A fusion result is determined according to the preset gain of the filter, the difference and the historical state.

[0059] In a possible implementation, the filter is a Kalman filter.

[0060] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0061] The memory stores computer-executable instructions;

[0062] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0063] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0064] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0065] The embodiments of the present application provide a method, device and electronic device for determining the posture of an autonomous driving vehicle. The method determines the visual posture increment through the current image and historical images collected by the camera, and determines the inertial posture increment based on the inertial data; then the visual posture increment and the inertial posture increment are fused to obtain the target posture increment; finally, the current posture is determined based on the initial posture and the target posture increment, and the visual posture increment and the inertial posture increment are fused, so that the high precision of the visual posture increment in positioning in static scenes can compensate for the positioning error of the inertial posture increment, and the high precision of the inertial posture increment in positioning in dynamic scenes can compensate for the positioning error of the visual posture increment, and the two complement each other, and at the same time reduce the influence of the environment and IMU components on the final posture positioning result, so that the posture result has more application scenarios and the posture positioning accuracy is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0067] Figure 1 A schematic diagram of a method for determining the position and posture of an autonomous vehicle provided in this application Figure 1 ;

[0068] Figure 2 A schematic diagram of a method for determining the position and posture of an autonomous vehicle provided in this application Figure 2 ;

[0069] Figure 3 A schematic diagram of the structure of a posture determination device for an autonomous driving vehicle provided in the present application;

[0070] Figure 4 A schematic diagram of the structure of an electronic device provided in this application.

[0071] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0072] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0073] As driverless technology is gradually put into use, the navigation and positioning required by driverless technology are also facing higher technical requirements. It is necessary to continuously improve the positioning accuracy of existing technologies in various scenarios so that driverless technology can be applied to more life scenarios.

[0074] In one example, a vehicle can achieve positioning requirements through a visual odometry, which can achieve good positioning results under the conditions of sufficient lighting, significantly more static scenes than dynamic scenes, sufficient texture, and sufficient overlapping areas between consecutive frame scenes.

[0075] However, in the above method, the visual odometer is easily affected by factors such as lighting changes, dynamic obstacles, and texture loss, resulting in feature point matching failure or error accumulation, thereby affecting the vehicle positioning accuracy.

[0076] In one example, a vehicle can achieve positioning requirements through an inertial odometer, which is usually used in dynamic scenarios. The inertial measurement unit collects the acceleration and angular velocity of the vehicle and integrates these data to determine the vehicle's position, speed and attitude information.

[0077] However, in the above method, the inertial measurement unit is a key component, and its own characteristics have a huge impact on the final positioning result. High-precision inertial measurement units are expensive, and civilian-grade inertial measurement units will have cumulative errors over time. The errors will continue to increase and cannot be eliminated, resulting in large deviations in vehicle positioning.

[0078] The present application provides a method for determining the position and posture of an autonomous driving vehicle, aiming to solve the above-mentioned problem.

[0079] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0080] Figure 1 A schematic diagram of a method for determining the position and posture of an autonomous vehicle provided in this application Figure 1 ,like Figure 1 As shown, the method includes:

[0081] S101. Obtain a visual pose increment and an inertial pose increment, wherein the visual pose increment is determined based on a current image and historical images captured by a camera, and the inertial pose increment is determined by inertial data.

[0082] Exemplarily, a set of feature points in the current image captured by the camera is obtained based on a feature extraction algorithm, and a set of feature points in the historical image frames is obtained based on the same feature extraction algorithm. Because if feature points are used to characterize a moving object, the movement of the object is the movement of the feature points, and the movement trajectory of the object is the movement trajectory of the feature points. Conversely, the movement trajectory of the feature points is the movement trajectory of the object. Therefore, the feature points on adjacent image frames can be matched, and the movement trajectory of the feature points can be obtained by matching feature points pairwise, thereby obtaining the movement trajectory of the corresponding object on the adjacent image frames.

[0083] According to the nearest neighbor algorithm, the feature points of the current image frame and the matching feature points of the historical image frame that meet the preset matching conditions are obtained, and then the matched feature point set is obtained. The pose increment is determined according to the feature point set, the camera intrinsic parameter matrix, and the essential matrix.

[0084] The posture increment is calculated through IMU (Inertial Measurement Unit) data. The IMU includes an accelerometer and a gyroscope, which measure the acceleration and angular velocity of the vehicle respectively, thereby calculating the inertial posture increment within the unit.

[0085] In one example, the visual odometer pose of the current image and the visual odometer pose of the historical image can be determined by the visual odometer calculation method based on the current image and the historical image captured by the camera, and the visual odometer pose of the current image and the visual odometer pose of the historical image are subtracted to obtain the visual pose increment. Similarly, the inertial odometer pose of the IMU data at the previous moment is calculated according to the inertial odometer calculation method, and the inertial odometer pose of the IMU data at the current moment is calculated according to the inertial odometer calculation method, and the inertial odometer pose of the current moment is calculated according to the inertial odometer calculation method, and the difference between the two is made to obtain the inertial pose increment at the current moment.

[0086] S102: Fusing the visual pose increment and the inertial pose increment to obtain a target pose increment.

[0087] Exemplarily, the visual pose increment is used as the observed quantity and the inertial pose increment is used as the predicted quantity, which are input into the filter to obtain the target pose increment based on the fusion of the visual pose increment and the inertial pose increment.

[0088] Because the error of inertial pose increment is usually certain, while the error of visual pose increment usually contains large uncertainty, therefore, taking inertial pose increment as prediction quantity and visual pose increment as observation quantity can give full play to the advantages of visual and inertial sensors and improve the performance of positioning system.

[0089] S103: Determine the current posture based on the initial posture and the target posture increment.

[0090] Exemplarily, the pose includes two parts, position and attitude, where the position refers to the three-dimensional coordinates of the object in space, that is, the specific position in space. In this embodiment, the pose refers to the specific position of the vehicle in space, and the attitude refers to the direction of the object relative to the reference coordinate system, usually expressed by a rotation matrix or Euler angles. In this embodiment, it refers to the orientation of the vehicle in space. Similarly, the pose increment also includes position and attitude, including changes in three-dimensional coordinates and changes in direction.

[0091] Therefore, the pose increments at each moment after fusion are accumulated to obtain the global pose increment, including the accumulated changes in the vehicle's three-dimensional coordinates at each moment, and the changes in the vehicle's rotation matrix or Euler angles at each moment in the reference coordinate system. The current pose is determined based on the accumulated results of the initial pose and the global pose increment.

[0092] An embodiment of the present application provides a method for determining the posture of an autonomous driving vehicle. The current image and historical images collected by a camera are used to determine a visual posture increment, and an inertial posture increment is determined based on inertial data. The visual posture increment and the inertial posture increment are then fused to obtain a target posture increment. Finally, the current posture is determined based on the initial posture and the target posture increment. The fusion of the visual posture increment and the inertial posture increment allows the high positioning accuracy of the visual posture increment in static scenes to compensate for the positioning error of the inertial posture increment. In dynamic scenes, the high positioning accuracy of the inertial posture increment compensates for the positioning error of the visual posture increment. The two complement each other and reduce the influence of the environment and IMU components on the final posture positioning results. This allows the posture results to be used in more scenarios and improves the posture positioning accuracy.

[0093] Figure 2 A schematic diagram of a method for determining the position and posture of an autonomous vehicle provided in this application Figure 2 ,like Figure 2 As shown, in this embodiment Figure 1 Based on the embodiment, a method for determining the position and posture of an autonomous driving vehicle is described in detail, and the method includes:

[0094] S201, inputting the current image captured by the camera into the visual odometer, and the visual odometer determines the visual pose increment according to the current image and historical images.

[0095] In one example, step S201 includes the following process:

[0096] Extract the first feature point from the current image.

[0097] The second feature point is extracted from the historical image.

[0098] Determine a feature point matching pair between the second feature points of the historical image and the first feature points of the current image.

[0099] Based on the feature point relationship between the feature point matching pair and the camera intrinsic parameter matrix, an essential matrix is ​​calculated; and a visual pose increment corresponding to the current image is parsed from the essential matrix.

[0100] Based on bundle adjustment, the temporal pose increments corresponding to multiple frames of images, including the visual pose increment corresponding to the current image, are optimized with the goal of minimizing the reprojection error.

[0101] The fine-tuned visual pose increment corresponding to the current image is used as the final visual pose increment of the current image.

[0102] Exemplarily, all image frames captured by the camera within a unit time length are obtained, and any two adjacent continuous image frames are a group, where the historical image is the previous frame of the current image. The feature points in the current image and the historical image are extracted respectively by the ORB (Oriented FAST and Rotated BRIEF) algorithm. Assuming that the historical image is the tth frame, the feature point set of the historical image is , that is, the second feature point is ,in, is the first feature points, N is the number of feature points in the historical image extracted by the ORB algorithm, and the current image is Frame, the feature point set of the current image is ,in, For the The first feature points, M is the number of feature points in the current image extracted by the ORB algorithm, that is, the first feature point is .

[0103] The first feature point in the historical image is matched with the second feature point in the current image through the nearest neighbor matching algorithm to obtain a matching pair that meets the preset conditions. , matching pair representation Feature points and The feature points are the feature points of the same object in different images. The set of matched feature points is , that is, the feature point matching pair is , Characterization Group matching pairs, K is the number of matching pairs of feature points between the historical image and the current image.

[0104] The camera intrinsic matrix represents the internal parameters of the camera, describes the optical characteristics of the camera and the geometric relationship of the imaging, and is calculated based on pre-test calibration. The essential matrix represents the rotation and translation relationship between the two camera coordinate systems. The relationship between the feature point matching pair, the camera intrinsic matrix Q, the historical image and the essential matrix E of the current image is as follows (1):

[0105] (1)

[0106] is the matching feature point matrix of the current image in the feature point matching pair, is the matching feature point matrix of the historical image in the feature point matching pair, and is the kth group of matching feature points in the historical image and the current image. According to the above relationship, the essential matrix E can be determined. Perform singular value decomposition on the essential matrix E to extract the visual pose increment. The visual pose increment consists of the pose increment R and the position increment L. The relationship between the essential matrix E and the pose increment R and the position increment L is as follows:

[0107] (2)

[0108] is the antisymmetric matrix of the position increment L.

[0109] For all image frames captured by the camera within a unit time length, the visual pose increments of all adjacent images are optimized by bundle adjustment (BA) to calculate the minimum reprojection error within the unit time length. The calculation process is as follows:

[0110] (3)

[0111] in, Represents the camera projection function. The posture increment R and position increment L corresponding to the minimized reprojection error are the final visual posture increment within the current unit time length.

[0112] S202, obtaining current inertial data from an inertial measurement unit; and determining an inertial posture increment according to the current inertial data and historical inertial data.

[0113] In one example, step S202 includes the following process:

[0114] The acceleration in the inertial data is integrated within the period corresponding to the current moment to obtain the position increment, and the angular velocity is integrated to obtain the attitude increment.

[0115] Exemplarily, the inertial odometer posture calculates the posture increment through IMU (Inertial Measurement Unit) data. The IMU includes an accelerometer and a gyroscope, which measure the acceleration and angular velocity of the vehicle respectively, thereby calculating the inertial posture increment within the unit.

[0116] Get the current inertial data, including the current The acceleration output by the IMU at this moment , angular velocity , acceleration Contains the vehicle's acceleration and angular velocity in three axes Indicates the rotation rate of the vehicle around three axes. Get historical inertial data, including the speed at a certain moment in history , and the pose matrix of the previous moment .

[0117] Through the angular velocity Integrate to get the attitude increment , the calculation process is as follows formula (4):

[0118] (4)

[0119] is the unit time step.

[0120] By integrating the acceleration, we get the position increment , the calculation process is as follows formula (5):

[0121] (5)

[0122] is the position increment at the current moment, and It is the inertial posture increment at the current moment.

[0123] S203, taking the visual pose increment as the observed quantity and the inertial pose increment as the predicted quantity, input them into the filter, and the filter outputs the target pose increment.

[0124] In one example, step S203 includes the following process:

[0125] The current moment's posture increment is predicted based on the historical inertial posture increment and the historical state, and the current moment's inertial posture increment prediction is obtained.

[0126] Calculate the difference between the visual pose increment and the predicted inertial pose increment at the current moment as a result of the preset observation model acting on it.

[0127] The fusion result is determined according to the preset gain, difference and historical state of the filter.

[0128] The filter is a Kalman filter.

[0129] For example, the error of inertial pose increment is usually certain, while the error of visual pose increment usually contains large uncertainty. Therefore, taking inertial pose increment as predictor and visual pose increment as observed quantity is based on comprehensive consideration of Kalman filter, sensor characteristics and practical applications. Through this design, the advantages of visual and inertial sensors can be fully utilized to improve the performance of the positioning system.

[0130] Through step S202, the inertial posture increment of the historical moment before the unit time length can be known. , based on the historical state of the historical moment before the unit time length calculated in real time by the filter , for the inertial posture increment and historical status Perform modeling prediction to obtain the predicted inertial posture increment at the current moment , the process is as follows formula (6):

[0131] (6)

[0132] The mapping relationship between the inertial posture increment, historical state, and state prediction at the historical moment obtained by calibration is represented. A vector space that represents the environment, time delay, and other parameters that affect the pose results. Represents the current pose increment predicted by the inertial odometer.

[0133] The visual pose increment at the current moment can be obtained through step S201 , increment the visual pose at the current moment As the observation, according to the visual pose increment at the current moment , the predicted amount of inertial posture increment at the current moment , Kalman filter preset gain , and historical status , determine the target pose increment after fusion , the process is as follows formula (7):

[0134] (7)

[0135] Preset Observation Model For alignment only and Dimension, preset gain of Kalman filter Between [0-1], because the visual information contained in the camera data contains more environmental features, the predicted state can be corrected by calculating the difference between the visual pose increment and the inertial pose increment predicted by the preset observation model at the current moment.

[0136] By accumulating the fused pose increments, the global pose update can be obtained. The process is as follows:

[0137] (8)

[0138] is the position increment per unit time after the above fusion process, is the initial position of the vehicle.

[0139] An embodiment of the present application provides a method for determining the posture of an autonomous driving vehicle. The method determines a visual posture increment based on a current image captured by a camera, determines an inertial posture increment based on current inertial data and historical inertial data acquired by an inertial measurement unit, inputs the visual posture increment and the inertial posture increment into a filter, obtains a target posture increment, and fuses the visual posture increment and the inertial posture increment. This allows the high-precision positioning of the visual posture increment in static scenes to compensate for the positioning error of the inertial posture increment, while the high-precision positioning of the inertial posture increment in dynamic scenes compensates for the positioning error of the visual posture increment. The two methods complement each other and reduce the influence of the environment and IMU components on the final posture positioning results. This allows the posture results to be used in more scenarios and improves the posture positioning accuracy.

[0140] Figure 3 A schematic diagram of a device for determining a posture of an autonomous vehicle provided in this application is shown in FIG. Figure 3 As shown, a posture determination device 30 for an autonomous driving vehicle provided in this embodiment includes:

[0141] An acquisition unit 301 is used to acquire a visual pose increment and an inertial pose increment, wherein the visual pose increment is determined based on a current image and a historical image captured by a camera, and the inertial pose increment is determined by inertial data;

[0142] A fusion unit 302 is used to fuse the visual pose increment and the inertial pose increment to obtain a target pose increment;

[0143] The determination unit 303 is used to determine the current posture based on the initial posture and the target posture increment.

[0144] In a possible implementation, the acquiring unit 301 includes:

[0145] The first determination module 3011 is used to input the current image captured by the camera into the visual odometer, and the visual odometer determines the visual pose increment according to the current image and the historical images.

[0146] In one possible implementation, the visual odometry determines the visual pose increment based on the following steps:

[0147] Extracting a first feature point from the current image;

[0148] Extracting a second feature point from the historical image;

[0149] Determine a feature point matching pair between a second feature point of the historical image and a first feature point of the current image;

[0150] Based on the feature point relationship between the feature point matching pair and the camera intrinsic parameter matrix, an essential matrix is ​​calculated;

[0151] Parse the visual pose increment corresponding to the current image from the essential matrix.

[0152] Based on bundle adjustment, the temporal pose increments corresponding to multiple frames of images, including the visual pose increment corresponding to the current image, are optimized with the goal of minimizing the reprojection error.

[0153] In one possible implementation, the visual odometer determines the visual pose increment based on the following steps, further comprising:

[0154] Based on bundle adjustment, the temporal pose increments corresponding to multiple frames of images, including the visual pose increment corresponding to the current image, are optimized with the goal of minimizing the reprojection error.

[0155] The fine-tuned visual pose increment corresponding to the current image is used as the final visual pose increment of the current image.

[0156] In a possible implementation, the acquiring unit 301 includes:

[0157] An acquisition module 3012 is used to acquire current inertial data from an inertial measurement unit;

[0158] The second determination module 3013 is used to determine the inertial posture increment according to the current inertial data and the historical inertial data.

[0159] In one possible implementation, the inertial measurement unit calculates the inertial pose increment based on the following steps:

[0160] The acceleration in the inertial data is integrated within the period corresponding to the current moment to obtain the position increment, and the angular velocity is integrated to obtain the attitude increment.

[0161] In a possible implementation, the fusion unit 302 includes:

[0162] The processing module 3021 is used to input the visual pose increment as the observed quantity and the inertial pose increment as the predicted quantity into the filter, and the filter outputs the target pose increment.

[0163] In one possible implementation, the filter fuses the visual pose increments and the inertial pose increments based on the following steps:

[0164] Predict the current moment's posture increment based on the historical inertial posture increment and historical state, and obtain the current moment's inertial posture increment prediction;

[0165] Calculate the difference between the visual pose increment and the predicted inertial pose increment at the current moment as a result of the preset observation model acting on it;

[0166] The fusion result is determined according to the preset gain, difference and historical state of the filter.

[0167] In a possible implementation, the filter is a Kalman filter.

[0168] This embodiment provides a posture determination device for an autonomous driving vehicle, which can execute the method provided in the above method embodiment. Its implementation principle and technical effects are similar, and are not described in detail in this embodiment.

[0169] Figure 4 This is a schematic diagram of the structure of an electronic device provided in this application. Figure 4 As shown, the electronic device 40 provided in this embodiment includes: at least one processor 401 and a memory 402. Optionally, the device 40 also includes a communication component 403. The processor 401, the memory 402 and the communication component 403 are connected via a bus 404.

[0170] In a specific implementation process, at least one processor 401 executes the computer-executable instructions stored in the memory 402, so that at least one processor 401 executes the above method.

[0171] The specific implementation process of the processor 401 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0172] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0173] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0174] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0175] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0176] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0177] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0178] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0179] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0180] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0181] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0182] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0183] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0184] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for determining the position and posture of an autonomous driving vehicle, characterized in that: include: Obtaining a visual pose increment and an inertial pose increment, wherein the visual pose increment is determined based on a current image and a historical image captured by a camera, and the inertial pose increment is determined by inertial data; Fusing the visual pose increment and the inertial pose increment to obtain a target pose increment; A current pose is determined based on the initial pose and the target pose increment.

2. The method according to claim 1, characterized in that The obtaining of visual pose increments comprises: The current image captured by the camera is input into a visual odometer, and the visual odometer determines a visual pose increment according to the current image and historical images.

3. The method according to claim 2, characterized in that The visual odometer determines the visual pose increment based on the following steps: Extracting a first feature point from the current image; extracting a second feature point from the historical image; Determine a feature point matching pair between a second feature point of the historical image and a first feature point of the current image; Based on the feature point relationship between the feature point matching pair and the camera intrinsic parameter matrix, an essential matrix is ​​calculated; The visual pose increment corresponding to the current image is parsed from the essential matrix.

4. The method according to claim 3, characterized in that The visual odometer determines the visual pose increment based on the following steps, further comprising: Based on bundle adjustment, the temporal pose increments corresponding to multiple frames of images, including the visual pose increment corresponding to the current image, are optimized with the goal of minimizing the reprojection error. The optimized visual pose increment corresponding to the current image is used as the final visual pose increment of the current image.

5. The method according to claim 1, characterized in that Get inertial pose increments, including: Get the current inertial data from the inertial measurement unit; An inertial posture increment is determined according to the current inertial data and the historical inertial data.

6. The method according to claim 5, characterized in that The inertial measurement unit calculates the inertial pose increment based on the following steps: Integrate the acceleration in the inertial data within the period corresponding to the current moment to obtain the position increment, and integrate the angular velocity to obtain the attitude increment.

7. The method according to any one of claims 1 to 6, characterized in that: The fusing the visual pose increment and the inertial pose increment to obtain a target pose increment includes: The visual pose increment is used as an observation quantity, and the inertial pose increment is used as a prediction quantity, which are input into a filter, and the filter outputs a target pose increment.

8. The method according to claim 7, characterized in that The filter fuses the visual pose increment and the inertial pose increment based on the following steps: Predicting the current moment's posture increment based on the historical inertial posture increment and the historical state to obtain the current moment's inertial posture increment prediction; Calculate the difference between the visual pose increment and the predicted inertial pose increment at the current moment as a result of the preset observation model acting on the visual pose increment; A fusion result is determined according to the preset gain of the filter, the difference and the historical state.

9. The method according to claim 7, characterized in that: The filter is a Kalman filter.

10. A device for determining the position and posture of an autonomous driving vehicle, characterized in that: include: An acquisition unit, used to acquire a visual pose increment and an inertial pose increment, wherein the visual pose increment is determined based on a current image and a historical image captured by a camera, and the inertial pose increment is determined by inertial data; A fusion unit, used for fusing the visual pose increment and the inertial pose increment to obtain a target pose increment; A determination unit is used to determine a current posture based on an initial posture and the target posture increment.

11. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 9 when executed by a processor.

13. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 9 when being executed by a processor.

Images