Pose correction method and apparatus

Abstract

A pose correction method and apparatus. The pose correction method comprises: acquiring a first pose of a vehicle (100) and first sensing data collected by a sensor outside a cabin when the vehicle (100) is in the first pose, wherein the first sensing data comprises information of one or more first road elements; on the basis of the first pose and the information of the one or more first road elements, acquiring information of a plurality of first directions; and on the basis of information of one or more road elements in a map and the information of the plurality of first directions, determining a second pose of the vehicle (100). In the method, first directions of a plurality of road elements at a current moment and information of road elements in a map are used to construct constraints to correct and optimize an initial pose of a vehicle (100), thereby facilitating improvement in the accuracy and real-time performance of pose correction, and thus obtaining a high-precision pose result of the vehicle (100).

Description

Posture Correction Method and Device Technical Field

[0001] This application relates to the field of intelligent driving technology, and more specifically, to a posture correction method and apparatus thereof. Background Technology

[0002] With societal development, more and more machines in modern life are becoming automated and intelligent, and cars for mobility are no exception; intelligent vehicles are gradually entering people's daily lives. In recent years, Advanced Driving Assistant Systems (ADAS) and Autonomous Driving Systems (ADS) have played a crucial role in intelligent vehicles. During long-distance driving, the onboard systems, such as odometers, accumulate pose drift errors when calculating position and attitude using recursive methods. To obtain more accurate vehicle position and attitude for intelligent driving, it is necessary to correct these accumulated pose drift errors in real time.

[0003] In areas with high-precision satellite positioning signals such as real-time kinematic (RTK) positioning technology, the initial vehicle position and attitude can usually be corrected using its high-precision pose positioning results. However, in areas where satellite positioning signals are lost, such as tunnels, canyons, under overpasses, and tree-lined avenues, other positioning methods are needed to correct and optimize the vehicle's pose in order to better utilize intelligent driving functions. Summary of the Invention

[0004] This application provides a pose correction method and apparatus, which helps to improve the accuracy of pose correction.

[0005] In a first aspect, a pose correction method is provided, the method comprising: acquiring a first pose of the vehicle and first sensing data collected by sensors outside the cockpit when the vehicle is in the first pose, the first sensing data including information of one or more first road elements; acquiring information of multiple first directions based on the first pose and the information of one or more first road elements; and determining a second pose of the vehicle based on the information of one or more road elements and the information of multiple first directions in the map.

[0006] Based on the above technical solution, the information of multiple first directions of road elements in the first sensor data when the vehicle is in the first pose is treated as a whole and combined with the information of road elements in the map to constrain and optimize the first pose of the vehicle. This helps to improve the accuracy and real-time performance of pose correction and obtain high-precision pose results for the vehicle.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, the first sensing data includes information on multiple first road elements. Before determining the second pose of the vehicle based on the information of one or more road elements in the map and the information of multiple first directions, the method further includes: matching the information of multiple road elements in the map with the information of multiple first directions to obtain a matching result; determining the second pose of the vehicle based on the information of one or more road elements in the map and the information of multiple first directions, including: determining the second pose based on the matching result, the information of multiple first directions, and the information of multiple road elements in the map.

[0008] Based on the above technical solution, by matching the information of multiple first directions of multiple road elements as a whole with the information of road elements in the map data, and constraining and optimizing the first pose of the vehicle based on the matching result, it is beneficial to improve the accuracy and real-time performance of pose correction and obtain high-precision pose results of the vehicle.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, information on multiple first directions is obtained based on the first pose and information on one or more first path elements, including: projecting the information on one or more first path elements onto a first plane to obtain information on multiple first directions.

[0010] Based on the above technical solution, by projecting the information of one or more first road elements onto the first plane, information of multiple first directions can be obtained based on the first pose and the information of one or more first road elements. This is beneficial for matching the information of multiple first directions as a whole with the information of road elements in the preset map, thereby facilitating the acquisition of high-precision vehicle pose results.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the first sensing data includes information about a first road element, and information about multiple first directions is obtained based on the first pose and information about one or more first road elements, including: obtaining information about multiple first directions based on at least two points where the distance between the first pose and the projection position of the first road element in the first plane is greater than or equal to a preset size.

[0012] Based on the above technical solution, by combining the information of multiple first directions in a road element with the information of road elements in the map data to constrain and optimize the first pose of the vehicle, it is beneficial to improve the accuracy and real-time performance of pose correction and obtain high-precision pose results of the vehicle.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, determining the second pose of the vehicle based on information from one or more road elements in the map and information from multiple first directions includes: determining the yaw angle and displacement of the second pose based on information from one or more road elements in the map and information from multiple first directions.

[0014] Based on the above technical solution, the yaw angle and displacement can be obtained from the road elements in the map and the information of the first direction, thereby obtaining the position and attitude of the second pose, which is beneficial to obtaining high-precision pose results of the vehicle.

[0015] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: acquiring a third pose of the vehicle and second sensing data collected by sensors outside the cockpit when the vehicle is in the third pose, the second sensing data including information of one or more second road elements; acquiring information of multiple second directions based on the third pose and the information of one or more second road elements; determining the second pose of the vehicle based on the information of one or more road elements in the map and the information of multiple first directions, including: determining multiple position information based on the intersection of the information of multiple first directions and the information of multiple second directions; and determining the second pose of the vehicle based on the information of one or more road elements in the map and the information of multiple position information.

[0016] Based on the above technical solution, multiple location information gathered from multiple first-direction and multiple second-direction information is combined as a whole with the information of road elements in the map, which helps to further improve the accuracy of position correction and thus obtain a more stable vehicle position correction effect.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, determining the second pose of the vehicle based on information from one or more road elements and multiple location information in the map includes: determining the yaw angle of the second pose based on information from one or more road elements and multiple location information in the map; and determining the displacement of the second pose based on information from one or more road elements, multiple location information, and the yaw angle in the map.

[0018] Based on the above technical solution, the yaw angle can be determined according to the information of road elements in the map and multiple location information. The displacement can be determined according to the information of road elements, location information and yaw angle in the map, thereby obtaining the position and attitude of the second pose, which is beneficial to obtaining high-precision pose results of the vehicle.

[0019] Secondly, a pose correction device is provided, comprising: an acquisition unit for acquiring a first pose of the vehicle and first sensing data collected by sensors outside the cockpit when the vehicle is in the first pose, the first sensing data including information of one or more first road elements; acquiring information of multiple first directions based on the first pose and the information of one or more first road elements; and a determination unit for determining a second pose of the vehicle based on the information of one or more road elements in the map and the information of multiple first directions.

[0020] In conjunction with the second aspect, in some implementations of the second aspect, the first sensing data includes information on multiple first road elements. Before determining the second pose of the vehicle based on the information of one or more road elements in the map and the information of multiple first directions, the device further includes: a matching unit, used to match the information of multiple road elements in the map with the information of multiple first directions to obtain a matching result; and a determining unit, specifically used to: determine the second pose based on the matching result, the information of multiple first directions, and the information of multiple road elements in the map.

[0021] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition unit is specifically used to: project the information of one or more first road elements onto a first plane to acquire information of multiple first directions.

[0022] In conjunction with the second aspect, in some implementations of the second aspect, the first sensing data includes information about a first road element. The acquisition unit is specifically used to: acquire information about multiple first directions based on at least two points where the distance between the first pose and the projection position of the first road element in the first plane is greater than or equal to a preset size.

[0023] In conjunction with the second aspect, in some implementations of the second aspect, the unit is specifically used to: determine the yaw angle and displacement of the second pose based on information from one or more road elements in the map and information from multiple first directions.

[0024] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition unit is further configured to: acquire the third pose of the vehicle and second sensing data collected by sensors outside the cockpit when the vehicle is in the third pose, the second sensing data including information of one or more second road elements; acquire information of multiple second directions based on the third pose and the information of one or more second road elements; the determination unit is specifically configured to: determine multiple location information based on the intersection of information of multiple first directions and information of multiple second directions; determine the second pose of the vehicle based on information of one or more road elements in the map and multiple location information.

[0025] In conjunction with the second aspect, in some implementations of the second aspect, the unit is specifically used to: determine the yaw angle of the second pose based on information of one or more road elements and multiple location information in the map; and determine the displacement of the second pose based on information of one or more road elements, multiple location information, and the yaw angle in the map.

[0026] Thirdly, a pose correction device is provided, comprising: a memory for storing a computer program; and a processor for executing the computer program stored in the memory, so that the device performs a method corresponding to any of the implementations in the first aspect above.

[0027] Fourthly, a vehicle is provided, which includes a pose correction device corresponding to any of the implementation methods of the second or third aspect above.

[0028] Fifthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a computer, enables the implementation of a method corresponding to any of the implementations in the first aspect above.

[0029] Sixthly, a computer program product is provided, comprising computer program code, which, when run on a computer, enables the computer to implement a method corresponding to any of the implementation methods in the first aspect above.

[0030] In a seventh aspect, a chip is provided, comprising: a circuit for performing a method corresponding to any of the implementations in the first aspect above. Attached Figure Description

[0031] Figure 1 is a functional schematic diagram of a vehicle 100 according to an embodiment of this application.

[0032] Figure 2 is a schematic block diagram of an intelligent driving system according to an embodiment of this application.

[0033] Figure 3 is a schematic diagram of the pose error of a vehicle according to an embodiment of this application.

[0034] Figure 4 is a schematic flowchart of a pose correction method 400 according to an embodiment of this application.

[0035] Figure 5 is a schematic diagram of the camera center and road element detection box positions according to an embodiment of this application.

[0036] Figure 6 is a projection diagram of an embodiment of this application.

[0037] Figure 7 is a schematic diagram of map elements within a first range near the first pose of a vehicle according to an embodiment of this application.

[0038] Figure 8 is a schematic diagram of directional information constraint according to an embodiment of this application.

[0039] Figure 9 is a schematic diagram of the camera center and road element detection box positions according to an embodiment of this application.

[0040] Figure 10 is a schematic diagram of directional information constraint according to an embodiment of this application.

[0041] Figure 11 is a schematic diagram of the location of converged road elements according to an embodiment of this application.

[0042] Figure 12 is a schematic flowchart of a pose correction method 1200 according to an embodiment of this application.

[0043] Figure 13 is a schematic flowchart of a pose correction method 1300 according to an embodiment of this application.

[0044] Figure 14 is a schematic block diagram of a pose correction device 1400 according to an embodiment of this application. Detailed Implementation

[0045] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0046] Figure 1 is a functional schematic diagram of a vehicle 100 according to an embodiment of this application. The vehicle 100 may include a sensing system 110, a computing platform 120, and a display device 130. The sensing system 110 may include one or more sensors for sensing information about the environment surrounding the vehicle 100. For example, the sensing system 110 may include a positioning system, which may be a Global Positioning System (GPS), a BeiDou system, or another positioning system. As another example, the sensing system 110 may include one or more of the following: an inertial measurement unit (IMU), an accelerometer, a lidar, a millimeter-wave radar, an ultrasonic radar, and a camera device (including a camera sensor, a camera, etc.).

[0047] Some or all of the functions of vehicle 100 can be controlled by computing platform 120. Computing platform 120 may include one or more processors, such as processors 121 to 12n (n being a positive integer). A processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In reconfigurable hardware circuits, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement some or all of the functions of the aforementioned units. Furthermore, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), etc. In addition, the computing platform 120 may also include a memory for storing instructions. Some or all of the processors 121 to 12n can call the instructions in the memory to implement the corresponding functions.

[0048] The in-cabin display devices 130 are mainly divided into two categories: the first is the in-vehicle display screen; the second is the projection display screen, such as the head-up display (HUD). An in-vehicle display screen is a physical display screen and an important component of the in-vehicle infotainment system. Multiple displays can be installed in the cabin, such as the digital instrument cluster display, the central control screen, the display screen in front of the front passenger (also known as the front-seat passenger), the display screen in front of the left rear passenger, the display screen in front of the right rear passenger, and even the car window can be used as a display screen. A head-up display, also known as a head-up display system, is mainly used to display driving information such as speed and navigation on a display device in front of the driver (such as the windshield). This reduces the driver's eye-shift time, avoids pupil changes caused by eye-shifting, and improves driving safety and comfort. Examples of HUDs include combiner-HUD (C-HUD) systems, windshield-HUD (W-HUD) systems, and augmented reality HUD (AR-HUD) systems. It should be understood that HUDs can also evolve into other types of systems as technology progresses, and this application does not limit them.

[0049] The above description of the display device 130 uses an in-vehicle display screen and a projection display screen as examples, but the embodiments of this application are not limited thereto. For example, the display device 130 can also be a light display screen or a projection screen.

[0050] Optionally, the structure of the vehicle 100 described above is merely illustrative. In actual applications, various components of the vehicle 100 may be added or removed as needed.

[0051] Vehicle 100 may include an intelligent driving system, which may include an advanced driver assistance system and an autonomous driving system. The intelligent driving system uses various sensors on the vehicle (including but not limited to: lidar, millimeter-wave radar, camera devices (also known as camera sensors), ultrasonic sensors, global positioning system, inertial measurement unit) to acquire information from the vehicle's surroundings, and analyzes and processes the acquired information to achieve functions such as obstacle perception, target recognition, vehicle positioning, path planning, driver monitoring / alerts, etc., thereby improving the safety, automation and comfort of driving the vehicle.

[0052] For example, Figure 2 shows a schematic block diagram of an intelligent driving system according to an embodiment of this application. The intelligent driving system may include three functional modules: a perception module 210, a planning module 220, and a control module 230. The perception module 210 perceives the environment surrounding the vehicle through sensors and outputs corresponding perception data to the planning module 220. The planning module 220 obtains information such as road topology and target objects based on the information acquired by the perception module 210. Based on the road topology and target object information, the planning module 220 can determine the vehicle's pose and a planned trajectory over a period of time. The planning module 220 can send this planned trajectory to the control module 230. After receiving the planned trajectory from the planning module 220, the control module 230 can output control signals to control actuators to take corresponding actions, such as steering, acceleration, and deceleration.

[0053] The above-mentioned sensing module 210 can be the above-mentioned sensing system 110, and the planning module 220 and the control module 230 can be located in the above-mentioned computing platform 120.

[0054] The above-mentioned sensing module 210 can also be located in the computing platform 120.

[0055] Vehicle-based driving automation systems are classified into five levels (or L0-L5) based on the degree to which they can perform dynamic driving tasks, according to the role allocation in performing these tasks and the presence or absence of an operational design domain (ODD), such as the external conditions (road, traffic, weather, lighting, etc.) defined during the system's design. Levels 0-2 represent driver assistance, where the system assists humans in performing dynamic driving tasks, but the driver remains the primary driver. Levels 3-5 represent autonomous driving, where the system performs dynamic driving tasks in place of the human under the designed operating conditions; when activated, the system becomes the primary driver. The names and definitions of each level are as follows:

[0056] Level 0 driving automation (also known as emergency assistance) systems cannot continuously perform lateral or longitudinal motion control of the vehicle during dynamic driving tasks, but they possess the ability to continuously perform partial target and event detection and response during dynamic driving tasks. Level 1 driving automation (also known as partial driver assistance) systems continuously perform lateral or longitudinal motion control of the vehicle during dynamic driving tasks under their design operating conditions, and possess the ability to perform partial target and event detection and response adapted to the performed lateral or longitudinal motion control. Level 2 driving automation (also known as combined driver assistance) systems continuously perform lateral and longitudinal motion control of the vehicle during dynamic driving tasks under their design operating conditions, and possess the ability to perform partial target and event detection and response adapted to the performed lateral and longitudinal motion control. Level 3 driving automation (also known as conditionally automated driving) systems continuously perform all dynamic driving tasks under their design operating conditions. Level 4 driving automation (also known as highly automated driving) systems continuously perform all dynamic driving tasks under their design operating conditions and automatically execute minimum risk strategies. Level 5 driving automation (also known as fully automated driving) systems continuously perform all dynamic driving tasks and automatically execute minimum risk strategies under any drivable conditions.

[0057] During long-distance, long-duration driving of intelligent driving vehicles, the vehicle's position and attitude may accumulate pose drift errors when onboard algorithms such as odometer calculation calculate the vehicle's position and attitude using a recursive method. Figure 3 shows a schematic diagram of vehicle pose error according to an embodiment of this application. As shown in Figure 3, the vehicle's current pose and true pose are projected onto a first plane. The black rectangular area represents the vehicle's true pose at the current moment, and the gray rectangular area represents the vehicle's calculated current pose. The error between the current pose and the true pose includes lateral error, longitudinal error, and attitude angle error. The lateral error refers to the position error perpendicular to the vehicle's driving direction, the longitudinal error refers to the position error parallel to the vehicle's driving direction, and the attitude angle error refers to the error caused by the vehicle's front-facing orientation. In order to obtain higher accuracy vehicle position and attitude during vehicle driving, enabling the vehicle to more accurately locate and plan its driving trajectory, it is necessary to correct the vehicle pose in real time. For example, the vehicle pose in Figure 3 needs to be corrected from the gray, error-laden current pose to the true pose at the black rectangle.

[0058] In areas with high-precision satellite positioning signals such as RTK, the initial vehicle position and attitude can usually be corrected using their high-precision positioning and attitude determination results. However, in areas where satellite positioning signals are lost, such as tunnels, canyons, under overpasses, and tree-lined avenues, other positioning methods are needed to correct and optimize the vehicle's position and attitude, so as to better utilize intelligent driving functions.

[0059] In one implementation, vehicle posture correction can be achieved by comparing the positions of road elements perceived and reconstructed by the vehicle during intelligent driving with the positions of road elements in a pre-set map.

[0060] For example, vehicle pose correction can be performed based on laser point cloud reconstruction. The accuracy of vehicle pose correction is affected by the extrinsic parameter calibration accuracy between the laser sensor and the camera device outside the vehicle cabin. When using a laser sensor, the position of road elements needs to be within the coverage area of ​​the laser sensor; otherwise, there are no laser points, and the position of road elements cannot be reconstructed. Therefore, this method is limited by the field of view of the laser point cloud. In addition, after the laser points scan the surface of a highly reflective target, the point cloud image will diffuse in all directions, making the original target point cloud image appear larger. When using a laser point cloud of the wrong depth, the position reconstruction of some road elements at different depths in adjacent viewing directions will lead to position reconstruction errors, further resulting in vehicle pose correction errors.

[0061] For example, vehicle pose correction can be performed based on the reconstructed road element positions using a purely visual method. Reconstructing road element positions using a purely visual method requires detecting and referencing road elements across multiple frames of images. This multi-frame visual reconstruction of road element positions has certain requirements regarding vehicle displacement and the number of observation frames for road elements. Often, the vehicle needs to travel for a period of time before the road element positions can be reconstructed for vehicle pose correction. Therefore, pose correction using this method has a certain time lag. When detecting and referencing road elements across multiple frames, if the reference is lost or the number of detection frames is insufficient, the road element positions cannot be reconstructed, or the reconstruction results will be poor. Furthermore, when the road element is far from the vehicle, the reconstruction of the position of a distant road element becomes an ill-conditioned multi-frame triangulation problem due to factors such as the small line-of-sight angle between multi-frame observations. Therefore, it is difficult to accurately estimate the road element positions in this situation. Multi-frame triangulation is the process of calculating the three-dimensional (3D) coordinates of a point in the world coordinate system by observing the same point from multiple camera frames. When using inaccurate road element positions for vehicle pose correction, it is difficult to obtain correct correction results.

[0062] In one implementation, the 3D point positions of all road elements in a pre-set map can be projected onto two-dimensional (2D) points in the vehicle camera image based on the initial vehicle pose. Then, 3D-2D perspective geometric constraints are constructed to optimize and correct the vehicle pose. This method requires projecting the positions of road elements in the map onto the camera image using the initial vehicle pose. Therefore, a slightly biased initial vehicle pose can lead to projection errors causing mismatches, or projections outside the image causing missed matches, resulting in incorrect vehicle poses. Furthermore, road elements of the same category in the map with similar line-of-sight directions but different depths will be projected onto the camera image at similar 2D positions. Finding matching 2D points in the camera image based on this distance can easily lead to 3D-2D matching errors, which in turn result in incorrect vehicle poses.

[0063] To address the issues of poor accuracy and real-time performance in the aforementioned vehicle posture correction methods, this application provides a posture correction method that can improve the accuracy and real-time performance of vehicle posture correction, thereby increasing the success rate of vehicle posture correction.

[0064] Figure 4 shows a schematic flowchart of a pose correction method 400 according to an embodiment of this application. This method 400 can be executed by the vehicle 100, or by the computing platform 120, or by a system-on-a-chip (SoC) in the computing platform 120, or by a processor, chip, or circuit in the computing platform 120, or by the intelligent driving system. The following embodiments use an intelligent driving system as an example for illustration. The method 400 includes:

[0065] S410: Acquire the first position of the vehicle and the first sensing data collected by sensors outside the cockpit when the vehicle is in the first position. The first sensing data includes information on one or more first road elements.

[0066] For example, the first pose can be the initial pose of the vehicle that needs to be corrected, which can be obtained through onboard methods such as odometer calculation. The sensor outside the cabin can be the camera sensor located outside the vehicle cabin in the perception system 110 in Figure 1, and the first sensing data can be the camera image captured when the vehicle is in the first pose.

[0067] For example, the first road element can be road elements around the vehicle in the camera image, such as traffic lights, traffic signs, utility poles, streetlights, etc.

[0068] Figure 5 shows a schematic diagram of the camera center and road element detection box positions according to an embodiment of this application. As shown in Figure 5, the outermost black box represents the image range acquired by the camera sensor outside the vehicle cabin at the current moment, and the gray squares represent the road element detection boxes determined according to the position and shape of the road elements. According to the shape of the road elements, the first road elements can be divided into planar road elements and pole-shaped road elements. Planar road elements can be traffic lights, traffic signs, etc., and pole-shaped road elements can be utility poles, streetlights, etc.

[0069] S420: Based on the information of the first pose and one or more first path elements, obtain information on multiple first directions.

[0070] For example, the information of the first direction can be the direction or angle between the first pose and each first path element in the camera image.

[0071] Optionally, obtaining information about multiple first directions based on the first pose and information about one or more first path elements may include: projecting information about one or more first path elements onto a first plane to obtain information about multiple first directions.

[0072] For example, the first plane may be the plane at the height of the camera sensor when the vehicle is in the first pose, or the plane of the ground where the vehicle is located, or other planes parallel to the plane of the ground where the vehicle is in the first pose. This application does not limit the height of the first plane.

[0073] S430: Determine the vehicle's second pose based on information from one or more road elements on the map and information from multiple first directions.

[0074] For example, the second pose can be the vehicle's actual pose or the corrected pose calculated based on the first pose. The information of the road elements in the map can include the type, location, size, etc. of the road elements, or it can include the location and size information of multiple points of a road element. These points can be the center point, corners, midpoints of each edge, etc. of the road element.

[0075] The second pose of the vehicle can be determined by matching information from one or more first directions in the first plane with information from one or more road elements in the map. The second pose is determined when multiple pieces of information from the first direction are closest to the information from one or more matching road elements in the map, or when the number of matching first direction information with the information from road elements in the map is the largest. Here, "matching" means that the information from the first direction and the information from road elements in the map are of the same category and correspond in position.

[0076] Optionally, if the first sensing data includes information on multiple first road elements, before determining the second pose of the vehicle based on the information of one or more road elements in the map and the information of multiple first directions, the method may further include: matching the information of multiple road elements in the map with the information of multiple first directions to obtain a matching result; determining the second pose of the vehicle based on the information of one or more road elements in the map and the information of multiple first directions, including: determining the second pose based on the matching result, the information of multiple first directions, and the information of multiple road elements in the map.

[0077] Figure 6 shows a projection diagram of an embodiment of this application, where the gray rectangular area represents the first pose and the black rectangular area represents the second pose. For example, at least one point within the road element detection boxes in Figure 5 can be projected onto a first plane. Based on the position of the projected point in the first plane and the position of the first pose in the first plane, information about multiple first directions can be obtained. This information about multiple first directions can be represented as five 2D rays with the position of the first pose as their endpoints, as shown by the dashed lines in Figure 6, including the first direction (pole) and the first direction (traffic light).

[0078] Figure 7 illustrates a schematic diagram of map elements within a first range near the first pose of a vehicle according to an embodiment of this application. Exemplarily, the first range can be a radius of 50m centered on the first pose. Alternatively, since the camera sensors outside the cockpit primarily acquire environmental information in the area in front of the vehicle's direction of travel, to save computing power, the first range can be a semicircle with a radius of 50m in front of the vehicle's direction of travel, plus a semicircle with a radius of 20m behind the vehicle's direction of travel. Alternatively, since most map elements are located within the road edge lines, in one embodiment, the first range can be a rectangular area within 50m in front of the vehicle's direction of travel, 20m behind the vehicle's direction of travel, and the road edge lines on both sides of the vehicle. The road edge lines can be solid lines on both sides of the lane, guardrails beside the road, the edges of green belts beside the road, curbs, etc.

[0079] The triangular area in Figure 7 represents information about multiple road elements in the map, including multiple map elements such as map elements (pole) and map elements (traffic lights). Optionally, the categories of these map elements can be further refined; for example, the map element (pole) can also be labeled as map element (streetlight) or map element (utility pole), etc.

[0080] Figure 8 shows a schematic diagram of a direction information constraint according to an embodiment of this application. Based on the information of multiple road elements in the map in Figure 7 and the information of multiple first directions, the second pose of the vehicle can be determined when the information of multiple first directions is closest to the information of one or more road elements in the matching map, or when the number of first direction information matches the information of road elements in the map is the largest.

[0081] Optionally, if the first sensing data includes information about a first road element, then based on the first pose and the information of one or more first road elements, information about multiple first directions is obtained, including: based on at least two points where the distance between the first pose and the projection position of the first road element in the first plane is greater than or equal to a preset size, information about multiple first directions is obtained.

[0082] For example, the preset size can be the minimum distance that the vehicle can recognize, which can be determined according to the vehicle's recognition accuracy or set manually by staff. For example, the preset size can be 1m.

[0083] Figure 9 shows a schematic diagram of the camera center and road element detection box position according to an embodiment of this application. Figure 10 shows a schematic diagram of a direction information constraint according to an embodiment of this application. As shown in Figure 9, the first sensing data includes a planar road element. Two points, the upper left and upper right corners of the planar road element, can be selected and projected onto a first plane to obtain information of two first directions. The information of the two first directions can be represented as two 2D rays with the position of the first pose as the endpoint, as shown by the dashed lines in Figure 10 (first direction 1 and first direction 2). Based on the information of these two first directions and the information of road elements in the map (as shown by map elements A and B in Figure 10, representing two points of the planar road element), when the information of multiple first directions is closest to the information of the matching road element in the map, the second pose of the vehicle can be determined.

[0084] Optionally, the second pose of the vehicle is determined based on information from one or more road elements in the map and information from multiple first directions, including: determining the yaw angle and displacement of the second pose based on information from one or more road elements in the map and information from multiple first directions.

[0085] For example, the yaw angle can be determined by the rotation matrix in the transformation matrix calculated when correcting the first attitude, and the displacement can be determined by the translation vector in the transformation matrix.

[0086] Optionally, the method 400 may further include: acquiring a third pose of the vehicle and second sensing data collected by sensors outside the cockpit when the vehicle is in the third pose, the second sensing data including information of one or more second road elements; acquiring information of multiple second directions based on the third pose and the information of one or more second road elements; determining the second pose of the vehicle based on the information of one or more road elements in the map and the information of multiple first directions, including: determining multiple position information based on the intersection of the information of multiple first directions and the information of multiple second directions; and determining the second pose of the vehicle based on the information of one or more road elements in the map and the information of multiple position information.

[0087] For example, the third pose can be another pose that is spaced a certain distance from the first pose, and this distance can be any value between 0.5m and 1m.

[0088] Figure 11 shows a schematic diagram of the location of converged road elements according to an embodiment of this application. In one implementation, after the vehicle has been traveling for a period of time, information on multiple directions of multiple poses of the vehicle at certain intervals during its travel can be obtained. As shown in Figure 11, the gray rectangles represent the vehicle's poses at different times, and the vehicle's travel direction is indicated by the arrows in the figure. According to the vehicle's travel time sequence, they are the first pose, the third pose, the fourth pose, and the fifth pose.

[0089] Taking the first and third poses as examples, the information in the first direction is shown as two 2D rays with the position of the first pose as the endpoint in Figure 11, and the information in the second direction is shown as five 2D rays with the position of the third pose as the endpoint in Figure 11. Based on the intersections of multiple first-direction and second-direction information, multiple positional information (positional information 1 and positional information 2 in Figure 11) are determined. By matching these multiple positional information as a whole with the information of one or more road elements in the map (map element 1 and map element 2 in Figure 11), the second pose of the vehicle can be determined.

[0090] Optionally, the second pose of the vehicle is determined based on information from one or more road elements and multiple location information in the map, including: determining the yaw angle of the second pose based on information from one or more road elements and multiple location information in the map; and determining the displacement of the second pose based on information from one or more road elements, multiple location information, and the yaw angle in the map.

[0091] Based on the above technical solution, by matching multiple first directions of one or more road elements with road elements in the map to construct constraints, it is beneficial to improve the accuracy of pose correction, thereby obtaining high-precision vehicle pose results.

[0092] Figure 12 shows a schematic flowchart of a pose correction method 1200 according to an embodiment of this application. As shown in Figure 12, the method 1200 may include steps such as acquiring initial data, constructing constraint information, correcting the initial pose of the vehicle, and outputting the correction result, as detailed below:

[0093] S1210: Obtain initial data.

[0094] Optionally, before acquiring the initial data, the method 1200 may further include: detecting that the satellite positioning signal has failed and that the vehicle's location meets the pose correction conditions.

[0095] To ensure the accuracy of pose correction, in one implementation, the pose correction condition can be that the number of road elements identified in the camera image and the number of road elements in the matching map are greater than or equal to two.

[0096] For example, as shown in Figure 12, the initial data may include the initial pose of the vehicle that needs to be corrected, the position of the road element detection box on the camera image when the vehicle is in the initial pose, and the transformation relationship between the camera sensor outside the vehicle cabin and the vehicle pose coordinate system. In one embodiment, the position of the camera sensor outside the vehicle cabin may be the position of the center or optical center of the camera sensor outside the vehicle cabin.

[0097] For example, the initial data may also include camera intrinsics from a camera sensor located outside the vehicle cabin. These camera intrinsics may include f... x f y c x c y , represented as a matrix Among them, f x f represents the length of the focal length along the x-axis, described in pixels. y This represents the length of the focal length along the y-axis, described in pixels; c x and c y The coordinates (principal point coordinates) of the center of the camera sensor in the pixel coordinate system are (c x c y ), c x and c y The unit is also pixels. The intrinsic parameters of the camera sensors outside the vehicle cabin reflect the camera's own properties and are fixed after the camera leaves the factory, and will not change during use.

[0098] Taking the road element detection box shown in Figure 5 as an example, planar road elements such as traffic lights and traffic signs can be simplified to the coordinates of their center point during subsequent calculations. Pole-shaped road elements such as streetlights and utility poles can be simplified to the straight line vector of their center line position during subsequent calculations.

[0099] For example, the initial pose of the vehicle requiring correction, and the transformation relationship between the camera sensor outside the vehicle cabin and the vehicle pose coordinate system, can be represented by a matrix. In one embodiment, the initial pose matrix of the vehicle requiring correction can be represented as follows: Where R world_veh The rotation matrix t represents the initial pose matrix of the vehicle that needs to be corrected. world_veh This represents the translation vector of the initial pose matrix for which vehicle correction is required. The pose transformation matrix is ​​a fixed value obtained from the vehicle's factory calibration, where R veh_cam The rotation matrix t represents the pose transformation matrix. veh_cam This represents the translation vector of the pose transformation matrix.

[0100] S1220: Construct constraint information.

[0101] As shown in Figure 12, step S1220 may further include the following sub-steps:

[0102] S1221: Based on the initial pose of the vehicle that needs to be corrected and the transformation relationship between the camera sensor outside the vehicle cabin and the vehicle pose coordinate system, calculate the initial pose of the camera sensor outside the vehicle cabin.

[0103] Based on the initial pose matrix and pose transformation matrix of the vehicle that need to be corrected, the initial pose matrix T of the camera sensor outside the vehicle cabin can be calculated using formula (1). world_cam :

[0104] Among them, R world_cam The rotation matrix t represents the initial pose matrix of the camera sensor outside the vehicle cockpit. world_cam This represents the translation vector of the initial pose matrix of the camera sensor outside the vehicle cabin.

[0105] S1222: Calculate the 3D plane formed by the centerline of the camera center and the centerline of the rod-shaped road element detection box and / or the 3D ray formed by the centerline of the camera center and the center point of the planar road element detection box.

[0106] Based on the transformation relationship, the initial pose matrix T of the vehicle needs to be corrected.world_veh If there is an error, then the initial pose matrix T of the camera sensor outside the vehicle cabin... world_cam There are also some errors, which need to be corrected.

[0107] For example, suppose the coordinates of a point on the center line of any rod-shaped road element detection box are (x, y) on the camera pixel plane. According to the equation of the line ax + by + c = 0, the center line of the rod-shaped road element detection box can be represented as line on the camera pixel plane. 3×1 =(a,b,c) T Where a and b are not both 0. As shown in Figure 5, this is combined with the intrinsic parameters f of the camera sensor outside the vehicle cabin. x f y c x c y The 3D plane formed by the center of the camera and the center line of the detection box of the observed rod-shaped road element at the current moment. 4×1 The parameters can be calculated using formula (2):

[0108] Assuming the center point of the detection box for any planar road element (e.g., a traffic light) has coordinates (u, v) on the camera pixel plane, the center point of the detection box for a planar road element can be represented as pt on the camera pixel plane. 3×1 =(u,v,1) T As shown in Figure 5, the intrinsic parameters f of the camera sensor located outside the vehicle cabin are combined. x f y c x c y The 3D ray L formed by the current camera center and the center point of the observed planar road element detection frame. 3×1 The parameters can be calculated using formula (3):

[0109] S1223: Project the above-mentioned 3D ray and / or 3D plane onto the first plane to obtain information in multiple first directions.

[0110] For example, based on the road element detection box obtained in Figure 5, formula (2) and formula (3), the 3D plane can be... 4×1 and 3D rays L 3×1Projected onto the first plane, information on multiple first directions is obtained (multiple 2D rays, as shown by the gray dashed lines in Figure 6). Each first direction has a corresponding road element category (traffic light or pole), indicating that the ray is calculated from the detection box of the corresponding road element and the camera center point. Optionally, the category of the first direction can be further refined; for example, the first direction (pole) can also be labeled as the first direction (streetlight) or the first direction (utility pole), etc.

[0111] S1224: Combine with a pre-set map, construct directional information constraints by satisfying the principle that "the distance between multiple first-direction information and one or more map elements in the matching map is the closest".

[0112] In this context, "matching" means that the information in the first direction and the map element indicate the same facility. For example, if the first direction indicates a traffic light, the map element also indicates the same traffic light.

[0113] For example, prior to step S1224, the method 1200 may further include: obtaining information on one or more road elements within a first range near the initial pose of the vehicle in a preset map.

[0114] As shown in Figure 8, in order to constrain the vehicle from its initial pose (equivalent to the first pose in method 400, the gray rectangular area) to its true pose (equivalent to the second pose in method 400, the black rectangular area), the initial pose of the camera sensor outside the vehicle cabin is optimized by satisfying the principle that the information of multiple first directions calculated by formulas (2) and (3) can cover the most matching map elements. When the vehicle is in the second pose, the direction information calculated by formulas (2) and (3) is shown by the thin black solid line in Figure 9, where each direction information can cover the corresponding matching map element.

[0115] For example, the information for each first direction obtained in step S1222 can be represented as a linear parametric form L. i =(a i ,b i ,c i ) T Assuming there are n pieces of information about the first direction in steps S1223, then the set of these n pieces of information about the first direction can be represented as L = {L1, ..., L...} i ,…,L n The 2D position of each map element that matches the information in the first direction can be represented as L′. i =(u′ i ,v′ i ) TIn this application, a rod-shaped map element, after being projected onto the first plane, appears as a point, and the location of this point can be used as the 2D position of the rod-shaped map element. A polygon-shaped map element, after being projected onto the first plane, appears as a straight line, and in this embodiment, the midpoint of this straight line is used as the 2D position of the polygon-shaped map element. Based on these 2D positions of map elements, the set of 2D positions of map elements on the preset map can be represented as L′={L′1,…,L′}. i ,…,L′ n}. Where n is an integer greater than 1, and i ranges from 1 to n.

[0116] In one implementation, a first transformation matrix can be used. Initial pose T of the camera sensor outside the vehicle cabin world_cam Corrective action is taken. Among them, R1 * The rotation matrix represents the first transformation matrix, and the yaw angle of the second pose can be determined based on the rotation matrix of the first transformation matrix; t1 * This represents the translation vector of the first transformation matrix, and the displacement of the second pose relative to the first pose can be determined based on the translation vector of the first transformation matrix.

[0117] Using the position of the camera sensor outside the vehicle cabin at the initial pose where the vehicle needs correction as the origin, each straight line L in the set L of information in the first direction is... i =(a i ,b i ,c i ) T Transform into a straight line vector L0 passing through the origin. i =(a i ,b i ) T Each 2D location L′ in the set L′ of 2D locations of map elements i =(u′ i ,v′ i ) T Convert to a 2D position L0′ passing through the origin. ′ =(u0′) i ,v0′ ′ ) T Theoretically, when the initial pose of the camera sensor outside the vehicle's cabin is completely error-free, meaning the initial pose for which the vehicle needs correction matches the vehicle's true pose, the straight-line vector L0... i The 2D position L0′ of the corresponding map element i =(u0′) i ,v0′ i ) T It should be located at L0 i On the straight line where L0′ is located, at this timei To L0 i The distance is 0. However, the initial pose error of the camera sensor outside the vehicle cabin causes the 2D position L0′ of the map element to be incorrect. i =(u0′) i ,v0′ ′ ) T to the corresponding straight line vector L0 i There is a certain distance, therefore it is necessary to use the first transformation matrix. rotation matrix R1 * For L0 i Perform the transformation.

[0118] Based on the principle in step S1224 that "the information from multiple first directions is closest to one or more map elements in the matching map," the 2D position L0′ of the map element is determined. i To the transformed straight line vector L0 i The distance should be as small as possible, from which we can obtain ||(T1) * ·L0 i ) T ·L0′ i || 2 The value should be close to 0. Based on this, directional information constraints can be constructed.

[0119] S1230: Initial vehicle position correction.

[0120] As shown in Figure 12, step S1230 may further include the following sub-steps:

[0121] S1231: Correct the initial pose of the camera sensor outside the vehicle cabin based on the orientation information constraint.

[0122] For example, the equation for optimizing the initial pose matrix of the camera sensor outside the vehicle cabin by combining information from multiple first directions can be found in the following formula (4):

[0123] The first transformation matrix T1 can be obtained by solving the equation in formula (4). * rotation matrix R1 * Translation vector t1 * Thus, the first transformation matrix can be obtained. According to formula (5), the initial pose T of the camera sensor outside the vehicle cabin can be determined. world_cam Perform correction and optimization:

[0124] Finally, the first pose matrix of the camera sensors outside the vehicle cabin after correction is obtained.

[0125] S1232: Calculate the true pose after correction based on the pose of the camera sensor outside the vehicle cabin after correction, the transformation relationship between the camera sensor outside the vehicle cabin and the vehicle pose coordinate system.

[0126] For example, combining the first pose matrix based on the camera sensor outside the vehicle cabin after correction. The transformation matrix T between the camera sensor outside the vehicle cockpit and the vehicle pose coordinate system obtained in step S1210. veh_cam The first true pose matrix of the vehicle after correction can be calculated according to formula (6). As shown below:

[0127] S1240: Output the correction results.

[0128] Through the above calculation steps, the corrected vehicle pose information can be output as the correction result. The second pose can be determined based on the corrected vehicle pose information. The corrected vehicle pose information can be the first true pose matrix of the vehicle calculated in the above steps.

[0129] In the above method 1200, the directional information constraint is not sensitive to the depth value of the road element in the preset map relative to the vehicle's location, does not require multiple frames of images, and the method constrains and optimizes the initial pose of the vehicle by treating the information of multiple first directions as a whole. Therefore, the vehicle pose correction method of this application embodiment can correct the vehicle's offset pose more accurately and quickly, and improve the success rate of vehicle pose correction.

[0130] In one implementation, after the vehicle has traveled a certain distance, the positions of road elements within a certain range can be converged based on information from multiple first directions. Then, based on the positions of these road elements, the positions of matching map elements in a pre-set map are searched. By satisfying the principle of "minimizing the difference between the positions of the converged road elements and the positions of the map elements," positional information constraints are constructed to constrain and optimize the initial pose of the camera sensor outside the vehicle's cockpit. This method of camera sensor pose correction outside the vehicle's cockpit using constraints based on the fusion of directional and positional information is beneficial for obtaining more stable correction results and further improving the correction success rate.

[0131] Figure 13 shows a schematic flowchart of a pose correction method 1300 according to an embodiment of this application. As shown in Figure 13, the method 1300 may include:

[0132] S1310: Obtain initial data.

[0133] Optionally, before acquiring the initial data, the method 1200 may further include: detecting that the satellite positioning signal has failed and that the vehicle's location meets the pose correction conditions.

[0134] For example, as shown in Figure 13, the initial data may include the initial pose of the vehicle that needs to be corrected, the location of the road element detection box on the camera image when the vehicle is in the initial pose, and the transformation relationship between the camera sensor outside the vehicle cabin and the vehicle pose coordinate system.

[0135] For example, the initial data may also include intrinsic parameters (f) of camera sensors located outside the vehicle cabin. x f y c x c y ).

[0136] The transformation relationship between the initial pose of the vehicle that needs to be corrected, the camera sensor outside the vehicle cabin, and the vehicle pose coordinate system can be represented by the matrix described in step S1210. The position of the road element detection box on the initial image of the camera sensor outside the vehicle cabin can be referred to Figure 5, and will not be repeated here.

[0137] S1320: Construct constraint information.

[0138] As shown in Figure 13, step S1320 may further include the following sub-steps:

[0139] S1321: Calculate the initial pose of the camera sensor outside the vehicle's cockpit based on the initial pose of the vehicle that needs to be corrected and the transformation relationship between the camera sensor outside the vehicle's cockpit and the vehicle's pose coordinate system.

[0140] S1322: Calculate the 3D plane formed by the centerline of the camera center and the centerline of the rod-shaped road element detection box and / or the 3D ray formed by the centerline of the camera center and the center point of the planar road element detection box.

[0141] S1323: Project the above-mentioned 3D ray and / or 3D plane onto the first plane to obtain information in multiple first directions.

[0142] S1324: Combine with a pre-set map, construct directional information constraints by satisfying the principle that "the distance between multiple first-direction information and one or more map elements in the matching map is closest".

[0143] The method for constructing the first directional constraint in the above steps can refer to the content described in steps S1221 to S1224 above, and will not be repeated here.

[0144] S1325: Determine whether the convergence condition of the positions of multiple road elements is met.

[0145] For example, when the convergence condition of the locations of multiple road elements is met, steps S1326 and S1331 are executed; otherwise, step S1332 is executed.

[0146] For example, the convergence condition for the positions of multiple road elements can be that at least two poses are obtained, and the sensing data collected by sensors outside the cabin when the vehicle is in these two poses includes at least two matching road elements.

[0147] S1326: Based on information from multiple directions, gather the 2D positions of multiple road elements and combine them with a pre-set map to construct location information constraints.

[0148] As shown in Figure 11, based on the information from the first direction, the second direction, and the third direction, position information 1 can be converged, which is the intersection of the information from the first direction, the second direction, and the third direction. Based on the information from the first direction and the second direction, position information 2 can be converged, which is the intersection of the information from the first direction and the second direction. Based on the information from the second direction and the third direction, position information 3 can be converged, which is the intersection of the information from the second direction and the third direction. Based on the information from the second direction, the third direction, and the fourth direction, position information 4 and position information 5 can be converged, which are the intersections of two different sets of information from the second direction, the third direction, and the fourth direction.

[0149] After obtaining multiple location information, location information constraints can be constructed by combining information from one or more road elements in a pre-set map. As shown in Figure 11, by combining the types and locations of multiple map elements in the pre-set map, the map element that is closest to location information 1 and matches the type is map element 1; the map element that is closest to location information 2 and matches the type is map element 2; the map element that is closest to location information 3 and matches the type is map element 3; the map element that is closest to location information 4 and matches the type is map element 4; and the map element that is closest to location information 5 and matches the type is map element 5.

[0150] In one implementation, a second transformation matrix can be used. Initial pose T of the camera sensor outside the vehicle cabin world_cam Corrective action is taken. Among them, R2 * The rotation matrix represents the second transformation matrix, and the yaw angle of the second pose can be determined based on this rotation matrix; t2 * This represents the translation vector of the second transformation matrix, and the displacement of the second pose relative to the first pose can be determined based on the translation vector of the second transformation matrix.

[0151] For example, the 2D position of each location information can be represented as P. i =(x j ,y j ) T Combining the coordinates of these 2D positions, the set of m positional information can be represented as P = {p1, ..., p...} j ,…,p m}, where m is an integer greater than 1, and j ranges from 1 to m. Based on the 2D position of each location information, the 2D position of the map element in the preset map that matches the location information can be represented as P′. j =(x′) j ,y′ j ) T The set of 2D locations of m map elements is represented as P′={p′1,…,p′}. j ,…,p′ m Then, calculate the centroid positions p and p' of the set of location information P and the set of 2D locations of map elements P′, and calculate the decentroid coordinates of each point according to formula (7): q j =p j -p,q′ j =p′1-p′ (7)

[0152] When the initial pose of the camera sensors outside the vehicle cabin is completely error-free, the position information converged from multiple directions should be consistent with the position of the corresponding map element. Therefore, based on the principle of "minimizing the difference between the positions of the converged road elements and the map elements," when using the second transformation matrix... rotation matrix R2 * For q j When performing the transformation, the transformed q j and q′ j The positions of the two points should be as close as possible, from which we can obtain ||R2 * ·q j -q′ j || 2 The value should be close to 0. Based on this, location information constraints can be constructed.

[0153] S1330: Initial vehicle position correction.

[0154] As shown in Figure 13, step S1330 may further include the following sub-steps:

[0155] S1331: Correct the initial pose of the camera sensor outside the vehicle cabin based on the orientation and position constraints. For example, the optimization equation combining orientation and position information is shown in formula (8) below:

[0156] The second transformation matrix T2 can be obtained by solving the equation in formula (8). * rotation matrix R2 * .

[0157] Based on the second transformation matrix T2 obtained from the solution * rotation matrix R2 * The centroid position p of the set of location information P, and the centroid position p′ of the set of 2D positions of map elements P′, can be used to calculate the second transformation matrix T2. * Translation vector t2 * For example, the second transformation matrix T2 * Translation vector t2 * t2 can be calculated using the following formula (9): * =p′-R2 * ·p (9)

[0158] The second transformation matrix can be obtained from formulas (8) and (9). According to formula (10), the initial pose T of the camera sensor outside the vehicle cabin can be determined. world_cam Perform correction and optimization:

[0159] Finally, the second pose matrix of the camera sensors outside the vehicle cabin after correction is obtained.

[0160] S1332: Correct the initial pose of the camera sensors outside the vehicle cabin based on orientation information constraints.

[0161] When the convergence condition of the positions of multiple road elements is not met, step S1332 can be executed to construct an optimization problem based solely on the directional information constraint to correct the initial pose of the camera sensor outside the vehicle cabin.

[0162] For example, the method for correcting the course based on orientation information constraints can refer to the aforementioned step S1231 to obtain the first pose matrix of the camera sensor outside the vehicle cabin after correction. I will not go into details here.

[0163] S1333: Calculate the true pose after correction based on the pose of the camera sensor outside the vehicle cabin after correction, the transformation relationship between the camera sensor outside the vehicle cabin and the vehicle pose coordinate system.

[0164] For example, combining the first pose matrix based on the camera sensor outside the vehicle cabin after correction. The transformation matrix T between the camera sensor outside the vehicle cockpit and the vehicle pose coordinate system obtained in step S1310. veh_cam The first true pose matrix of the vehicle after correction can be calculated according to formula (6).

[0165] For example, combining the second pose matrix based on the corrected camera sensor outside the vehicle cabin. The transformation matrix T between the camera sensor outside the vehicle cockpit and the vehicle pose coordinate system obtained in step S1310. veh_cam The second true pose matrix of the vehicle after correction can be calculated according to formula (11). As shown below:

[0166] S1340: Output the correction results.

[0167] Through the above calculation steps, the corrected vehicle pose information can be output as the correction result. The second pose can be determined based on the corrected vehicle pose information. The corrected vehicle pose information can be the first true pose matrix of the vehicle calculated in the above steps. Or the second true pose matrix

[0168] Optionally, steps S1325 and S1332 can be omitted, that is, steps S1326, S1331, and S1333 are executed after step S1324 is completed. When the vehicle's travel time or travel distance meets the preset conditions, position information constraints can be constructed directly after the direction information constraints are constructed, and the initial pose of the vehicle can be corrected based on the direction information and position information.

[0169] Figure 14 shows a schematic block diagram of a pose correction device 1400 according to an embodiment of this application. The pose correction device 1400 includes: an acquisition unit 1410, used to acquire a first pose of the vehicle and first sensing data collected by sensors outside the cabin when the vehicle is in the first pose, the first sensing data including information of one or more first road elements; and to acquire information of multiple first directions based on the first pose and the information of one or more first road elements; and a determination unit 1420, used to determine a second pose of the vehicle based on the information of one or more road elements in the map and the information of multiple first directions.

[0170] Optionally, the device may also include a matching unit.

[0171] Optionally, the first sensing data includes information on multiple first road elements. Before determining the second pose of the vehicle based on the information of one or more road elements in the map and the information of multiple first directions, the device further includes: a matching unit, used to match the information of multiple road elements in the map with the information of multiple first directions to obtain a matching result; and a determining unit 1420, specifically used to: determine the second pose based on the matching result, the information of multiple first directions and the information of multiple road elements in the map.

[0172] Optionally, the acquisition unit 1410 is specifically used to: project the information of one or more first road elements onto a first plane to acquire information of multiple first directions.

[0173] Optionally, the first sensing data includes information about a first road element. The acquisition unit 1410 is specifically used to: acquire information about multiple first directions based on at least two points where the distance between the first pose and the projection position of the first road element in the first plane is greater than or equal to a preset size.

[0174] Optionally, the determining unit 1420 is specifically used to: determine the yaw angle and displacement of the second pose based on information from one or more road elements in the map and information from multiple first directions.

[0175] Optionally, the acquisition unit 1410 is further configured to: acquire the third pose of the vehicle and second sensing data collected by sensors outside the cabin when the vehicle is in the third pose, the second sensing data including information of one or more second road elements; acquire information of multiple second directions based on the third pose and information of one or more second road elements; the determination unit 1420 is specifically configured to: determine multiple location information based on the intersection of information of multiple first directions and information of multiple second directions; determine the second pose of the vehicle based on information of one or more road elements in the map and multiple location information.

[0176] Optionally, the determining unit 1420 is specifically used to: determine the yaw angle of the second pose based on information of one or more road elements and multiple location information in the map; and determine the displacement of the second pose based on information of one or more road elements, multiple location information and yaw angle in the map.

[0177] For example, the acquisition unit 1410 can be the computing platform 120 in Figure 1, or the processing circuit, processor, or controller in the computing platform 120. Taking the processor 121 in the computing platform as an example, the acquisition unit 1410 can acquire the first posture of the vehicle and the first sensing data collected by the sensors outside the cockpit when the vehicle is in the first posture. The first sensing data includes information on one or more first road elements; based on the first posture and the information of one or more first road elements, information on multiple first directions is acquired.

[0178] For example, the determining unit 1420 can be the computing platform 120 in Figure 1, or the processing circuit, processor, or controller in the computing platform 120. Taking the determining unit 1420 as the processor 122 in the computing platform as an example, the processor 122 can determine the second pose of the vehicle based on the information of one or more road elements in the map and the information of multiple first directions.

[0179] For example, the matching unit can be the computing platform 120 in Figure 1, or the processing circuit, processor, or controller in the computing platform 120. Taking the processor 123 in the computing platform as an example, the processor 123 can match the information of multiple road elements in the map with the information of multiple first directions to obtain the matching result.

[0180] The functions implemented by the acquisition unit 1410, the determination unit 1420, and the matching unit can be implemented by different processors, or by the same processor, or some functions can be implemented by the same processor. This application embodiment does not limit this.

[0181] It should be understood that the division of units in the above device is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, the units in the device can be implemented by a processor calling software; for example, the device includes a processor connected to memory, which stores instructions. The processor calls the instructions stored in memory to implement any of the above methods or to implement the functions of each unit in the device. The processor can be, for example, a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. The functions of some or all units can be implemented through the design of the hardware circuits, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all units are implemented through the design of the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files, thereby implementing the functions of some or all units. All units of the above devices can be implemented entirely through processor calling software, or entirely through hardware circuits, or partially through processor calling software with the remaining parts implemented through hardware circuits.

[0182] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration file to configure the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0183] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0184] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together as a System-on-a-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and AI processor, CPU and GPU, etc.

[0185] This application also provides a pose correction device, which includes: a memory for storing a computer program; and a processor for executing the computer program stored in the memory, so that the device can execute any of the pose correction methods described in the above embodiments.

[0186] Alternatively, if the device is located in a vehicle, the processor may be the processor 121-12n shown in FIG1.

[0187] This application also provides a vehicle that may include the above-described posture correction device 1400 or the system shown in FIG2.

[0188] This application also provides a computer-readable medium storing instructions that, when executed by a processor, cause the processor to implement any of the pose correction methods described in the above embodiments.

[0189] This application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to execute any of the pose correction methods described in the above embodiments.

[0190] This application also provides a chip, which includes a circuit that can be used to execute any of the pose correction methods described in the above embodiments.

[0191] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, power-on erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0192] It should be understood that in the embodiments of this application, the memory may include read-only memory and random access memory, and provides instructions and data to the processor.

[0193] It should also be understood that, in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0194] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0195] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0196] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0197] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0198] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0199] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0200] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A pose correction method, characterized in that, The method includes: The vehicle's first pose and the first sensing data collected by sensors outside the cabin when the vehicle is in the first pose are obtained. The first sensing data includes information on one or more first road elements. Based on the first pose and the information of one or more first road elements, information on multiple first directions is obtained; The second pose of the vehicle is determined based on information from one or more road elements on the map and information from the multiple first directions.

2. The method according to claim 1, characterized in that, The first sensing data includes information on multiple first road elements. Before determining the second pose of the vehicle based on the information of one or more road elements in the map and the information on the multiple first directions, the method further includes: The information of the multiple road elements in the map is matched with the information of the multiple first directions to obtain a matching result; Determining the second pose of the vehicle based on information from one or more road elements in the map and information from the multiple first directions includes: The second pose is determined based on the matching result, the information of the plurality of first directions, and the information of the plurality of road elements in the map.

3. The method according to claim 1 or 2, characterized in that, The step of obtaining information on multiple first directions based on the first pose and the information of one or more first road elements includes: The information of one or more first road elements is projected onto a first plane to obtain information of the multiple first directions.

4. The method according to claim 1, characterized in that, The first sensing data includes information about a first road element. The step of obtaining information about multiple first directions based on the first pose and the information of the one or more first road elements includes: Information about the plurality of first directions is obtained based on at least two points where the distance between the first pose and the projection position of the first road element in the first plane is greater than or equal to a preset size.

5. The method according to any one of claims 1 to 4, characterized in that, Determining the second pose of the vehicle based on information from one or more road elements in the map and information from the multiple first directions includes: Based on the information of one or more road elements in the map and the information of the multiple first directions, the yaw angle and displacement of the second pose are determined.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: The vehicle's third pose and second sensing data collected by sensors outside the cabin when the vehicle is in the third pose are obtained, the second sensing data including information of one or more second road elements; Based on the third pose and the information of one or more second road elements, information on multiple second directions is obtained; Determining the second pose of the vehicle based on information from one or more road elements in the map and information from the multiple first directions includes: Based on the intersection of the information from the multiple first directions and the information from the multiple second directions, multiple location information is determined; The second pose is determined based on the information of one or more road elements in the map and the multiple location information.

7. The method according to claim 6, characterized in that, Determining the second pose based on the information of one or more road elements in the map and the multiple location information includes: The yaw angle of the second pose is determined based on the information of one or more road elements in the map and the multiple location information. The displacement of the second pose is determined based on the information of one or more road elements in the map, the multiple location information, and the yaw angle.

8. A posture correction device, characterized in that, The device includes: The acquisition unit is used to acquire the first pose of the vehicle and the first sensing data collected by sensors outside the cabin when the vehicle is in the first pose, wherein the first sensing data includes information of one or more first road elements; and to acquire information of multiple first directions based on the first pose and the information of the one or more first road elements. The determining unit is used to determine the second pose of the vehicle based on information from one or more road elements in the map and information from the multiple first directions.

9. The apparatus according to claim 8, characterized in that, The first sensing data includes information on multiple first road elements. Before determining the second pose of the vehicle based on the information of one or more road elements in the map and the information on the multiple first directions, the device further includes: A matching unit is used to match the information of the plurality of road elements in the map with the information of the plurality of first directions to obtain a matching result; The determining unit is specifically used to: determine the second pose based on the matching result, the information of the plurality of first directions, and the information of the plurality of road elements in the map.

10. The apparatus according to claim 8 or 9, characterized in that, The acquisition unit is specifically used for: The information of one or more first road elements is projected onto a first plane to obtain information of the multiple first directions.

11. The apparatus according to claim 8, characterized in that, The first sensing data includes information about a first road element, and the acquisition unit is specifically used for: Information about the plurality of first directions is obtained based on at least two points where the distance between the first pose and the projection position of the first road element in the first plane is greater than or equal to a preset size.

12. The apparatus according to any one of claims 8 to 11, characterized in that, The determining unit is specifically used for: Based on the information of one or more road elements in the map and the information of the multiple first directions, the yaw angle and displacement of the second pose are determined.

13. The apparatus according to any one of claims 8 to 12, characterized in that, The acquisition unit is further configured to: The vehicle's third pose and second sensing data collected by sensors outside the cabin when the vehicle is in the third pose are obtained, the second sensing data including information of one or more second road elements; based on the third pose and the information of the one or more second road elements, information of multiple second directions is obtained; The determining unit is specifically used to: determine multiple location information based on the intersection of the multiple first direction information and the multiple second direction information; and determine the second pose based on the information of one or more road elements in the map and the multiple location information.

14. The apparatus according to claim 13, characterized in that, The determining unit is specifically used for: The yaw angle of the second pose is determined based on the information of one or more road elements in the map and the multiple location information. The displacement of the second pose is determined based on the information of one or more road elements in the map, the multiple location information, and the yaw angle.

15. A posture correction device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program stored in the memory to cause the apparatus to perform the method as described in any one of claims 1 to 7.

16. A vehicle, characterized in that, Includes the posture correction device as described in any one of claims 8 to 15.

17. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a computer, enables the implementation of the method as described in any one of claims 1 to 7.

18. A computer program product, characterized in that, The computer program product includes computer program code that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 7.

19. A chip, characterized in that, include: A circuit for performing the method as described in any one of claims 1 to 7.

Images