Pose estimation method, apparatus, device and computer readable storage medium

Abstract

Embodiments of the present application provide a pose estimation method, device, equipment and computer readable storage medium, the method comprises: obtaining the environment shooting image corresponding to the target vehicle and the initial pose information; generating a plurality of reference pose information according to the initial pose information and the preset pose offset value; according to a plurality of reference pose information and preset map data, determine the environment estimation image corresponding to the reference pose information; according to each reference pose information and its corresponding environment estimation image, and the environment shooting image determine the real pose information of the target vehicle. The pose estimation method provided by the embodiments of the present application can utilize the existing high-precision map data, cooperate with low-cost low-precision pose equipment, and stably realize high-precision estimation of vehicle pose at low cost.

Description

Technical Field

[0001] This application relates to the field of vehicle driving technology, specifically to a pose estimation method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] High-precision pose estimation is a key task for autonomous driving, and all modules related to autonomous driving, such as decision-making, planning, and control, are highly dependent on the localization results.

[0003] Currently, commonly used GNSS-based positioning methods can achieve centimeter-level accuracy in open scenarios, but their reliability is insufficient under occlusion and multipath conditions, and the equipment cost is high, making it difficult to apply them widely. Summary of the Invention

[0004] This application provides a pose estimation method, apparatus, device, and computer-readable storage medium, aiming to solve the technical problem that accurate pose estimation in the prior art relies on high-precision equipment, resulting in high implementation cost and poor stability.

[0005] On one hand, embodiments of this application provide a pose estimation method, including:

[0006] Acquire environmental images and initial pose information corresponding to the target vehicle;

[0007] Multiple reference pose information are generated based on the initial pose information and the preset pose offset value;

[0008] Based on the multiple reference pose information and preset map data, determine the environment estimation image corresponding to the reference pose information;

[0009] The true pose information of the target vehicle is determined based on the reference pose information and its corresponding environmental estimation image, as well as the environmental captured image.

[0010] As a feasible embodiment of this application, determining the true pose information of the target vehicle based on each of the reference pose information and its corresponding environmental estimation image, as well as the environmental captured image, includes:

[0011] For each reference pose information, the image loss value corresponding to the reference pose information is determined based on the image difference between the environment estimation image corresponding to the reference pose information and the environment captured image.

[0012] The reference pose information is updated according to the image loss value corresponding to each reference pose information to obtain the updated pose information;

[0013] Once the updated pose information meets the preset conditions, the target pose information is filtered out from the updated pose information, and the target pose information is determined as the true pose information of the target vehicle.

[0014] As a feasible embodiment of this application, determining the image loss value corresponding to the reference pose information based on the image difference between the environment estimation image corresponding to the reference pose information and the environment captured image includes:

[0015] Perform semantic segmentation on the environmental image to obtain a first semantic segmentation result corresponding to the environmental image;

[0016] Based on the first semantic segmentation result, a distance transformation is performed on the environmental image to obtain a distance transformation matrix;

[0017] The image loss value corresponding to the reference pose information is determined based on the second semantic segmentation result of the environment-estimated image and the distance transformation matrix.

[0018] As a feasible embodiment of this application, the step of determining the image loss value corresponding to the reference pose information based on the second semantic segmentation result of the image estimated by the environment and the distance transformation matrix includes:

[0019] Based on the pixel coordinates of each asset in the second semantic segmentation result of the environmental estimation image, the distance values ​​corresponding to each asset are extracted from the distance transformation matrix;

[0020] Based on the statistical values ​​of the distance values, the image loss value corresponding to the reference pose information is determined.

[0021] As a feasible embodiment of this application, the updated pose information obtained by updating each reference pose information according to the image loss value corresponding to each reference pose information includes:

[0022] Based on the comparison results between the image loss values ​​corresponding to each of the reference pose information, the current optimal pose information is selected from the reference pose information;

[0023] For each of the reference pose information, the reference pose information is updated according to the historical best pose information corresponding to the reference pose information and the current best pose information to obtain the updated pose information.

[0024] As a feasible embodiment of this application, determining the environment estimation image corresponding to the reference pose information based on the plurality of reference pose information and preset map data includes:

[0025] For each reference pose information, the target asset corresponding to the reference pose information and the world coordinate information corresponding to each target asset are extracted from the map data based on the reference pose information.

[0026] Based on the coordinate transformation parameters corresponding to the reference pose information, coordinate transformation is performed on the world coordinate information corresponding to each target asset to obtain the camera coordinate information corresponding to each target asset.

[0027] Based on the camera intrinsic parameters corresponding to the reference pose information, the camera coordinate information corresponding to each target asset is processed to obtain the image coordinate information corresponding to each target asset.

[0028] An environment estimation image corresponding to the reference pose information is generated based on the image coordinate information corresponding to each target asset.

[0029] As a feasible embodiment of this application, the step of acquiring the environmental image and initial pose information corresponding to the target vehicle includes:

[0030] An environmental image corresponding to the target vehicle is obtained through a preset image acquisition device on the target vehicle;

[0031] The initial pose information of the target vehicle is obtained through the inertial measurement unit and positioning system preset on the target vehicle; the initial pose information includes at least longitude, latitude, altitude, pitch angle, yaw angle and roll angle.

[0032] On the other hand, embodiments of this application also provide a pose estimation device, including:

[0033] The acquisition module is used to acquire environmental images and initial pose information corresponding to the target vehicle.

[0034] The generation module is used to generate multiple reference pose information based on the initial pose information and the preset pose offset value;

[0035] The estimation module is used to determine the environment estimation image corresponding to the reference pose information based on the multiple reference pose information and preset map data.

[0036] The determination module is used to determine the true pose information of the target vehicle based on each of the reference pose information and its corresponding environmental estimation image, as well as the environmental captured image.

[0037] On the other hand, embodiments of this application also provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the steps in the pose estimation method described above.

[0038] On the other hand, embodiments of this application also provide a computer-readable storage medium storing a computer program, which is executed by a processor to implement the steps in the pose estimation method described above.

[0039] The pose estimation method provided in this application first generates multiple possible reference pose information after obtaining low-precision pose information of the target vehicle. Then, it extracts the estimated image corresponding to each reference pose from the preset high-precision map data and compares it with the actual captured environmental image. Thus, the real pose information of the target vehicle is calculated using each reference pose information. It can use existing high-precision map data and low-cost low-precision pose equipment to stably achieve high-precision estimation of vehicle pose at low cost. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A schematic diagram illustrating an implementation scenario of a pose estimation method provided in an embodiment of this application;

[0042] Figure 2 A schematic flowchart illustrating the steps of a pose estimation method provided in this application embodiment;

[0043] Figure 3 This application provides a flowchart illustrating the steps for determining true pose information in an embodiment of the present application.

[0044] Figure 4 This application provides a schematic flowchart of steps for determining an image loss value in an embodiment of the present application.

[0045] Figure 5 This is a schematic diagram illustrating a distance transformation of an image, provided as an embodiment of this application.

[0046] Figure 6 This application provides a schematic flowchart of a step for determining image loss values ​​based on a distance transformation matrix in an embodiment of the present application.

[0047] Figure 7 This application provides a schematic flowchart illustrating the steps for updating pose information in an embodiment of the present application.

[0048] Figure 8This application provides a schematic flowchart illustrating the steps for obtaining an environmental estimation image based on pose information in an embodiment of the present application.

[0049] Figure 9 A flowchart illustrating the steps of a complete implementation scheme for a pose estimation method provided in this application embodiment;

[0050] Figure 10 This is a schematic diagram of the structure of a pose estimation device provided in an embodiment of this application;

[0051] Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0052] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of the present invention.

[0053] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to implement and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be implemented without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in the embodiments of this application.

[0054] This application provides a pose estimation method, apparatus, device, and computer-readable storage medium, which will be described in detail below.

[0055] In this embodiment, the pose estimation method is deployed as a program on a pose estimation device, which is installed in a computer device as a processor. The pose estimation device in the computer device executes the following steps by running the program corresponding to the pose estimation method: acquiring an environmental image and initial pose information corresponding to the target vehicle; generating multiple reference pose information based on the initial pose information and a preset pose offset value; determining an environmental estimation image corresponding to the multiple reference pose information and preset map data; and determining the true pose information of the target vehicle based on each reference pose information and its corresponding environmental estimation image, as well as the environmental image.

[0056] like Figure 1 As shown, Figure 1 This is a schematic diagram illustrating an implementation scenario of a pose estimation method provided in an embodiment of this application. Specifically, the implementation scenario includes a target vehicle 100, and a camera 200, an inertial measurement unit 300, a positioning system 400, and a pose estimation device 500 installed on the target vehicle 100 for realizing vehicle pose estimation.

[0057] The imaging device 200 is typically installed at a fixed position on the vehicle and captures environmental images of the vehicle in real time during its movement, transmitting these images to the pose estimation device 500. The inertial measurement unit 300 and the positioning system 400 are primarily used to acquire the target vehicle's pose information in real time and transmit it to the pose estimation device 500. However, the pose information acquired here is generally low-precision, meaning it contains some error compared to the target vehicle's true pose. Specifically, the pose information includes the vehicle's attitude angles (pitch, yaw, roll) acquired by the inertial measurement unit 300, and the vehicle's position information (longitude, latitude, altitude, etc.) acquired by the positioning system.

[0058] The pose estimation device 500 mainly processes the environmental images and pose information transmitted by the aforementioned shooting device 200, inertial measurement unit 300 and positioning system 400, and then executes the pose estimation method provided in this application embodiment to complete the data processing and output high-precision real pose information of the target vehicle.

[0059] It should be noted that, Figure 1 The schematic diagram illustrating the implementation scenario of the pose estimation method shown is merely an example. The schematic diagram of the implementation scenario of the pose estimation method described in the embodiments of this application is intended to more clearly illustrate the technical solutions of the embodiments of this application and does not constitute a limitation on the technical solutions provided in the embodiments of this application.

[0060] Based on the schematic diagram of the above-mentioned pose estimation method, a specific implementation method for the pose estimation method is proposed.

[0061] like Figure 2 As shown, Figure 2 This is a flowchart illustrating the steps of a pose estimation method provided in an embodiment of this application. The pose estimation method in this embodiment includes steps 201 to 204:

[0062] 201. Acquire environmental images and initial pose information corresponding to the target vehicle.

[0063] In this embodiment of the application, in conjunction with the foregoing Figure 1 As shown in the provided implementation scenario diagram, environmental images are acquired through a camera mounted on the target vehicle, while initial pose information is acquired through an inertial measurement unit (IMU) and a positioning system mounted on the target vehicle. Specifically, the initial pose information mainly includes vehicle attitude angles such as pitch, yaw, and roll angles acquired by the IMU, and vehicle position information such as latitude, longitude, and altitude acquired by a positioning system such as GPS (Global Positioning System). It should be noted that the IMU and positioning system can be low-cost, low-precision devices; in this case, the acquired initial pose information will also be low-precision, meaning it will have some error compared to the target vehicle's true pose information.

[0064] Of course, as a feasible embodiment of this application, the target vehicle can also be equipped with a positioning system with high positioning accuracy, such as GNSS (Global Navigation Satellite System), which can be used to collect more accurate pose information. When the positioning system with high positioning accuracy fails due to signal or other reasons, resulting in the inability to collect more accurate pose information, the accurate pose information can then be estimated by the pose estimation method provided in the embodiments of this application.

[0065] 202. Generate multiple reference pose information based on the initial pose information and the preset pose offset value.

[0066] In this embodiment, considering that the initial pose information is low-precision, meaning it has a certain error compared to the true pose information, multiple reference pose information can be generated near the initial pose information according to some pre-set pose offset values. This facilitates the subsequent generation of richer environmental estimation images, thereby obtaining more accurate true pose information.

[0067] Specifically, as a feasible embodiment of this application, the initial pose information can be used as a reference, and 5 values ​​can be generated evenly within a range of 10m forward and backward offset and 10m left and right offset, 2 values ​​can be generated evenly within a range of 0.5m vertical offset, 5 values ​​can be generated evenly within a range of 10° yaw angle offset, and 3 values ​​can be generated evenly within a range of 5° pitch angle offset and 5° roll angle offset. After combining all the values, a total of 5×5×2×5×3×3=2250 reference pose information can be obtained.

[0068] Specifically, the smaller the pose offset value, the higher the pose search accuracy, the more reference pose information is generated, and the higher the accuracy of the final true pose information. However, the computational load is also higher. Conversely, the higher the pose offset value, the lower the pose search accuracy, the fewer reference pose information is generated, and the lower the accuracy of the final true pose information. However, the computational load is relatively lower. Therefore, the pose offset value and specific generation rules mentioned above can be set based on actual needs, and the obtained reference pose information and its quantity will be adjusted accordingly.

[0069] 203. Based on the multiple reference pose information and the preset map data, determine the environment estimation image corresponding to the reference pose information.

[0070] In this embodiment, the map data is high-precision map data pre-stored in a database, containing rich map information. Therefore, with the camera and vehicle poses relatively fixed, the camera pose information can be determined based on the reference pose information. Using the camera pose information, the corresponding field of view can be extracted from the map data, thereby generating an environment estimation image corresponding to the reference pose information. Specifically, the implementation scheme for generating the environment estimation image can be found in the following sections. Figure 8 And its explanations and descriptions.

[0071] 204. Determine the true pose information of the target vehicle based on the reference pose information and its corresponding environmental estimation image, as well as the environmental captured image.

[0072] In this embodiment, after obtaining each reference pose information and its corresponding environment estimation image, the true pose information of the target vehicle can be determined by utilizing the difference between the environment estimation image corresponding to the reference pose information and the environment captured image. For example, as a simple and feasible implementation, the image similarity between each environment estimation image and the environment captured image is calculated, and the image most similar to the environment captured image is selected from the environment estimation images. Then, the reference pose information corresponding to this image is used as the true pose information of the target vehicle. The image similarity can be calculated based on comparing pixel variance, perceptual hashing algorithms, or template matching, which will not be elaborated further in this embodiment.

[0073] Of course, the aforementioned true pose information is merely the most probable pose information selected from numerous possible poses using image similarity. While this improves pose accuracy to some extent, it may still contain certain errors. To further improve pose accuracy, as a feasible embodiment of this application, a scheme for determining true pose information based on particle swarm optimization is provided. Specific implementation details can be found in subsequent sections. Figure 3 And its explanations and descriptions.

[0074] like Figure 3 As shown, Figure 3 A flowchart illustrating the steps for determining true pose information provided in this application embodiment is described in detail below.

[0075] In this embodiment of the application, a scheme for determining the true pose information based on the particle swarm optimization idea is provided, specifically including steps 301 to 303:

[0076] 301. Based on the image difference between the environment estimation image corresponding to the reference pose information and the environment captured image, determine the image loss value corresponding to the reference pose information.

[0077] In this embodiment, each reference pose information obtained above is considered as a particle in a particle swarm optimization algorithm. The image difference between the corresponding estimated environment image and the captured environment image is used as the loss value of that particle, thereby optimizing each particle, i.e., each reference pose information. Specifically, the particle loss value, i.e., the image loss value, is determined based on the image difference between the estimated environment image and the captured environment image. This loss value can be obtained based on semantic segmentation and distance transformation of the image. Specific implementation schemes can be found in subsequent sections. Figure 4 And its explanations and descriptions.

[0078] 302. Update each of the reference pose information according to the image loss value corresponding to each of the reference pose information to obtain the updated pose information.

[0079] In this embodiment, the image loss value describes the difference between the particle, i.e., the reference position information, and the final accurate pose information. A larger image loss value indicates a larger difference between the reference pose information and the final accurate pose information; conversely, a smaller image loss value indicates a smaller difference between the reference pose information and the final accurate pose information. Therefore, based on this theory, the image loss value corresponding to each reference pose information can be used to update each reference pose information separately. The updated pose information will cause each particle to move towards a location with a smaller difference from the final accurate pose information. Specific implementation details can be found in the following sections. Figure 7 And its explanations and descriptions.

[0080] 303, until the updated pose information meets the preset conditions, the target pose information is filtered out from the updated pose information, and the target pose information is determined as the true pose information of the target vehicle.

[0081] In this embodiment, after updating the pose information, the pose estimation device continues to determine the environment estimation image corresponding to each updated pose information from preset map data, and determines the image loss value corresponding to each updated pose information based on the method provided in step 301 above. Since the image loss value corresponding to the pose information describes the difference between the updated pose information and the final accurate pose information, the preset condition here can be comparing the image loss value corresponding to the updated pose information with a preset loss value threshold.

[0082] Specifically, the image loss value corresponding to each updated pose information is compared with a preset loss value threshold. If the image loss value corresponding to any pose information is less than the preset loss value threshold, it indicates that the difference between this pose information and the final accurate pose information is small enough, and this pose information can be regarded as the true pose information. Conversely, if the image loss values ​​corresponding to all updated pose information are greater than the preset loss value threshold, step 302 is repeated to continue iteratively updating the updated pose information and obtaining updated image loss values ​​again, until the image loss value corresponding to any of the updated pose information is less than the preset loss value threshold. At this point, the pose information can be output as the true pose information of the target vehicle. Of course, if the image loss values ​​corresponding to multiple pose information are all less than the preset loss value threshold, the pose information with the smallest image loss value is taken as the true pose information of the target vehicle. Therefore, in actual operation, after obtaining the image loss values ​​corresponding to multiple updated pose information, it is also possible to first select the best pose information with the smallest image loss value, and then compare the image loss value corresponding to the best pose information with the preset loss value threshold. This will not be elaborated further in the embodiments of this application.

[0083] It should be noted that the above only provides the core steps for determining the true pose information based on the particle swarm optimization (PSO) approach. In fact, the process of determining the true pose information using PSO requires additional configuration of PSO parameters, such as the number of particles, inertia factor, acceleration constant, maximum number of iterations, and minimum allowable error for algorithm termination. More details will follow. Figure 9 It will provide a complete and detailed implementation process.

[0084] like Figure 4 As shown, Figure 4 A flowchart illustrating the steps for determining an image loss value, as provided in an embodiment of this application, is described in detail below.

[0085] In this embodiment of the application, an implementation scheme for determining image loss values ​​based on semantic segmentation and distance changes is provided, specifically including steps 401 to 403:

[0086] 401, Perform semantic segmentation on the environmental image to obtain the first semantic segmentation result corresponding to the environmental image.

[0087] In this embodiment of the application, semantic segmentation refers to segmenting assets in an image. Assets can generally be understood as entity objects in the image, such as lane lines, streetlights, speed bumps, guardrails, etc. The final first semantic segmentation result includes the pixel positions of these entity objects in the image.

[0088] 402. Perform distance transformation on the environmental image based on the first semantic segmentation result to obtain a distance transformation matrix.

[0089] Distance transformation refers to converting the value of a foreground pixel in an image into the distance to the nearest background pixel, or the value of a background pixel into the distance to the nearest foreground pixel. In this embodiment, it refers to using the pixels included in the first semantic segmentation result as foreground pixels, and converting the pixel values ​​of other pixels in the image into distances to the nearest foreground pixels, thereby obtaining a distance transformation matrix corresponding to the captured image. The distances used here can be Euclidean distance, street distance, or chessboard distance; this application does not impose any limitations. The specific definitions and calculation formulas for each distance are not detailed in this embodiment.

[0090] For ease of understanding, Figure 5 A schematic diagram of a distance transformation on an image is shown; please refer to the attached diagram for details. Figure 5 And its explanations and descriptions.

[0091] 403. Based on the second semantic segmentation result of the environment-estimated image and the distance transformation matrix, determine the image loss value corresponding to the reference pose information.

[0092] In this embodiment, after obtaining the distance transformation matrix corresponding to the captured environment image, a second semantic segmentation result of the environment estimation image is obtained based on the same semantic segmentation operation. Then, the image loss value corresponding to the reference pose information is determined using the second semantic segmentation result and the distance transformation matrix. For example, as a feasible embodiment of this application, the same distance transformation operation can be performed on the environment estimation image based on the second semantic segmentation result to obtain the distance transformation matrix corresponding to the environment estimation image. Then, the similarity between the distance transformation matrix corresponding to the environment estimation image and the distance transformation matrix corresponding to the captured environment image is calculated to obtain the image loss value corresponding to the reference pose information.

[0093] Of course, considering that the distance transformation matrix requires certain computational resources, as another feasible embodiment of this application, another implementation scheme for determining the image loss value is proposed. Specifically, the distance values ​​are directly extracted from the distance transformation matrix for calculating the image loss value. For details of the implementation scheme, please refer to the following sections. Figure 6 And its explanations and descriptions.

[0094] like Figure 5 As shown, Figure 5 A schematic diagram of a distance transformation of an image provided in an embodiment of this application is described in detail below.

[0095] like Figure 5 As shown, Figure 5The left image shows the result of semantic segmentation of the image. The dark patches in the upper left and lower right corners represent the segmented foreground regions, while the remaining light patches represent the background regions. The distance transformation matrix shown in the right image is obtained by calculating the distance between each patch and its nearest neighbor within the foreground region. Here, the distance used is the street distance, which is the sum of the differences in the horizontal and vertical directions.

[0096] like Figure 6 As shown, Figure 6 This application provides a flowchart illustrating steps for determining image loss values ​​based on a distance transformation matrix, specifically including steps 601-602:

[0097] 601. Based on the pixel coordinates of each asset in the second semantic segmentation result of the environment estimation image, the distance values ​​corresponding to each asset are extracted from the distance transformation matrix.

[0098] In this embodiment, as described in step 401 above, the first semantic segmentation result corresponding to the environmental image describes the pixel positions of each entity object in the environmental image. Therefore, the second semantic segmentation result of the environmental estimation image also includes the pixel positions, i.e., pixel coordinates, of each entity object, i.e., each asset, in the environmental estimation image. Based on this, and combining the mapping relationship between the environmental image and the environmental estimation image, the distance values ​​corresponding to each asset in the environmental image can be extracted from the distance transformation matrix corresponding to the environmental image.

[0099] Specifically, as mentioned above Figure 5 Taking the distance transformation matrix shown as an example, if for the "L"-shaped asset in the upper left corner, if the "L"-shaped asset in the upper left corner of the second semantic segmentation result corresponding to the environment estimation image is exactly one cell to the left of the "L"-shaped asset in the upper left corner of the first semantic segmentation result corresponding to the environment shooting image, then the distance value of the asset extracted from the distance transformation matrix includes four 1s and one 0.

[0100] 602. Based on the statistical value of the distance value, determine the image loss value corresponding to the reference pose information.

[0101] In this embodiment, after extracting the distance values ​​corresponding to each asset, the image loss value corresponding to the reference pose information can be obtained by performing data statistical processing on these distance values, such as calculating the average value, sum, etc. For example, taking the average value as an example, the smaller the average value, the smaller the distance between the corresponding assets in the environmental captured image and the environmental estimated image, that is, the smaller the image loss value, and the closer the corresponding pose results are.

[0102] like Figure 7 As shown, Figure 7 A flowchart illustrating the steps for updating pose information, as provided in this application embodiment, is described in detail below.

[0103] In this embodiment of the application, an implementation scheme for updating pose information based on particle swarm optimization algorithm is provided, including steps 701-702:

[0104] 701. Based on the comparison results between the image loss values ​​corresponding to each of the reference pose information, the current optimal pose information is selected from the reference pose information.

[0105] In this embodiment of the application, after obtaining the image loss value corresponding to each reference pose information, the image loss value corresponding to each reference pose information is compared, and the reference pose information with the smallest image loss value is taken as the current optimal pose information.

[0106] 702. The reference pose information is updated based on the historical best pose information corresponding to the reference pose information and the current best pose information to obtain the updated pose information.

[0107] In this embodiment of the application, considering that the particle swarm optimization algorithm usually needs to update each reference pose information multiple times during the process of updating each particle, that is, each reference pose information, the pose estimation device records the image loss value of each reference pose information during the update process, determines the historical best pose information corresponding to each reference pose information, and then uses the historical best pose information and the aforementioned selected current best pose information to update the reference pose information to obtain the updated pose information.

[0108] Specifically, updating the particle's pose information (i.e., the reference pose information) requires using velocity and position iteration formulas. The detailed iterative calculation formulas will be discussed later. Figure 8 The details are given in the text.

[0109] like Figure 8 As shown, Figure 8 A flowchart illustrating the steps for obtaining an environmental estimation image based on pose information, as provided in this application embodiment, is described in detail below.

[0110] In this embodiment of the application, steps 801 to 804 are specifically included:

[0111] 801, Based on the reference pose information, extract the target assets corresponding to the reference pose information and the world coordinate information corresponding to each target asset from the map data.

[0112] In this embodiment, since the pose of the camera relative to the target vehicle is fixed, after determining the reference pose information of the target vehicle, the pose information of the camera can be estimated, that is, the position and attitude of the camera relative to the origin of the world coordinate system can be obtained. Based on this, combined with map data, the target assets falling within the field of view of the camera can be extracted from the map data, and the world coordinate information of the target assets in the world coordinate system can be obtained.

[0113] 802. Based on the coordinate transformation parameters corresponding to the reference pose information, perform coordinate transformation on the world coordinate information corresponding to each target asset to obtain the camera coordinate information corresponding to each target asset.

[0114] In this embodiment, the transformation relationship from the world coordinate system to the camera coordinate system can be determined based on the position and orientation of the camera relative to the origin of the world coordinate system, i.e., the coordinate transformation parameters, including the rotation matrix R and the translation vector t. Then, based on these coordinate transformation parameters, the assets within the camera's field of view can be transformed from world coordinates to camera coordinates.

[0115] 803. Based on the camera intrinsic parameters corresponding to the reference pose information, the camera coordinate information corresponding to each target asset is processed to obtain the image coordinate information corresponding to each target asset.

[0116] In this embodiment of the application, after obtaining the camera coordinate information corresponding to the target asset, the image coordinate information of each target asset on the camera imaging can be obtained by simulating asset imaging using camera intrinsic parameters.

[0117] 804. Generate an environment estimation image corresponding to the reference pose information based on the image coordinate information corresponding to each of the target assets.

[0118] Specifically, for ease of understanding, the coordinates of each pixel in the final environment estimation image are calculated using the following formula:

[0119]

[0120] Here, [R|t] is the coordinate transformation parameter, used to transform the world coordinate information corresponding to the target asset. This is converted into camera coordinate information, and This refers to the camera's intrinsic parameters, used to convert the image to its physical coordinate system. This is then used to further transform the image from the physical coordinate system to the image pixel coordinate system, where u0 and v0 represent the actual positions of the principal points, also in pixels, thereby generating the final environment estimation image.

[0121] Specifically, in the above parameters, f is the camera focal length. This represents the number of pixels within 1 millimeter in the x-direction. This represents the number of pixels within 1 millimeter in the y-direction. u0 and v0 represent the actual position of the principal point, also in pixels. γ is the camera distortion parameter, indicating the magnitude of distortion along the two axes of the pixel coordinate system. These parameters are camera-related and can be directly queried, or obtained through existing camera calibration methods. Furthermore, Xw, Yw, and Zw are the three-dimensional coordinates of the target asset in the world coordinate system.

[0122] To facilitate understanding of the pose estimation method provided in the embodiments of this application, the following will be combined with the foregoing. Figures 1 to 8 The provided embodiments offer a complete implementation flow for a pose estimation method. Specifically, as shown... Figure 9 As shown, Figure 9 A flowchart illustrating the complete implementation of a pose estimation method provided in this application embodiment is shown, specifically including the following steps:

[0123] 1. Based on low-precision GPS, low-precision IMU and vehicle heading angle, obtain the original six-degree-of-freedom estimated pose of the current vehicle;

[0124] 2. Using the original six-DOF pose as a reference, set the forward / backward offset range to 10m, the left / right offset range to 10m, the vertical offset range to 0.5m, the pitch offset range to 5°, the roll offset range to 5°, and the yaw offset range to 10°. This search range around the original six-DOF pose is the search space. For forward / backward, generate 5 values ​​evenly in the forward / backward, left / right, and yaw dimensions within the search range; generate 3 values ​​in the pitch and roll dimensions; and generate 2 values ​​in the vertical dimension. After combining all values, the total number of iterative pose particles is 5*5*2*3*3*5 = 2250.

[0125] 3. Set a velocity range for each dimension of the search space and randomly initialize the particle velocities;

[0126] 4. Initialize hyperparameters, including inertia factor, speedup constant, maximum number of iterations, and algorithm termination condition;

[0127] 5. For each particle's six-degree-of-freedom pose, extract the base plate assets within the field of view of that pose from the high-precision map, and simulate imaging of these base plate assets in the current pose to obtain the pixel coordinates of these base plate assets in the corresponding particle pose. This represents the pixel coordinates (pixels) of the simulated image of asset obj at the i-th particle pose.

[0128] 6. Based on the category information of the assets within the field of view of each particle, extract the semantic segmentation of the category assets in the image, with pixel values ​​as follows:

[0129] 7. Segment pixels semantically For the foreground, perform a distance transformation and extract the asset projection pixel position under each particle's pose. The average pixel value over the distance transformation is used as the loss of the current particle;

[0130] 8. Select the position with the minimum loss from the historical positions of each particle as the individual optimal position; select the position with the minimum loss from the historical positions of all particles as the global optimal position.

[0131] 9. Based on the velocity iteration formula Given the current speed, the individual's optimal position, and the global optimal position, obtain the speed at the next moment and determine whether the speed exceeds the set upper and lower limits. If it does, set the speed to the corresponding limit value.

[0132] 10. Based on the position iteration formula The particle's position at the next moment can be obtained by given its current position and its velocity at the next moment.

[0133] 11. Determine whether the loss corresponding to the global optimal position in the current iteration satisfies the algorithm termination condition. If it does, stop the iteration and obtain the optimal pose. If the algorithm termination condition is not met, repeat steps 5, 6, 7, 8, 9, and 10 until the number of iterations reaches the limit.

[0134] To better implement the pose estimation method provided in the embodiments of this application, a pose estimation device is also provided in the embodiments of this application, based on the pose estimation method. For example... Figure 10 As shown, Figure 10 This is a schematic diagram of a pose estimation device provided in an embodiment of this application. Specifically, the pose estimation device includes:

[0135] The acquisition module 1001 is used to acquire environmental images and initial pose information corresponding to the target vehicle.

[0136] The generation module 1002 is used to generate multiple reference pose information based on the initial pose information and the preset pose offset value;

[0137] The estimation module 1003 is used to determine the environment estimation image corresponding to the reference pose information based on the multiple reference pose information and preset map data.

[0138] The determination module 1004 is used to determine the true pose information of the target vehicle based on each of the reference pose information and its corresponding environmental estimation image, as well as the environmental captured image.

[0139] In some embodiments of this application, the determining module is configured to, for each reference pose information, determine an image loss value corresponding to the reference pose information based on the image difference between the environment estimation image corresponding to the reference pose information and the environment captured image; update each reference pose information according to the image loss value corresponding to each reference pose information to obtain updated pose information; until the updated pose information meets a preset condition, filter out target pose information from the updated pose information and determine the target pose information as the true pose information of the target vehicle.

[0140] In some embodiments of this application, the determining module is configured to perform semantic segmentation on the environmental image to obtain a first semantic segmentation result corresponding to the environmental image; perform distance transformation on the environmental image based on the first semantic segmentation result to obtain a distance transformation matrix; and determine the image loss value corresponding to the reference pose information based on the second semantic segmentation result of the environmental estimation image and the distance transformation matrix.

[0141] In some embodiments of this application, the determining module is used to extract the distance value corresponding to each asset from the distance transformation matrix based on the pixel coordinate position of each asset in the second semantic segmentation result of the environment estimated image; and to determine the image loss value corresponding to the reference pose information based on the statistical value of the distance value.

[0142] In some embodiments of this application, the determining module is used to filter out the current optimal pose information from the reference pose information based on the comparison results between the image loss values ​​corresponding to each reference pose information; and for each reference pose information, update the reference pose information according to the historical optimal pose information corresponding to the reference pose information and the current optimal pose information to obtain the updated pose information.

[0143] In some embodiments of this application, the generation module is configured to, for each reference pose information, extract the target asset corresponding to the reference pose information and the world coordinate information corresponding to each target asset from the map data based on the reference pose information; perform coordinate transformation on the world coordinate information corresponding to each target asset according to the coordinate transformation parameters corresponding to the reference pose information to obtain the camera coordinate information corresponding to each target asset; process the camera coordinate information corresponding to each target asset according to the camera intrinsic parameters corresponding to the reference pose information to obtain the image coordinate information corresponding to each target asset; and generate an environment estimation image corresponding to the reference pose information based on the image coordinate information corresponding to each target asset.

[0144] In some embodiments of this application, the acquisition module is used to obtain an environmental image of the target vehicle through a preset image acquisition device on the target vehicle; and to obtain the initial pose information of the target vehicle through a preset inertial measurement unit and positioning system on the target vehicle; the initial pose information includes at least longitude, latitude, altitude, pitch angle, yaw angle, and roll angle.

[0145] This application also provides a computer device, such as... Figure 11 As shown, Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application.

[0146] The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the pose estimation method provided in any embodiment of this application.

[0147] Specifically, a computer device may include components such as a processor 1101 with one or more processing cores, a memory 1102 with one or more storage media, a power supply 1103, and an input unit 1104. Those skilled in the art will understand that... Figure 11 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0148] The processor 1101 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 1102, and by calling data stored in the memory 1102, thereby providing overall monitoring of the computer device. Optionally, the processor 1101 may include one or more processing cores; preferably, the processor 1101 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1101.

[0149] The memory 1102 can be used to store software programs and modules. The processor 1101 executes various functional applications and data processing by running the software programs and modules stored in the memory 1102. The memory 1102 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 1102 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 1102 may also include a memory controller to provide the processor 1101 with access to the memory 1102.

[0150] The computer device also includes a power supply 1103 that supplies power to the various components. Preferably, the power supply 1103 can be logically connected to the processor 1101 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 1103 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0151] The computer device may also include an input unit 1104, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0152] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 1101 in the computer device loads the executable files corresponding to the processes of one or more applications into the memory 1102 according to the following instructions, and the processor 1101 runs the applications stored in the memory 1102, thereby implementing the steps in the pose estimation method provided in any embodiment of this application.

[0153] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc. A computer program is stored on the computer-readable storage medium, and when executed by a processor, the computer program implements the steps in the pose estimation method provided in any embodiment of this application.

[0154] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0155] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0156] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0157] The pose estimation method provided by the embodiments of this application has been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A pose estimation method, characterized in that, include: Acquire environmental images and initial pose information corresponding to the target vehicle; Multiple reference pose information are generated based on the initial pose information and the preset pose offset value; Based on the multiple reference pose information and preset map data, determine the environment estimation image corresponding to the reference pose information; The true pose information of the target vehicle is determined based on the reference pose information and its corresponding environmental estimation image, as well as the environmental captured image. Determining the true pose information of the target vehicle based on each of the reference pose information and its corresponding environmental estimation image, as well as the environmental captured image, includes: For each reference pose information, the image loss value corresponding to the reference pose information is determined based on the image difference between the environment estimation image corresponding to the reference pose information and the environment captured image. The reference pose information is updated according to the image loss value corresponding to each reference pose information to obtain the updated pose information; Once the updated pose information meets the preset conditions, the target pose information is filtered out from the updated pose information, and the target pose information is determined as the true pose information of the target vehicle. The acquisition of the environmental images and initial pose information corresponding to the target vehicle includes: An environmental image corresponding to the target vehicle is obtained through a preset image acquisition device on the target vehicle; The initial pose information of the target vehicle is obtained through the inertial measurement unit and positioning system preset on the target vehicle; the initial pose information includes at least longitude, latitude, altitude, pitch angle, yaw angle, and roll angle. The step of determining the environment estimation image corresponding to the reference pose information based on the plurality of reference pose information and preset map data includes: For each reference pose information, the target asset corresponding to the reference pose information and the world coordinate information corresponding to each target asset are extracted from the map data based on the reference pose information. Based on the coordinate transformation parameters corresponding to the reference pose information, coordinate transformation is performed on the world coordinate information corresponding to each target asset to obtain the camera coordinate information corresponding to each target asset. Based on the camera intrinsic parameters corresponding to the reference pose information, the camera coordinate information corresponding to each target asset is processed to obtain the image coordinate information corresponding to each target asset. An environment estimation image corresponding to the reference pose information is generated based on the image coordinate information corresponding to each target asset.

2. The pose estimation method according to claim 1, characterized in that, The step of determining the image loss value corresponding to the reference pose information based on the image difference between the environment estimation image corresponding to the reference pose information and the environment captured image includes: Perform semantic segmentation on the environmental image to obtain a first semantic segmentation result corresponding to the environmental image; Based on the first semantic segmentation result, a distance transformation is performed on the environmental image to obtain a distance transformation matrix; The image loss value corresponding to the reference pose information is determined based on the second semantic segmentation result of the environment-estimated image and the distance transformation matrix.

3. The pose estimation method according to claim 2, characterized in that, The step of determining the image loss value corresponding to the reference pose information based on the second semantic segmentation result of the image estimated from the environment and the distance transformation matrix includes: Based on the pixel coordinates of each asset in the second semantic segmentation result of the environmental estimation image, the distance values ​​corresponding to each asset are extracted from the distance transformation matrix; Based on the statistical values ​​of the distance values, the image loss value corresponding to the reference pose information is determined.

4. The pose estimation method according to claim 1, characterized in that, The updated pose information obtained by updating each reference pose information according to the image loss value corresponding to each reference pose information includes: Based on the comparison results between the image loss values ​​corresponding to each of the reference pose information, the current optimal pose information is selected from the reference pose information; For each of the reference pose information, the reference pose information is updated according to the historical best pose information corresponding to the reference pose information and the current best pose information to obtain the updated pose information.

5. A pose estimation device, characterized in that, include: The acquisition module is used to acquire environmental images and initial pose information corresponding to the target vehicle. Specifically, it is used to obtain environmental images corresponding to the target vehicle through a preset image acquisition device on the target vehicle; and to obtain the initial pose information corresponding to the target vehicle through a preset inertial measurement unit and positioning system on the target vehicle; the initial pose information includes at least longitude, latitude, altitude, pitch angle, yaw angle, and roll angle. The generation module is used to generate multiple reference pose information based on the initial pose information and the preset pose offset value; The estimation module is used to determine the environment estimation image corresponding to the reference pose information based on the multiple reference pose information and preset map data. Specifically, for each reference pose information, the system extracts the target asset corresponding to the reference pose information and the world coordinate information corresponding to each target asset from the map data; performs coordinate transformation on the world coordinate information corresponding to each target asset according to the coordinate transformation parameters corresponding to the reference pose information to obtain the camera coordinate information corresponding to each target asset; processes the camera coordinate information corresponding to each target asset according to the camera intrinsic parameters corresponding to the reference pose information to obtain the image coordinate information corresponding to each target asset; and generates an environment estimation image corresponding to the reference pose information based on the image coordinate information corresponding to each target asset. The determination module is used to determine the true pose information of the target vehicle based on each of the reference pose information and its corresponding environmental estimation image, as well as the environmental captured image; Determining the true pose information of the target vehicle based on each of the reference pose information and its corresponding environmental estimation image, as well as the environmental captured image, includes: For each reference pose information, the image loss value corresponding to the reference pose information is determined based on the image difference between the environment estimation image corresponding to the reference pose information and the environment captured image. The reference pose information is updated according to the image loss value corresponding to each reference pose information to obtain the updated pose information; Once the updated pose information meets the preset conditions, the target pose information is filtered out from the updated pose information, and the target pose information is determined as the true pose information of the target vehicle.

6. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the steps of the pose estimation method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a processor to implement the steps of the pose estimation method according to any one of claims 1 to 4.

Images