Vehicle positioning method and device

Abstract

The present application proposes a vehicle positioning method and device, which relates to the field of autonomous driving. The method comprises: obtaining a distance transformation image of a road marking object in a road image according to a road image of the environment in which the vehicle is currently located; obtaining a vector sub-image of the road marking object from a global vector map; and determining the target positioning posture of the vehicle according to the distance transformation image and the vector sub-image. The present application avoids the use of an additional positioning layer, which can improve the positioning speed, reduce the cost of vehicle positioning, avoid the influence of light and obstacles on vehicle positioning, and improve the accuracy of vehicle positioning.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving, and in particular to a vehicle positioning method and device. Background Art

[0002] In the related art, vehicle positioning is generally performed using positioning layers or high-precision vector maps. However, due to the influence of light and dynamic obstacles, the method of using positioning layers to perform vehicle positioning is not very robust. Due to the influence of hardware and system configuration, the method of using high-precision vector maps to perform vehicle positioning is relatively expensive. Summary of the invention

[0003] The present application aims to solve one of the technical problems in the related art at least to a certain extent. To this end, one purpose of the present application is to propose a vehicle positioning method.

[0004] The second objective of the present application is to provide a vehicle positioning device.

[0005] The third objective of the present application is to provide another vehicle positioning device.

[0006] A fourth object of the present application is to provide a non-transitory computer-readable storage medium.

[0007] A fifth object of the present application is to provide a computer program product.

[0008] To achieve the above-mentioned purpose, the first embodiment of the present application proposes a vehicle positioning method, comprising:

[0009] According to the road image of the environment where the vehicle is currently located, a distance transformation image of the road marking object in the road image is obtained;

[0010] Get the vector submap of the road marking object from the global vector map;

[0011] The target positioning pose of the vehicle is determined based on the distance transformation image and the vector sub-image.

[0012] In a possible implementation, the target positioning posture of the vehicle is determined according to the distance transformation image and the vector subgraph, including: predicting the state information of the vehicle at the current moment based on the state information of the vehicle at the previous moment and the measurement data of the inertial measurement unit IMU at the current moment; obtaining the actual speed of the vehicle at the current moment; determining the target constraint function of the vehicle according to the state information at the current moment and the previous moment, the distance transformation image and the vector subgraph, as well as the measurement data of the IMU and the actual speed; optimizing the state information with the minimum target constraint function as a constraint condition to output the target positioning posture of the vehicle.

[0013] In one possible implementation, the target constraint function of the vehicle is determined based on the state information, the distance transformation image and the vector sub-graph, as well as the measurement data of the IMU and the actual vehicle speed, including: mapping the vector sub-graph to the distance transformation image according to the state information at the current moment, and obtaining the first image coordinates of the first pixel point on the vector sub-graph on the distance transformation image; determining the target constraint function based on the first image coordinates, the state information, the measurement data of the IMU and the actual vehicle speed.

[0014] In a possible implementation, a target constraint function of the vehicle is determined based on the first image coordinates, state information, IMU measurement data and actual vehicle speed, including: obtaining the grayscale value of each pixel from the distance transformation image according to the first image coordinates, and generating a first constraint parameter based on the grayscale value; converting the state information at the current moment to the world coordinate system, and generating a second constraint parameter based on the converted state information; converting the measured vehicle speed in the IMU measurement data to the vehicle body coordinate system, and generating a third constraint parameter based on the converted measured vehicle speed and the actual vehicle speed; performing IMU pre-integration on the IMU measurement data between the previous moment and the current moment, obtaining the IMU calculation increments at adjacent moments, and generating a fourth constraint parameter based on the IMU measured increments and the IMU calculation increments at adjacent moments; generating the target constraint function of the vehicle based on the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter.

[0015] In one possible implementation, a target constraint function of the vehicle is generated according to the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter, including: obtaining multiple historical moments that are adjacent to the current moment and temporally continuous; generating the target constraint function based on the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter corresponding to each historical moment, and the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter corresponding to the current moment.

[0016] In a possible implementation, after determining the target positioning posture of the vehicle, it also includes: obtaining a posture compensation amount based on IMU measurement data from the previous moment to the current moment; and correcting the target positioning posture according to the posture compensation amount to obtain the final positioning posture of the vehicle.

[0017] In a possible implementation, obtaining a vector subgraph of a road marking object from a global vector map includes: positioning the vehicle on the global vector map according to the position of the vehicle to obtain a candidate section of the road where the vehicle is located; and extracting a vector subgraph of the road marking object within the candidate section.

[0018] In a possible implementation, obtaining a distance transformation image of a road marking object in a road image includes: performing semantic segmentation on the road image to extract the road marking object from the road image; for any pixel point in the road image, selecting a target second pixel point corresponding to any pixel point from second pixel points of the road marking object; and obtaining a grayscale value of any pixel point based on the distance between the target second pixel point and any pixel point to generate a distance transformation image.

[0019] In a possible implementation, a target second pixel point corresponding to any pixel point is selected from the second pixel points of the road marking object, including: for each second pixel point, obtaining the distance between any pixel point and the second pixel point according to the second image coordinates of the second pixel point and the image coordinates of any pixel point; and selecting the second pixel point closest to any pixel point as the target second pixel point according to the distance between any pixel point and each second pixel point.

[0020] This application avoids the use of additional positioning layers, which can improve the positioning speed, reduce the cost of vehicle positioning, determine the target positioning posture of the vehicle based on the distance transformation image and vector sub-image, avoid the influence of light and obstacles on vehicle positioning, and improve the accuracy of vehicle positioning.

[0021] To achieve the above-mentioned purpose, the second embodiment of the present application proposes a vehicle positioning device, comprising:

[0022] A first acquisition module is used to acquire a distance transformation image of a road marking object in the road image according to the road image of the environment where the vehicle is currently located;

[0023] A second acquisition module is used to acquire a vector sub-map of a road marking object from a global vector map;

[0024] The third acquisition module is used to determine the target positioning posture of the vehicle according to the distance transformation image and the vector sub-image.

[0025] In a possible implementation, the third acquisition module is also used to: predict the state information of the vehicle at the current moment based on the state information of the vehicle at the previous moment and the measurement data of the inertial measurement unit IMU at the current moment; obtain the actual speed of the vehicle at the current moment; determine the target constraint function of the vehicle according to the state information at the current moment and the previous moment, the distance transformation image and the vector subgraph, the measurement data of the IMU and the actual speed; optimize the state information with the minimum target constraint function as the constraint condition to output the target positioning posture of the vehicle.

[0026] In a possible implementation, the third acquisition module is also used to: map the vector sub-image to the distance transformation image according to the state information at the current moment, and obtain the first image coordinates of the first pixel point on the vector sub-image on the distance transformation image; determine the target constraint function based on the first image coordinates, state information, IMU measurement data and actual vehicle speed.

[0027] In a possible implementation, the third acquisition module is also used to: acquire the grayscale value of each pixel point from the distance transformation image according to the first image coordinates, and generate a first constraint parameter based on the grayscale value; convert the state information at the current moment to the world coordinate system, and generate a second constraint parameter based on the converted state information; convert the measured vehicle speed in the IMU measurement data to the vehicle body coordinate system, and generate a third constraint parameter based on the converted measured vehicle speed and the actual vehicle speed; perform IMU pre-integration on the IMU measurement data between the previous moment and the current moment, acquire the IMU calculation increments at adjacent moments, and generate a fourth constraint parameter based on the IMU measured increments and the IMU calculation increments at adjacent moments; generate a target constraint function for the vehicle based on the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter.

[0028] In a possible implementation, the third acquisition module is also used to: acquire multiple historical moments that are adjacent to the current moment and continuous in time; generate a target constraint function based on the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter corresponding to each historical moment, and the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter corresponding to the current moment.

[0029] In a possible implementation, the device also includes a correction module, which is used to: obtain a posture compensation amount based on the IMU measurement data from the previous moment to the current moment; and correct the target positioning posture based on the posture compensation amount to obtain the final positioning posture of the vehicle.

[0030] In a possible implementation, the second acquisition module is further used to: locate the vehicle on the global vector map according to the position of the vehicle to obtain candidate sections of the road where the vehicle is located; and extract vector sub-graphs of road marking objects within the candidate sections.

[0031] In a possible implementation, the first acquisition module is also used to: perform semantic segmentation on the road image and extract the road marking object from the road image; for any pixel point in the road image, select the target second pixel point corresponding to any pixel point from the second pixel points of the road marking object; and obtain the grayscale value of any pixel point based on the distance between the target second pixel point and any pixel point to generate a distance transformation image.

[0032] In a possible implementation, the first acquisition module is also used to: for each second pixel point, acquire the distance between any pixel point and the second pixel point based on the second image coordinates of the second pixel point and the image coordinates of any pixel point; and select the second pixel point closest to any pixel point as the target second pixel point based on the distance between any pixel point and each second pixel point.

[0033] To achieve the above-mentioned purpose, the third aspect of the present application provides another vehicle positioning device, including:

[0034] at least one processor; and

[0035] a memory communicatively connected to at least one processor; wherein,

[0036] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can execute the vehicle positioning method provided in the embodiment of the first aspect of the present application.

[0037] To achieve the above-mentioned purpose, the fourth aspect embodiment of the present application proposes a computer-readable storage medium on which computer instructions are stored, wherein the computer instructions are used to enable a computer to execute the vehicle positioning method provided in the first aspect embodiment of the present application.

[0038] To achieve the above-mentioned purpose, the fifth aspect embodiment of the present application proposes a computer program product, including a computer program, which, when executed by a processor, implements the vehicle positioning method provided in the first aspect embodiment of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of a vehicle positioning method according to an embodiment of the present application;

[0040] Figure 2 is a schematic diagram of a road image according to an embodiment of the present application;

[0041] Figure 3 is a schematic diagram of a distance transformation image according to an embodiment of the present application;

[0042] Figure 4 is a flow chart of a vehicle positioning method according to another embodiment of the present application;

[0043] Figure 5 is a flowchart of determining a target constraint function according to another embodiment of the present application;

[0044] Figure 6 is a schematic diagram of a process for correcting a target positioning posture according to another embodiment of the present application;

[0045] Figure 7is a flow chart of a vehicle positioning method according to another embodiment of the present application;

[0046] Figure 8 is a schematic diagram of a road image according to an embodiment of the present application;

[0047] Fig. 9 is a schematic diagram of a distance transformation image according to an embodiment of the present application;

[0048] Fig.10 It is a flow chart of a vehicle positioning method according to an embodiment of the present application;

[0049] Fig.11 is a structural block diagram of a vehicle positioning device according to an embodiment of the present application;

[0050] Fig.12 It is a structural schematic diagram of a vehicle positioning device according to an embodiment of the present application. DETAILED DESCRIPTION

[0051] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0052] The vehicle positioning method and device according to the embodiments of the present application are described below with reference to the accompanying drawings.

[0053] Figure 1 is a flow chart of a vehicle positioning method according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0054] S101, acquiring a distance transformation image of a road marking object in the road image according to a road image of an environment in which a vehicle is currently located.

[0055] In the embodiment of the present application, an image acquisition device is arranged in front of the vehicle, through which the road image of the vehicle can be acquired. Optionally, the image acquisition device can be a camera. Optionally, the image acquisition device can be installed on the rearview mirror of the vehicle, or on the center console, roof, etc. of the vehicle.

[0056] In some implementations, image recognition is performed on the road image to extract the road marking object from the road image. In some implementations, semantic segmentation is performed on the road image to extract the road marking object from the road image. Semantic segmentation can classify the pixels in the road image, thereby distinguishing all the pixels in the road image that belong to the road marking object. For example, a fully convolutional network can be used to perform semantic segmentation on the road to extract the road marking object from the road image. In other implementations, edge detection can also be performed on the road image, such as using the Sobel operator or the Roberts operator to extract the contour in the road image, and then extract the road marking object.

[0057] Optionally, the road marking object may be one or more of a traffic sign, a lane line, and a traffic pole. In the embodiments of the present application, lane lines and traffic poles are used as examples for description.

[0058] like Figure 2 As shown in FIG. 1 , after extracting the lane lines and traffic poles in the road image, the road image is blurred. For any pixel in the road image, the gray value of any pixel in the road image represents the distance between the pixel and the road marking object, thereby obtaining the following Figure 3 For example, the road image can be converted into a grayscale image using a weighted method, a mean method, or a maximum method, and then the grayscale value in the grayscale image is corrected using the distance between each pixel in the road image and the road marking image, thereby obtaining a distance transformed image of the road marking object.

[0059] S102: Acquire a vector sub-map of a road marking object from a global vector map.

[0060] The global vector map contains a variety of three-dimensional graphics such as buildings, traffic signs, lane lines, traffic poles, etc., and the data volume is relatively large. In some implementations, in order to reduce unnecessary calculations, the global vector map can be screened for vector sub-graphs, and based on the road marking object, the vector sub-graph corresponding to the road marking object is obtained from the global vector map, that is, the vector sub-graph of the lane line and the traffic pole is obtained. Optionally, in the embodiment of the present application, the global vector map is a pre-stored vector map of the province or city where the vehicle is located.

[0061] S103, determining the target positioning posture of the vehicle according to the distance transformation image and the vector sub-image.

[0062] The coordinate system of the vector sub-graph is transformed, the vector sub-graph is back-projected to a 2D plane, and matched with the distance transformation image. Based on the matching degree of the back-projected vector sub-graph and the distance transformation image, the target positioning posture of the vehicle is determined. This method can avoid the influence of light and obstacles on vehicle positioning and improve the accuracy of vehicle positioning. For example, by evaluating the matching degree of the back-projected vector sub-graph and the distance transformation image, the target constraint function is determined, and the positioning posture is optimized. When the matching degree of the back-projected vector sub-graph and the distance transformation image is the highest, that is, the target constraint function is the minimum, the positioning posture converges to the true positioning posture, that is, the target positioning posture of the vehicle is determined.

[0063] In the embodiment of the present application, based on the road image of the environment in which the vehicle is currently located, a distance transformation image of the road marking object in the road image is obtained, a vector sub-map of the road marking object is obtained from the global vector map, and the target positioning posture of the vehicle is determined based on the distance transformation image and the vector sub-map. In the present application, the use of additional positioning layers is avoided, which can increase the positioning speed, reduce the cost of vehicle positioning, avoid the influence of light and obstacles on vehicle positioning, and improve the accuracy of vehicle positioning.

[0064] Figure 4 is a flow chart of a vehicle positioning method according to an embodiment of the present application. Figure 4 As shown, based on the above embodiment, the method further includes the following steps:

[0065] S401, predicting the state information of the vehicle at the current moment based on the state information of the vehicle at the previous moment and the measurement data of the inertial measurement unit IMU at the current moment.

[0066] In an embodiment of the present application, an inertial measurement unit IMU is provided on the vehicle to detect data such as the acceleration, angular velocity, and gravity of the vehicle, and to predict the status information of the vehicle at the current moment in combination with the status information of the vehicle at the previous moment, that is, to update the real-time status information of the vehicle.

[0067] In the embodiment of the present application, the status information of the vehicle at the last moment includes the position and speed information of the vehicle at the last moment.

[0068] Since the sampling frequency of IMU is high and the amount of data is very large, a measurement data is usually extracted every certain period of time, such as one every 0.1 seconds. The measurement data of IMU between two adjacent time intervals Δt can be integrated and recursively calculated. The recursive formula from time i to time j contains a lot of IMU measurement data in the middle. Optionally, in order to reduce the amount of calculation, IMU pre-integration can be used to predict the state information of the vehicle at the current moment, and write it in the form of IMU pre-integration, so that the pre-integration amount is not related to the state quantity at time i and time j, so that each time the state quantity at time i and time j is updated, there is no need to recalculate the pre-integration amount. In other words, by performing IMU pre-integration on the IMU measurement data, the current posture, speed and other information of the vehicle at the current moment can be quickly and accurately obtained.

[0069] S402, obtaining the actual speed of the vehicle at the current moment.

[0070] The speed detection device is used to obtain the actual speed of the vehicle at the current moment. For example, the actual speed at the current moment can be detected by a speed sensor. For example, the wheel speed sensor can be used to measure the rotation speed of the wheel, and then the actual speed at the current moment can be obtained based on the rotation speed of the wheel.

[0071] S403, determining the target constraint function of the vehicle according to the state information at the current moment and the previous moment, the distance transformation image and the vector sub-graph, as well as the measurement data of the IMU and the actual vehicle speed.

[0072] According to the current state information, the vector sub-image is mapped to the distance transformation image, that is, the vector sub-image is back-projected on the distance transformation image, so as to obtain the first image coordinates of the first pixel point on the vector sub-image on the distance transformation image.

[0073] Furthermore, the target constraint function is determined based on the first image coordinates, the state information at the current moment and the previous moment, the measurement data of the IMU and the actual vehicle speed.

[0074] In some implementations, the grayscale value of each pixel point is obtained from the distance transformation image according to the first image coordinates, and the first constraint parameter is generated based on the grayscale value. Optionally, the sum of the grayscale values ​​can be used as the first constraint parameter, or the average value of the grayscale values ​​can be used as the first constraint parameter.

[0075] In some implementations, the state information at the current moment is converted to the world coordinate system, and a second constraint parameter is generated based on the converted state information. Optionally, the pitch angle, roll angle and distance between the vehicle and the road surface on which the vehicle is located can be obtained based on the converted state information, wherein the pitch angle constrains the angle between the vehicle and the road surface, the roll angle constrains the angle at which the vehicle flips, and the distance between the vehicle and the road surface on which the vehicle is located constrains the vertical distance between the center point of the vehicle and the road plane. The sum or weighted value of the pitch angle, roll angle and distance between the vehicle and the road surface on which the vehicle is located is used as the second constraint parameter. Among them, the world coordinate system is the absolute coordinate system of this application, and before the vehicle body coordinate system is established, the coordinates of all points on the screen are determined by the origin of the coordinate system.

[0076] In some implementations, the measured vehicle speed in the measurement data of the IMU is converted to the vehicle body coordinate system, and a third constraint parameter is generated based on the converted measured vehicle speed and the actual vehicle speed. Optionally, the difference between the measured vehicle speed and the actual vehicle speed can be used as the third constraint parameter. The vehicle body coordinate system is a special moving coordinate system used to describe the movement of the vehicle; its origin coincides with the center of mass. When the vehicle is stationary on a horizontal road, the X-axis is parallel to the ground and points to the front of the vehicle, the Z-axis passes through the center of mass of the vehicle and points upward, and the Y-axis points to the left of the driver. In the present application, converting the world coordinate system into the vehicle body coordinate system can obtain a more accurate instantaneous speed of the vehicle, thereby improving the accuracy of vehicle positioning.

[0077] In some implementations, IMU pre-integration is performed on the IMU measurement data between the previous moment and the current moment, the IMU calculation increments at adjacent moments are obtained, and the fourth constraint parameter is generated based on the IMU measured increments and the IMU calculation increments at adjacent moments. Optionally, the difference between the IMU measured increment and the IMU calculation increment can be used as the fourth constraint parameter.

[0078] The target constraint function of the vehicle is generated according to the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter. For example, the sum of the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter can be used as the target constraint function.

[0079] S404, optimizing the state information with the minimum target constraint function as a constraint condition to output the target positioning posture of the vehicle.

[0080] Optionally, the objective constraint function can be input into the optimizer, and the state information is iteratively updated with the minimum objective constraint function as a constraint condition to optimize the state information. The state information iterated when the objective constraint function is minimized is determined as the final state information at the current moment, and then the target positioning posture of the vehicle is obtained based on the optimized state information at the current moment.

[0081] In the embodiment of the present application, based on the state information of the vehicle at the previous moment and the measurement data of the inertial measurement unit IMU at the current moment, the state information of the vehicle at the current moment is predicted, and the actual speed of the vehicle at the current moment is obtained. According to the state information, the distance transformation image and the vector subgraph, as well as the measurement data of the IMU and the actual speed, the target constraint function of the vehicle is determined, and the state information is optimized with the minimum target constraint function as the constraint condition to output the target positioning posture of the vehicle. In the present application, avoiding the use of additional positioning layers can increase the positioning speed, reduce the cost of vehicle positioning, avoid the influence of light and obstacles on vehicle positioning, and improve the accuracy of vehicle positioning.

[0082] In some implementations, in order to improve the accuracy of vehicle positioning, when obtaining the target positioning posture of the vehicle, it is necessary not only to refer to the constraint parameters of the current moment, but also to consider multiple historical moments that are adjacent to the current moment and continuous in time. That is to say, multiple historical moments that are adjacent to the current moment and continuous in time are obtained, and a target constraint function is generated based on the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter corresponding to each historical moment, and the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter corresponding to the current moment, and then the state information is optimized with the minimum target constraint function as the constraint condition to output the target positioning posture of the vehicle.

[0083] like Figure 5 As shown, in the embodiment of the present application, an example is given for explanation. If the current moment is moment t, the state information of moments t-4, t-3, t-2, t-1, and t are obtained respectively, wherein moment t-4 corresponds to state information A, moment t-3 corresponds to state information B, moment t-2 corresponds to state information C, moment t-1 corresponds to state information D, and moment t corresponds to state information E. For any moment, the first constraint parameter, the second constraint parameter, and the third constraint parameter of the moment are obtained according to the state information of any moment, and the fourth constraint parameter is obtained according to the state information of any moment and the previous moment of any moment. For example, for moment t-3, the first constraint parameter X is obtained according to the state information B at moment t-3. B , the second constraint parameter Y B and the third constraint parameter Z B , obtain the fourth constraint parameter F according to the state information B at time t-3 and the state information A at time t-4 B , and then generate the target constraint function according to the constraint parameters at each moment, and optimize the state information with the minimum target constraint function as the constraint condition to output the target positioning posture of the vehicle.

[0084] In the implementation, when executing step S101 and step S102, it may take a certain amount of time, so that some data in the state information has a lag, such as Figure 6 As shown in order to reduce errors and improve the positioning accuracy of the vehicle, in an embodiment of the present application, state prediction is performed based on the IMU measurement data from the previous moment to the current moment, and the posture compensation amount is obtained. Then, based on the posture compensation amount, the target positioning posture is corrected to obtain the final positioning posture of the vehicle.

[0085] This application avoids the use of additional positioning layers, which can improve the positioning speed, reduce the cost of vehicle positioning, avoid the influence of light and obstacles on vehicle positioning, and use posture compensation to correct the target positioning posture, thereby improving the accuracy of vehicle positioning.

[0086] Figure 7 is a flow chart of a vehicle positioning method according to another embodiment of the present application. Figure 7 As shown, based on the above embodiment, the method further includes the following steps:

[0087] S701, performing semantic segmentation on a road image, and extracting road marking objects from the road image.

[0088] For the description of step S701, reference may be made to the relevant contents of the above embodiment, which will not be repeated here.

[0089] S702: For any pixel point in the road image, select a target second pixel point corresponding to the any pixel point from the second pixel points of the road marking object.

[0090] For each second pixel point included in the road marking object, the distance between any pixel point and the second pixel point is obtained according to the second image coordinates of the second pixel point and the image coordinates of any pixel point.

[0091] According to the distance between any pixel point and each second pixel point, the second pixel point closest to any pixel point is selected as the target second pixel point.

[0092] S703, acquiring the gray value of any pixel point according to the distance between the target second pixel point and any pixel point, so as to generate a distance transformation image.

[0093] According to the distance between the second pixel point of the target and any pixel point, the gray value of any pixel point is obtained, that is, the smaller the distance between the second pixel point of the target and any pixel point, the smaller the gray value of any pixel point, and the larger the distance between the second pixel point of the target and any pixel point, the larger the gray value of any pixel point. Thus, a distance transformation image is generated. The distance transformation image in the embodiment of the present application is a gray image.

[0094] In this application, the use of additional positioning layers is avoided, which can increase the positioning speed, reduce the cost of vehicle positioning, avoid the influence of light and obstacles on vehicle positioning, and improve the accuracy of vehicle positioning.

[0095] like Figure 8 , Fig. 9 As shown, in some implementations, obtaining a vector subgraph of a road marking object from a global vector map also includes positioning the vehicle on the global vector map according to the vehicle's position, obtaining a candidate section of the road where the vehicle is located, and extracting a vector subgraph of the road marking object within the candidate section. In some implementations, the position of the next intersection is determined according to the vehicle's position and the road where the vehicle is located, and the section between the vehicle's position and the position of the next intersection is used as a candidate section, and the vector subgraph of the road marking object within the candidate section is extracted. In some implementations, a section within a preset distance in the driving direction is determined according to the vehicle's position and the road where the vehicle is located, and the section within the preset distance is used as a candidate section, and the vector subgraph of the road marking object within the candidate section is extracted.

[0096] In the embodiment of the present application, according to the position of the vehicle, positioning is performed on the global vector map, the candidate road section where the vehicle is located is obtained, and the vector sub-map of the road marking object in the candidate road section is extracted. The embodiment of the present application reduces the amount of calculation, increases the speed of vehicle positioning, and improves the accuracy of vehicle positioning.

[0097] Fig.10 FIG. 1 is a flow chart of a vehicle positioning method according to an embodiment of the present application. Fig.11 As shown, based on the above embodiment, the method further includes the following steps:

[0098] Collect the road image at the current moment, perform semantic segmentation on the road image, obtain the road marking object, and then obtain the distance transformation image through distance transformation. Back-project the global vector map to obtain the vector sub-map of the road marking object. Based on the actual speed of the collected vehicle and the measurement data of the inertial measurement unit IMU at the current moment, input the state information, distance transformation image and vector sub-map, as well as the IMU measurement data and actual speed into the sliding window optimizer to obtain the predicted state information and optimize the predicted state information through the determined objective function. Perform state prediction based on the IMU measurement data from the previous moment to the current moment, correct the predicted state information, and obtain the target positioning posture of the vehicle.

[0099] In this application, the use of additional positioning layers is avoided, which can improve the positioning speed, reduce the cost of vehicle positioning, avoid the influence of light and obstacles on vehicle positioning, and improve the accuracy of vehicle positioning.

[0100] Fig.11 is a structural block diagram of a vehicle positioning device according to an embodiment of the present application. Fig.11 As shown, based on the same application concept, the embodiment of the present application further provides a vehicle positioning device 1100, including:

[0101] The first acquisition module 1110 is used to acquire a distance transformation image of a road marking object in the road image according to the road image of the environment where the vehicle is currently located;

[0102] A second acquisition module 1120, configured to acquire a vector sub-map of a road marking object from a global vector map;

[0103] The third acquisition module 1130 is used to determine the target positioning posture of the vehicle according to the range transformation image and the vector sub-image.

[0104] Furthermore, in a possible implementation of the embodiment of the present application, the third acquisition module 1130 is further configured to:

[0105] Based on the vehicle's state information at the previous moment and the measurement data of the inertial measurement unit IMU at the current moment, predict the vehicle's state information at the current moment;

[0106] Get the actual speed of the vehicle at the current moment;

[0107] Determine the target constraint function of the vehicle based on the state information, the distance transformation image and the vector sub-graph, as well as the IMU measurement data and the actual vehicle speed;

[0108] Taking the minimization of the objective constraint function as the constraint condition, the state information is optimized to output the target positioning posture of the vehicle.

[0109] Furthermore, in a possible implementation of the embodiment of the present application, the third acquisition module 1130 is further configured to:

[0110] According to the state information at the current moment, the vector sub-image is mapped to the distance transformation image to obtain the first image coordinates of the first pixel point on the vector sub-image on the distance transformation image;

[0111] The target constraint function is determined based on the first image coordinates, the state information, the measurement data of the IMU and the actual vehicle speed.

[0112] Furthermore, in a possible implementation of the embodiment of the present application, the third acquisition module 1130 is further configured to:

[0113] Acquire a grayscale value of each pixel point from the distance transformation image according to the first image coordinates, and generate a first constraint parameter based on the grayscale value;

[0114] Convert the current state information to the world coordinate system, and generate the second constraint parameter according to the converted state information;

[0115] The measured vehicle speed in the IMU measurement data is converted into a vehicle body coordinate system, and a third constraint parameter is generated according to the converted measured vehicle speed and the actual vehicle speed;

[0116] Performing IMU pre-integration on the IMU measurement data between the previous moment and the current moment, obtaining the IMU calculation increments at adjacent moments, and generating the fourth constraint parameter according to the IMU measured increments and the IMU calculation increments at adjacent moments;

[0117] A target constraint function of the vehicle is generated according to the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter.

[0118] Furthermore, in a possible implementation of the embodiment of the present application, the third acquisition module 1130 is further configured to:

[0119] Get multiple historical moments that are adjacent to the current moment and continuous in time;

[0120] Based on the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter corresponding to each historical moment and the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter corresponding to the current moment, an objective constraint function is generated.

[0121] Furthermore, in a possible implementation of the embodiment of the present application, the vehicle positioning device 1100 further includes a correction module 1140, which is used to:

[0122] Obtain the posture compensation amount based on the IMU measurement data from the previous moment to the current moment;

[0123] According to the posture compensation amount, the target positioning posture is corrected to obtain the final positioning posture of the vehicle.

[0124] Furthermore, in a possible implementation of the embodiment of the present application, the second acquisition module 1120 is further configured to:

[0125] According to the position of the vehicle, the vehicle is positioned on the global vector map to obtain the candidate road section where the vehicle is located;

[0126] Extract vector sub-graphs of road marking objects within the candidate road segment.

[0127] Furthermore, in a possible implementation of the embodiment of the present application, the first acquisition module 1110 is further configured to:

[0128] Perform semantic segmentation on road images and extract road marking objects from road images;

[0129] For any pixel point in the road image, a target second pixel point corresponding to any pixel point is selected from the second pixel points of the road marking object;

[0130] According to the distance between the second pixel point of the target and any pixel point, the gray value of any pixel point is obtained to generate a distance transformation image.

[0131] Furthermore, in a possible implementation of the embodiment of the present application, the first acquisition module 1110 is further configured to:

[0132] For each second pixel point, according to the second image coordinates of the second pixel point and the image coordinates of any pixel point, obtain the distance between any pixel point and the second pixel point;

[0133] According to the distance between any pixel point and each second pixel point, the second pixel point closest to any pixel point is selected as the target second pixel point.

[0134] In this application, the use of additional positioning layers is avoided, which can improve the positioning speed, reduce the cost of vehicle positioning, avoid the influence of light and obstacles on vehicle positioning, and improve the accuracy of vehicle positioning.

[0135] It should be noted that the above description of the vehicle positioning method is also applicable to the vehicle positioning device of the present application and will not be repeated here.

[0136] Based on the same application concept, an embodiment of the present application also provides a vehicle positioning device.

[0137] Fig.12 This is a schematic diagram of the structure of the vehicle positioning device provided in the embodiment of the present application. Fig.12 As shown, the vehicle positioning device 1200 includes a storage medium 1210, a processor 1220, and a computer program product stored in the storage 1210 and executable on the processor 1220. When the processor executes the computer program, the aforementioned vehicle positioning method is implemented.

[0138] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0139] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0140] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0141] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0142] Based on the same application concept, an embodiment of the present application further provides a computer-readable storage medium on which computer instructions are stored, wherein the computer instructions are used to enable a computer to execute the vehicle positioning method in the above embodiment.

[0143] Based on the same application concept, an embodiment of the present application further provides a computer program product, including a computer program, which, when executed by a processor, performs the vehicle positioning method in the above embodiment.

[0144] It should be noted that in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present application may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0145] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0146] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0147] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A vehicle positioning method, characterized in that: include: According to a road image of an environment in which the vehicle is currently located, obtaining a distance transformation image of a road marking object in the road image; Acquire a vector sub-map of the road marking object from a global vector map; Determining a target positioning posture of the vehicle according to the range transform image and the vector sub-graph; Determining the target positioning posture of the vehicle according to the distance transformation image and the vector sub-image includes: Predicting the state information of the vehicle at the current moment based on the state information of the vehicle at the previous moment and the measurement data of the inertial measurement unit IMU at the current moment; Obtaining the actual speed of the vehicle at the current moment; Determine the target constraint function of the vehicle according to the state information at the current moment and the previous moment, the distance transformation image and the vector subgraph, as well as the measurement data of the IMU and the actual vehicle speed; Taking the minimization of the objective constraint function as a constraint condition, the state information at the current moment is optimized to output the target positioning posture of the vehicle.

2. The method according to claim 1, characterized in that Determining the target constraint function of the vehicle according to the state information at the current moment and the previous moment, the distance transformation image and the vector subgraph, and the measurement data of the IMU and the actual vehicle speed includes: According to the state information at the current moment, mapping the vector sub-image to the distance transformation image, and obtaining a first image coordinate of a first pixel point on the vector sub-image on the distance transformation image; The objective constraint function is determined based on the first image coordinates, the state information at the current moment and the previous moment, the measurement data of the IMU and the actual vehicle speed.

3. The method according to claim 2, characterized in that The determining of the target constraint function of the vehicle based on the first image coordinates, the state information at the current moment and the previous moment, the measurement data of the IMU and the actual vehicle speed includes: Acquire a grayscale value of each pixel point from the distance transformation image according to the first image coordinates, and generate a first constraint parameter based on the grayscale value; Converting the state information at the current moment into a world coordinate system, and generating a second constraint parameter according to the converted state information; Converting the measured vehicle speed in the measurement data of the IMU into a vehicle body coordinate system, and generating a third constraint parameter according to the converted measured vehicle speed and the actual vehicle speed; Performing IMU pre-integration on the IMU measurement data between the previous moment and the current moment, obtaining the IMU calculation increments at adjacent moments, and generating fourth constraint parameters according to the IMU measured increments at the adjacent moments and the IMU calculation increments; An objective constraint function of the vehicle is generated according to the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter.

4. The method according to claim 3, characterized in that The step of generating the target constraint function of the vehicle according to the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter comprises: Get multiple historical moments that are adjacent to the current moment and continuous in time; The objective constraint function is generated based on the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter corresponding to each historical moment, and the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter corresponding to the current moment.

5. The method according to any one of claims 1 to 4, characterized in that: After determining the target positioning posture of the vehicle, the method further includes: Obtain the posture compensation amount based on the IMU measurement data from the previous moment to the current moment; The target positioning posture is corrected according to the posture compensation amount to obtain the final positioning posture of the vehicle.

6. The method according to any one of claims 1 to 4, characterized in that: The step of acquiring the vector sub-map of the road marking object from the global vector map comprises: According to the position of the vehicle, positioning is performed on the global vector map to obtain a candidate section of the road where the vehicle is located; A vector sub-graph of the road marking object within the candidate road segment is extracted.

7. The method according to any one of claims 1 to 4, characterized in that: The step of acquiring a distance transformation image of a road marking object in the road image comprises: Performing semantic segmentation on the road image to extract the road marking object from the road image; For any pixel point in the road image, selecting a target second pixel point corresponding to the any pixel point from the second pixel points of the road marking object; According to the distance between the target second pixel point and the any pixel point, the grayscale value of the any pixel point is acquired to generate the distance transformation image.

8. The method according to claim 7, characterized in that The step of selecting a target second pixel point corresponding to any one of the pixel points from the second pixel points of the road marking object comprises: For each of the second pixel points, acquiring a distance between the any pixel point and the second pixel point according to the second image coordinates of the second pixel point and the image coordinates of the any pixel point; According to the distance between any one pixel point and each of the second pixel points, a second pixel point closest to any one pixel point is selected as the target second pixel point.

9. A vehicle positioning device, characterized in that: include: A first acquisition module is used to acquire a distance transformation image of a road marking object in the road image according to a road image of an environment in which the vehicle is currently located; A second acquisition module, used for acquiring a vector sub-map of the road marking object from a global vector map; A third acquisition module, used to determine the target positioning posture of the vehicle according to the distance transformation image and the vector sub-image; The third acquisition module is further used for: Predicting the state information of the vehicle at the current moment based on the state information of the vehicle at the previous moment and the measurement data of the inertial measurement unit IMU at the current moment; Obtaining the actual speed of the vehicle at the current moment; Determine the target constraint function of the vehicle according to the state information at the current moment and the previous moment, the distance transformation image and the vector subgraph, as well as the measurement data of the IMU and the actual vehicle speed; The state information is optimized with the minimum of the objective constraint function as a constraint condition to output the target positioning posture of the vehicle.

10. The device according to claim 9, characterized in that The third acquisition module is further used for: According to the state information at the current moment, mapping the vector sub-image to the distance transformation image, and obtaining a first image coordinate of a first pixel point on the vector sub-image on the distance transformation image; The objective constraint function is determined based on the first image coordinates, the state information at the current moment and the previous moment, the measurement data of the IMU and the actual vehicle speed.

11. The device according to claim 10, characterized in that The third acquisition module is further used for: Acquire a grayscale value of each pixel point from the distance transformation image according to the first image coordinates, and generate a first constraint parameter based on the grayscale value; Converting the state information at the current moment into a world coordinate system, and generating a second constraint parameter according to the converted state information; Converting the measured vehicle speed in the measurement data of the IMU into a vehicle body coordinate system, and generating a third constraint parameter according to the converted measured vehicle speed and the actual vehicle speed; Performing IMU pre-integration on the IMU measurement data between the previous moment and the current moment, obtaining the IMU calculation increments at adjacent moments, and generating fourth constraint parameters according to the IMU measured increments at the adjacent moments and the IMU calculation increments; An objective constraint function of the vehicle is generated according to the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter.

12. The device according to claim 11, characterized in that The third acquisition module is further used for: Get multiple historical moments that are adjacent to the current moment and continuous in time; The objective constraint function is generated based on the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter corresponding to each historical moment, and the first constraint parameter, the second constraint parameter, the third constraint parameter and the fourth constraint parameter corresponding to the current moment.

13. The device according to any one of claims 9 to 12, characterized in that: Also includes correction modules for: Obtain the posture compensation amount based on the IMU measurement data from the previous moment to the current moment; The target positioning posture is corrected according to the posture compensation amount to obtain the final positioning posture of the vehicle.

14. The device according to any one of claims 9 to 12, characterized in that: The second acquisition module is further used for: According to the position of the vehicle, positioning is performed on the global vector map to obtain a candidate section of the road where the vehicle is located; A vector sub-graph of the road marking object within the candidate road segment is extracted.

15. The device according to any one of claims 9 to 12, characterized in that: The first acquisition module is further used for: Performing semantic segmentation on the road image to extract the road marking object from the road image; For any pixel point in the road image, selecting a target second pixel point corresponding to the any pixel point from the second pixel points of the road marking object; According to the distance between the target second pixel point and the any pixel point, the grayscale value of the any pixel point is acquired to generate the distance transformation image.

16. The device according to claim 15, characterized in that The first acquisition module is further used for: For each of the second pixel points, acquiring a distance between the any pixel point and the second pixel point according to the second image coordinates of the second pixel point and the image coordinates of the any pixel point; According to the distance between any one pixel point and each of the second pixel points, a second pixel point closest to any one pixel point is selected as the target second pixel point.

17. A vehicle positioning device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

18. A computer-readable storage medium having computer instructions stored thereon, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

Images