Vehicle positioning method, device, equipment, medium and vehicle

Abstract

The present disclosure relates to a vehicle positioning method, device, equipment, medium and vehicle. The vehicle positioning method includes: obtaining a target map, the target map includes point cloud features of static objects in a preset area; obtaining image information, the image information includes camera image information and radar image information; performing semantic recognition on the camera image information to obtain pixel information of static objects in the camera image information; extracting point cloud features of static objects from radar image information based on the pixel information of static objects; matching the point cloud features of static objects extracted from radar image information with target point cloud features of static objects in the target map to obtain the positioning result of the vehicle. According to the embodiment of the present disclosure, the rich semantic information of camera vision and the point cloud features of radar can be combined to improve the robustness of vehicle positioning.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent driving technology, and in particular to a vehicle positioning method, device, equipment, medium and vehicle. Background Art

[0002] At present, multi-source positioning information is required in autonomous driving technology, among which the main positioning methods in autonomous driving technology are GPS positioning and lidar positioning.

[0003] The most important issues in the process of using LiDAR for positioning are feature extraction and association. For example, static features such as curbs, signboards, street lights, lane lines, etc. are very stable positioning features. However, due to the sparseness of LiDAR beams, it is difficult to directly perform semantic extraction through deep learning like vision. Currently, extraction is mostly based on geometric rules, which results in complicated rule design. It is even difficult to design geometric rules for some features. At the same time, features without height values ​​such as lane lines can only be extracted through laser intensity, and most of the LiDAR intensity information on the market is not very stable. For many reasons, the robustness of positioning using only a single LiDAR cannot be guaranteed. Summary of the invention

[0004] In order to solve the above technical problems, the present disclosure provides a vehicle positioning method, device, equipment, medium and vehicle.

[0005] A first aspect of an embodiment of the present disclosure provides a vehicle positioning method, including:

[0006] Acquire a target map, the target map including point cloud features of static objects in a preset area;

[0007] Acquire image information, the image information includes camera image information and radar image information;

[0008] Perform semantic recognition on the camera image information to obtain pixel information of static objects in the camera image information;

[0009] Based on the pixel information of the static object, the point cloud features of the static object are extracted from the radar image information;

[0010] The point cloud features of the static objects extracted from the radar image information are matched with the target point cloud features of the static objects in the target map to obtain the vehicle positioning result.

[0011] A second aspect of an embodiment of the present disclosure provides a vehicle positioning device, comprising:

[0012] A map acquisition module is used to acquire a target map, wherein the target map includes point cloud features of static objects in a preset area;

[0013] An image acquisition module is used to acquire image information, which includes camera image information and radar image information;

[0014] A semantic recognition module is used to perform semantic recognition on the camera image information to obtain pixel information of static objects in the camera image information;

[0015] A feature extraction module is used to extract point cloud features of static objects from radar image information based on pixel information of static objects;

[0016] The feature matching module is used to match the point cloud features of static objects extracted from the radar image information with the target point cloud features of static objects in the target map to obtain the vehicle positioning result.

[0017] A third aspect of the embodiments of the present disclosure provides an electronic device, including:

[0018] processor;

[0019] A memory for storing executable instructions;

[0020] The processor is used to read executable instructions from the memory and execute the executable instructions to implement the vehicle positioning method provided in the first aspect above.

[0021] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the vehicle positioning method provided by the first aspect.

[0022] A fifth aspect of an embodiment of the present disclosure provides a vehicle, which includes the electronic device provided by the third aspect.

[0023] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0024] The vehicle positioning method, device, equipment, medium and vehicle provided by the embodiments of the present disclosure can obtain a target map, the target map includes point cloud features of static objects in a preset area, obtain image information, the image information includes camera image information and radar image information, perform semantic recognition on the acquired camera image information, obtain pixel information of the static objects in the camera image information, and then extract point cloud features of the static objects from the radar image information based on the pixel information of the static objects in the camera image information, match the point cloud features of the static objects extracted from the radar image information with the target point cloud features of the static objects in the target map, and obtain the positioning result of the vehicle. Therefore, based on the camera image information and the radar image information, the robustness of the vehicle positioning can be improved by combining the rich semantic information of the camera vision and the point cloud features of the radar. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0027] Figure 1 is a flow chart of a vehicle positioning method provided by an embodiment of the present disclosure;

[0028] Figure 2 is a schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0029] Figure 3 is a flow chart of another vehicle positioning method provided by an embodiment of the present disclosure;

[0030] Figure 4 is a structural schematic diagram of a vehicle positioning device provided by an embodiment of the present disclosure;

[0031] Figure 5 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0033] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0034] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0035] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0036] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0037] Generally speaking, the most important thing in the process of laser radar positioning is feature extraction and association. For example, static features such as curbs, signboards, street lights, lane lines, etc. are very stable positioning features. However, due to the sparse laser radar beams, it is difficult to directly extract semantics through deep learning like vision. At present, it is mostly extracted based on geometric rules, which results in complicated rule design. Some features are even difficult to design geometric rules. At the same time, features without height values ​​such as lane lines can only be extracted through laser intensity, and most of the laser radar intensity information on the market is not very stable. For many reasons, the robustness of positioning using only a single laser radar cannot be guaranteed. Camera vision is naturally rich in semantic information, and image-based deep learning is very mature, but it has the disadvantage of not being able to accurately measure distance without depth information. In response to this problem, the embodiments of the present disclosure provide a vehicle positioning method, which is introduced in conjunction with specific embodiments below.

[0038] Figure 1 is a flow chart of a vehicle positioning method provided by an embodiment of the present disclosure. The method can be executed by a vehicle positioning device. The vehicle positioning device can be implemented in software and / or hardware. The vehicle positioning device can be configured in an electronic device, such as a server or a terminal, wherein the terminal specifically includes a mobile phone, a computer or a tablet computer, etc. In addition, the method can be applied to Figure 2 The application scenario shown includes a vehicle 21 and an electronic device 22. It can be understood that the vehicle positioning method provided in the embodiment of the present disclosure can also be applied in other scenarios.

[0039] Combine the following Figure 2The application scenario shown is Figure 1 The vehicle positioning method shown is introduced, for example, Figure 2 The electronic device 22 in the embodiment can perform the method. Figure 1 As shown, the vehicle positioning method provided in this embodiment includes the following steps.

[0040] S110: Acquire a target map, where the target map includes point cloud features of static objects in a preset area.

[0041] In an embodiment of the present disclosure, the electronic device obtains a target map when or before the vehicle needs to be positioned, wherein the target map includes point cloud features of static objects within a preset area.

[0042] In the embodiment of the present disclosure, a static object refers to a static object that does not move due to the movement of the vehicle. For example, it can be Figure 2 23 and 24, wherein 23 represents a road sign and 24 represents a tree. Of course, they can also be some other stationary objects, such as buildings, etc., which are not limited here.

[0043] Furthermore, the target map includes point cloud features of static objects such as curbs, signboards, street lights, lane lines, etc.

[0044] In the disclosed embodiment, the preset area may be an area pre-selected and stored in the electronic device, and the vehicle in the preset area may be located by static objects in the preset area. For example, the preset area may be a certain provincial or municipal administrative area, or a certain port area, etc., which is not limited here.

[0045] Optionally, the point cloud features of the static object can be acquired through a radar device or a combined navigation device.

[0046] In some embodiments of the present disclosure, when or before the vehicle needs to be positioned, the electronic device obtains a map corresponding to a preset area from a database pre-stored in a memory and uses it as a target map.

[0047] In other embodiments of the present disclosure, when there is a need to locate a vehicle or before there is a need to locate the vehicle, the electronic device obtains point cloud data within a preset area through a point cloud acquisition device such as a radar device, a combined navigation device, etc. The point cloud acquisition device uploads the acquired point cloud data within the preset area to the electronic device, and the electronic device obtains a target map based on the point cloud data within the preset area.

[0048] S120: Acquire image information, where the image information includes camera image information and radar image information.

[0049] In the embodiment of the present disclosure, the electronic device obtains image information when it is necessary to locate the vehicle, and the image information includes camera image information and radar image information.

[0050] In the embodiment of the present disclosure, the image information may be understood as image information of objects around the vehicle to be located, including image information of static objects and dynamic objects.

[0051] In the embodiment of the present disclosure, a camera device and a radar device are simultaneously provided on the vehicle, and the timestamps of the camera device and the radar device are synchronized, and the internal and external parameters of the camera device and the internal and external parameters of the radar device are pre-calibrated and stored in the electronic device.

[0052] Furthermore, the camera device may be any device that forms and records images using the principle of optical imaging, such as an optical camera.

[0053] The radar device may be a radar system that transmits detection signals to detect characteristic quantities such as the position and speed of a target, and may be, for example, a laser radar, a microwave radar, etc., without limitation herein.

[0054] In the embodiment of the present disclosure, the camera image information is image information acquired by a camera device.

[0055] Radar image information is image information obtained by radar equipment.

[0056] Specifically, when the electronic device needs to locate the vehicle, it starts the camera device and the radar device installed on the vehicle, obtains camera image information and radar image information through the camera device and the radar device respectively, and the camera device and the radar device upload the camera image information and radar image information respectively obtained to the electronic device, and the electronic device receives and responds to the camera image information and the radar image information, thereby obtaining the image information.

[0057] S130: Perform semantic recognition on the camera image information to obtain pixel information of static objects in the camera image information.

[0058] In the disclosed embodiment, after acquiring the image information, the electronic device performs semantic recognition on the camera image information to obtain pixel information of the static object in the camera image information.

[0059] In an embodiment of the present disclosure, semantic recognition may include semantic segmentation of objects in camera image information according to semantic information, such as classifying objects, etc., wherein objects may be semantically segmented according to category labels, object identifiers, colors, etc.

[0060] Specifically, after acquiring the image information, the electronic device performs semantic recognition on the camera image information in the image information to obtain a semantic recognition result, and obtains pixel information of the static object in the camera image information according to the semantic recognition result.

[0061] S140: extracting point cloud features of the static object from the radar image information based on the pixel information of the static object.

[0062] In the embodiment of the present disclosure, after obtaining the pixel information of the static object in the camera image information, the electronic device extracts the point cloud features of the static object from the radar image information based on the pixel information of the static object.

[0063] Specifically, after obtaining the pixel information of the static object in the camera image information, the electronic device obtains the coordinate information of the static object through the pixel information of the static object, and then extracts the point cloud features of the static object from the radar image information corresponding to the camera image information based on the coordinates of the static object in the camera image information.

[0064] S150 , matching the point cloud features of the static object extracted from the radar image information with the target point cloud features of the static object in the target map to obtain a positioning result of the vehicle.

[0065] In the disclosed embodiment, after the electronic device extracts the point cloud features of the static object from the radar image information, it matches the point cloud features of the static object extracted from the radar image information with the target point cloud features of the static object in the target map to obtain the vehicle positioning result.

[0066] In some embodiments of the present disclosure, the electronic device may match the point cloud features of static objects extracted from radar image information with the target point cloud features of static objects in the target map through an iterative nearest point algorithm, and then obtain a vehicle positioning result based on the matching result.

[0067] In some other embodiments of the present disclosure, the electronic device can match the point cloud features of static objects extracted from radar image information with the target point cloud features of static objects in the target map through a normal distribution transformation algorithm, and then obtain the vehicle positioning result based on the matching result.

[0068] In the disclosed embodiment, the electronic device is able to obtain a target map, the target map including point cloud features of static objects within a preset area, obtain image information, the image information including camera image information and radar image information, perform semantic recognition on the obtained camera image information, obtain pixel information of the static objects in the camera image information, and then extract point cloud features of the static objects from the radar image information based on the pixel information of the static objects in the camera image information, match the point cloud features of the static objects extracted from the radar image information with the target point cloud features of the static objects in the target map, and obtain a vehicle positioning result. As a result, the robustness of vehicle positioning can be improved based on the camera image information and the radar image information, combined with the rich semantic information of camera vision and the point cloud features of the radar.

[0069] Based on the above embodiment, obtaining the target map in S110 may specifically include: obtaining target point cloud data within a preset area, and obtaining a first map within the preset area based on the target point cloud data; and performing data processing on the point cloud data in the first map to obtain the target map.

[0070] In the embodiment of the present disclosure, the target point cloud data may be acquired by a radar sensor.

[0071] Optionally, real-time location information can be obtained through integrated navigation.

[0072] Furthermore, combined navigation can be understood as a navigation system that integrates various navigation devices and is controlled by a monitor and a computer. For example, it can be an Inertial Navigation System (INS) / Global Positioning System (GPS) combined navigation, which is not limited here.

[0073] In some embodiments of the present disclosure, the electronic device can obtain real-time position information through combined navigation, and at the same time obtain target point cloud data within a preset area through a radar sensor, and perform trajectory stitching on the target point cloud data according to the trajectory stitching method based on the real-time position information and the target point cloud data to obtain a first map within the preset area.

[0074] In some other embodiments of the present disclosure, the electronic device may construct a map based on the target point cloud data in a Simultaneous Localization and Mapping (SLAM) manner to obtain a first map within a preset area.

[0075] Furthermore, after obtaining the first map within the preset area, the electronic device processes the point cloud data in the first map to obtain the target map.

[0076] In the embodiments of the present disclosure, the data processing of point cloud data may include data preprocessing, data stitching, feature line extraction, data modeling and other processing.

[0077] Among them, data preprocessing may include removing noise points in the point cloud data, such as isolated points, etc., and may also include deleting unwanted or unnecessary point cloud data in the point cloud data. For example, if a vehicle is traveling on the road and the vehicle has no effect on the construction of the map, the point cloud data of the vehicle may be deleted.

[0078] Data stitching can be to directly stitch the point cloud data to construct a prototype of the object corresponding to the point cloud data, or to construct local geometric shapes on the point cloud data and finally stitch the geometric shapes.

[0079] Feature line extraction can be performed by outlining feature lines of point cloud data according to point cloud data to obtain feature lines corresponding to the point cloud data.

[0080] Data modeling can be based on the feature lines of the extracted and processed point cloud data, and the feature lines are input into the preset modeling software to perform two-dimensional or three-dimensional modeling of the object.

[0081] In some embodiments of the present disclosure, after obtaining a first map within a preset area, the electronic device may, in response to a data processing operation on the point cloud data in the first map, perform data processing on the point cloud data in the first map to obtain a target map.

[0082] In some other embodiments of the present disclosure, the electronic device may input the target point cloud data into a preset data processing model, so that the preset data processing model performs data processing on the target point cloud data to obtain a target map.

[0083] In the disclosed embodiment, the electronic device can obtain a first map within a preset area by acquiring target point cloud data within the preset area, and then process the point cloud data in the first map to obtain a target map, thereby improving the accuracy of the target map.

[0084] In the disclosed embodiment, S130 may specifically include: inputting the camera image information into a pre-trained visual semantic segmentation model, so that the visual semantic segmentation model performs semantic recognition on the camera image information to obtain pixel information of static objects in the camera image information.

[0085] Specifically, after acquiring the camera image information, the electronic device inputs the camera image information into a pre-trained visual semantic segmentation model, which performs semantic recognition on the camera image information, and then, based on the semantic recognition result, identifies, detects, and semantically segments the corresponding objects in the camera image information to obtain a segmentation result, extracts pixel information of static objects in the camera image information from the segmentation result, and then acquires the pixel information of the static objects in the camera image information.

[0086] In the disclosed embodiment, the electronic device can perform semantic recognition on the camera image information through a pre-trained visual semantic segmentation model to obtain pixel information of static objects in the camera image information, thereby improving the accuracy and efficiency of obtaining pixel information of static objects in the camera image information.

[0087] In an embodiment of the present disclosure, before inputting the camera image information into a pre-trained visual semantic segmentation model so that the visual semantic segmentation model performs semantic recognition on the camera image information and obtains pixel information of static objects in the camera image information, the vehicle positioning method also includes: obtaining the pre-trained visual semantic segmentation model.

[0088] In some embodiments of the present disclosure, the electronic device may directly select a visual semantic segmentation model from a trained and pre-stored machine learning model, and thereby obtain a pre-trained visual semantic segmentation model.

[0089] In some other embodiments of the present disclosure, the electronic device may train a preset machine learning model using the acquired training sample set, thereby obtaining a pre-trained visual semantic segmentation model.

[0090] In an embodiment of the present disclosure, obtaining a pre-trained visual semantic segmentation model may specifically include: obtaining a training sample set and a labeling result corresponding to the training sample set; inputting the training sample set into a preset machine learning model, and using the preset machine learning model to perform semantic segmentation on the training sample set to obtain a predicted segmentation result; based on the labeling result and the predicted segmentation result corresponding to the training sample set, obtaining a loss value of the preset machine learning model; according to the loss value, adjusting the parameters of the preset machine learning model to obtain a pre-trained visual semantic segmentation model.

[0091] Specifically, the electronic device can obtain a training sample set through a camera device, use a labeling tool to label the training sample set, obtain the labeling results corresponding to the training sample set, input the training sample set into a preset machine learning model, use the preset machine learning model to perform semantic segmentation on the training sample set, and obtain the predicted segmentation results corresponding to the objects in the training sample set. According to the labeling results and predicted segmentation results corresponding to the training sample set, a preset loss function is used to calculate the loss value of the preset machine learning model in the process of semantic segmentation of the training sample set, and then according to the loss value, the parameters in the preset machine learning model are adjusted to determine the parameters in the preset machine learning model. The above steps can also be iterated to iteratively adjust the parameters of the preset machine learning model until the loss value of the predicted segmentation result of the preset machine learning model tends to be stable or less than the set loss value threshold, and then the training of the preset machine learning model is stopped to obtain the pre-trained visual semantic segmentation model.

[0092] Among them, the preset machine learning model can be a SEgementation TRansformer model, etc.

[0093] In the disclosed embodiment, the electronic device can train a preset machine learning model by acquiring a training sample set and the annotation results corresponding to the training sample set, and thereby obtain a pre-trained visual semantic segmentation model. Thus, semantic recognition of camera image information is performed through the pre-trained visual semantic segmentation model to improve the accuracy of semantic segmentation.

[0094] In the disclosed embodiment, extracting point cloud features of static objects from radar image information based on pixel information of static objects in S140 may specifically include: acquiring internal and external parameters of the camera and the posture relationship between the camera and the radar, and projecting the pixel information of the static objects into the radar coordinate system; and extracting point cloud features of the static objects from the radar coordinate system.

[0095] Specifically, the electronic device can obtain the internal and external parameters of the camera and the external parameters of the radar from a pre-stored database, obtain the posture relationship between the camera and the radar based on the external parameters of the camera and the radar, obtain the position coordinates of the static object in the camera coordinate system based on the internal and external parameters of the camera, and further project the pixel information of the static object to the radar coordinate system based on the posture relationship between the camera and the radar, and then extract the point cloud features of the static object from the radar coordinates.

[0096] In the disclosed embodiment, the electronic device can extract point cloud features of static objects from the radar coordinate system based on the internal and external parameters of the camera and the posture relationship between the camera and the radar, thereby improving the accuracy of point cloud feature extraction of static objects.

[0097] In some embodiments of the present disclosure, S150 may specifically include: iteratively matching the point cloud features of the static object with the target point cloud features to the nearest point, and obtaining rotation error parameters and translation error parameters of the point cloud features of the static object and the target point cloud features of the static object in the target map; using the rotation error parameters to rotate the position and initial posture of the vehicle in the radar coordinate system, and using the translation error parameters to translate the position and initial posture of the vehicle in the radar coordinate system to obtain corrected vehicle position information, and determining the corrected vehicle position information as the vehicle positioning result.

[0098] In the embodiment of the present disclosure, the position and initial posture of the vehicle in the radar coordinate system are the position and posture of the vehicle acquired by the radar.

[0099] In the disclosed embodiment, the iterative closest point matching is to achieve the minimum cumulative distance between matching points by iteratively minimizing the point cloud features of the static object and the target point cloud features, thereby determining the location of the vehicle.

[0100] Specifically, the electronic device iteratively matches the point cloud features of the static object with the target point cloud features through an iterative closest point (ICP) algorithm, and can calculate the rotation error parameter R and the translation error parameter T between the point cloud features of the static object and the target point cloud features, wherein the rotation error parameter R and the translation error parameter T are the errors between the point cloud features of the static object and the target point cloud features. Further, the position and initial posture of the vehicle in the radar coordinate system are rotated through the rotation error parameter R, and the position and initial posture of the vehicle in the radar coordinate system are translated through the translation error parameter T to obtain the corrected vehicle position information, and the corrected vehicle position information is determined as the vehicle positioning result.

[0101] In the disclosed embodiment, the electronic device can match the point cloud features of a static object with the target point cloud features through an iterative closest point algorithm to obtain a vehicle positioning result, thereby improving the accuracy of vehicle positioning while improving the robustness of vehicle positioning.

[0102] In some other embodiments of the present disclosure, S150 may specifically include: performing normal distribution transformation matching on the point cloud features of the static object and the target point cloud features to obtain a transformation matrix between the point cloud features of the static object and the target point cloud features; and obtaining a positioning result of the vehicle based on the transformation matrix.

[0103] Specifically, the electronic device grids the space where the target point cloud features are located to obtain multiple unit grids, calculates the normal distribution state density parameters of the multiple unit grids according to the distribution of the target point clouds in the multiple unit grids, and then converts the point cloud features of the static object to the map coordinate system where the target point cloud features are located, calculates the distribution of the point clouds of the static object corresponding to the converted point cloud features of the static object in the multiple unit grids, and then calculates the corresponding probability distribution function, further adjusts the posture transformation relationship between the point cloud features of the static object and the target point cloud features, obtains the transformation matrix between the point cloud features of the static object and the target point cloud features, and then calculates the vehicle's position information based on the transformation matrix to obtain the vehicle's positioning result.

[0104] In the disclosed embodiment, the electronic device can perform normal distribution transformation matching on the point cloud features of the static object and the target point cloud features through the normal distribution transformation algorithm to obtain the vehicle positioning result, thereby improving the accuracy of vehicle positioning while improving the robustness of vehicle positioning.

[0105] Figure 3 It is a flow chart of another vehicle positioning method provided by an embodiment of the present disclosure.

[0106] like Figure 3 As shown, the specific steps of the vehicle positioning method are as follows:

[0107] S310: Obtain target point cloud data within a preset area, and obtain a first map within the preset area based on the target point cloud data.

[0108] S320: Process the point cloud data in the first map to obtain a target map.

[0109] S330: Acquire image information, where the image information includes camera image information and radar image information.

[0110] S340, obtaining a pre-trained visual semantic segmentation model.

[0111] S350: input the camera image information into a pre-trained visual semantic segmentation model, so that the visual semantic segmentation model performs semantic recognition on the camera image information to obtain pixel information of static objects in the camera image information.

[0112] S360: extracting point cloud features of the static object from the radar image information based on the pixel information of the static object.

[0113] S370: Match the point cloud features of the static object extracted from the radar image information with the target point cloud features of the static object in the target map to obtain a positioning result of the vehicle.

[0114] In the disclosed embodiment, the electronic device can obtain target point cloud data within a preset area, obtain a first map within the preset area based on the target point cloud data, perform data processing on the point cloud data in the first map to obtain a target map, obtain image information and a pre-trained visual semantic segmentation model, input camera image information in the image information into the pre-trained visual semantic segmentation model, so that the visual semantic segmentation model performs semantic recognition on the camera image information to obtain pixel information of static objects in the camera image information, and then extract point cloud features of static objects from radar image information based on the pixel information of static objects, match the point cloud features of static objects extracted from radar image information with target point cloud features of static objects in the target map to obtain a vehicle positioning result, thereby improving the robustness of vehicle positioning based on camera image information and radar image information, combined with the rich semantic information of camera vision and the point cloud features of radar, while improving the accuracy of vehicle positioning.

[0115] Figure 4 It is a structural schematic diagram of a vehicle positioning device provided in an embodiment of the present disclosure.

[0116] In the embodiments of the present disclosure, the vehicle positioning device can be arranged in an electronic device, and is understood as a part of the functional modules in the above electronic device. Specifically, the electronic device can be a server or a terminal, wherein the terminal specifically includes a mobile phone, a computer or a tablet computer, etc., and can also be any device capable of processing vehicle positioning, which is not limited here.

[0117] like Figure 4 As shown, the vehicle positioning device 400 may include a map acquisition module 410 , an image acquisition module 420 , a semantic recognition module 430 , a feature extraction module 440 and a feature matching module 450 .

[0118] The map acquisition module 410 can be used to acquire a target map, where the target map includes point cloud features of static objects within a preset area.

[0119] The image acquisition module 420 can be used to acquire image information, where the image information includes camera image information and radar image information.

[0120] The semantic recognition module 430 may be used to perform semantic recognition on the camera image information to obtain pixel information of static objects in the camera image information.

[0121] The feature extraction module 440 may be used to extract point cloud features of static objects from radar image information based on pixel information of the static objects.

[0122] The feature matching module 450 can be used to match the point cloud features of the static object extracted from the radar image information with the target point cloud features of the static object in the target map to obtain the vehicle positioning result.

[0123] In the disclosed embodiment, a target map can be acquired, the target map including point cloud features of static objects in a preset area, image information can be acquired, the image information including camera image information and radar image information, semantic recognition is performed on the acquired camera image information, pixel information of the static objects in the camera image information is acquired, and then based on the pixel information of the static objects in the camera image information, point cloud features of the static objects are extracted from the radar image information, the point cloud features of the static objects extracted from the radar image information are matched with the target point cloud features of the static objects in the target map, and the positioning result of the vehicle is acquired. Thus, the robustness of vehicle positioning can be improved based on the camera image information and the radar image information, combined with the rich semantic information of camera vision and the point cloud features of the radar.

[0124] In some embodiments of the present disclosure, the map acquisition module 410 may include a point cloud acquisition unit 4101 and a data processing unit 4102 .

[0125] The point cloud acquisition unit 4101 can be used to acquire target point cloud data within a preset area, and obtain a first map within the preset area based on the target point cloud data.

[0126] The data processing unit 4102 can be used to process the point cloud data in the first map to obtain a target map.

[0127] In some embodiments of the present disclosure, the semantic recognition module 430 can be specifically used to input camera image information into a pre-trained visual semantic segmentation model, so that the visual semantic segmentation model performs semantic recognition on the camera image information to obtain pixel information of static objects in the camera image information.

[0128] In some embodiments of the present disclosure, the vehicle positioning device 400 may further include a model acquisition module 460 .

[0129] The model acquisition module 460 can be used to acquire the pre-trained visual semantic segmentation model before inputting the camera image information into the pre-trained visual semantic segmentation model so that the visual semantic segmentation model performs semantic recognition on the camera image information and obtains the pixel information of the static objects in the camera image information.

[0130] In some embodiments of the present disclosure, the model acquisition module 460 may include a sample set acquisition unit 4601 , a prediction result acquisition unit 4602 , a loss value acquisition unit 4603 and a parameter adjustment unit 4604 .

[0131] The sample set acquisition unit 4601 can be used to acquire a training sample set and a labeling result corresponding to the training sample set.

[0132] The prediction result acquisition unit 4602 can be used to input the training sample set into a preset machine learning model, use the preset machine learning model to perform semantic segmentation on the training sample set, and obtain a prediction segmentation result.

[0133] The loss value acquisition unit 4603 can be used to obtain the loss value of the preset machine learning model based on the annotation results and predicted segmentation results corresponding to the training sample set.

[0134] The parameter adjustment unit 4604 can be used to adjust the parameters of the preset machine learning model according to the loss value to obtain a pre-trained visual semantic segmentation model.

[0135] In some embodiments of the present disclosure, the feature extraction module 440 may include a coordinate system conversion unit 4401 and a feature extraction unit 4402 .

[0136] The coordinate system conversion unit 4401 can be used to obtain the internal and external parameters of the camera and the posture relationship between the camera and the radar, and project the pixel information of the static object into the radar coordinate system.

[0137] The feature extraction unit 4402 can be used to extract point cloud features of static objects from the radar coordinate system.

[0138] In some embodiments of the present disclosure, the feature matching module 450 may include a parameter acquisition unit 4501 and a position correction unit 4502 .

[0139] The parameter acquisition unit 4501 can be used to iteratively perform closest point matching on the point cloud features of the static object and the target point cloud features to obtain rotation error parameters and translation error parameters of the point cloud features of the static object and the target point cloud features of the static object in the target map.

[0140] The position correction unit 4502 can be used to rotate the position and initial posture of the vehicle in the radar coordinate system using a rotation error parameter, and translate the position and initial posture of the vehicle in the radar coordinate system using a translation error parameter to obtain corrected vehicle position information, and determine the corrected vehicle position information as the vehicle positioning result.

[0141] In some embodiments of the present disclosure, the feature matching module 450 may include a transformation matrix acquisition unit 4503 and a result acquisition unit 4504 .

[0142] The transformation matrix acquisition unit 4503 can be used to perform normal distribution transformation matching on the point cloud features of the static object and the target point cloud features to obtain the transformation matrix between the point cloud features of the static object and the target point cloud features.

[0143] The result acquisition unit 4504 can be used to obtain the vehicle positioning result based on the transformation matrix.

[0144] It should be noted that Figure 4 The vehicle positioning device 400 shown can execute each step in the above method embodiment and realize each process and effect in the above method embodiment, which will not be described in detail here.

[0145] Figure 5 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure.

[0146] In the disclosed embodiment, Figure 5 The electronic device shown may be a server or a terminal, wherein the terminal specifically includes a mobile phone, a computer or a tablet computer, etc. It may also be any device capable of processing vehicle positioning, which is not limited here.

[0147] like Figure 5 As shown, the electronic device may include a processor 510 and a memory 520 storing computer program instructions.

[0148] Specifically, the processor 510 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present disclosure.

[0149] The memory 520 may include a large capacity memory for information or instructions. By way of example and not limitation, the memory 520 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 520 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 520 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 520 is a non-volatile solid-state memory. In a particular embodiment, the memory 520 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (Electrically Erasable Programmable ROM, EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0150] The processor 510 reads and executes the computer program instructions stored in the memory 520 to perform the steps of the vehicle positioning method provided in the embodiment of the present disclosure.

[0151] In one example, the electronic device may further include a transceiver 530 and a bus 540. Figure 5 As shown, the processor 510, the memory 520 and the transceiver 530 are connected via a bus 540 and communicate with each other.

[0152] The bus 540 includes hardware, software, or both. For example, but not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a Memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus 540 may include one or more buses.

[0153] The embodiment of the present disclosure further provides a computer-readable storage medium, which may store a computer program. When the computer program is executed by a processor, the processor implements the vehicle positioning method provided by the embodiment of the present disclosure.

[0154] The above-mentioned storage medium may, for example, include a memory 520 of computer program instructions, and the above-mentioned instructions may be executed by the processor 510 of the electronic device to complete the vehicle positioning method provided by the embodiment of the present disclosure. Optionally, the storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (Random Access Memory, RAM), a compact disc read-only memory (Compact Disc ROM, CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0155] The embodiments of the present disclosure also provide a vehicle, which includes electronic equipment and can implement the various processes and effects in the above embodiments of the present disclosure, which will not be elaborated here.

[0156] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A vehicle positioning method, characterized in that: include: Acquire a target map, wherein the target map includes point cloud features of static objects within a preset area; Acquire image information of objects around the vehicle to be located, the image information including camera image information and radar image information; Performing semantic recognition on the camera image information to obtain pixel information of static objects in the camera image information; Extracting point cloud features of the static object from the radar image information based on pixel information of the static object; Matching the point cloud features of the static object extracted from the radar image information with the target point cloud features of the static object in the target map to obtain a positioning result of the vehicle; The step of matching the point cloud features of the static object extracted from the radar image information with the target point cloud features of the static object in the target map to obtain a positioning result of the vehicle includes: Gridding the space where the target point cloud feature is located to obtain a plurality of unit grids, and calculating the normal distribution state density parameters of the plurality of unit grids based on the distribution of the target point cloud corresponding to the target point cloud feature in the plurality of unit grids; The point cloud features of the static object are converted into the map coordinate system corresponding to the target point cloud features, and the distribution of the point clouds of the static object corresponding to the converted point cloud features of the static object in the multiple unit grids is determined; based on the distribution, a probability distribution function corresponding to the point cloud of the static object is calculated, and based on the probability distribution function, a position transformation relationship between the point cloud features of the static object and the target point cloud features is adjusted to obtain a transformation matrix between the point cloud features of the static object and the target point cloud features; Based on the transformation matrix, the position information of the vehicle is calculated to obtain the positioning result of the vehicle.

2. The method according to claim 1, characterized in that: The obtaining of the target map comprises: Acquire target point cloud data within the preset area, and obtain a first map within the preset area based on the target point cloud data; The point cloud data in the first map is processed to obtain the target map.

3. The method according to claim 1, characterized in that The performing semantic recognition on the camera image information to obtain pixel information of static objects in the camera image information includes: The camera image information is input into a pre-trained visual semantic segmentation model, so that the visual semantic segmentation model performs semantic recognition on the camera image information to obtain pixel information of static objects in the camera image information.

4. The method according to claim 3, characterized in that Before inputting the camera image information into a pre-trained visual semantic segmentation model so that the visual semantic segmentation model performs semantic recognition on the camera image information to obtain pixel information of static objects in the camera image information, the method further includes: Get a pre-trained visual semantic segmentation model; The step of obtaining a pre-trained visual semantic segmentation model includes: Obtaining a training sample set and a labeling result corresponding to the training sample set; Inputting the training sample set into a preset machine learning model, performing semantic segmentation on the training sample set using the preset machine learning model, and obtaining a predicted segmentation result; Based on the annotation results and predicted segmentation results corresponding to the training sample set, obtaining the loss value of the preset machine learning model; According to the loss value, the parameters of the preset machine learning model are adjusted to obtain the pre-trained visual semantic segmentation model.

5. The method according to claim 1, characterized in that The step of extracting point cloud features of the static object from the radar image information based on the pixel information of the static object includes: Acquire the internal and external parameters of the camera and the posture relationship between the camera and the radar, and project the pixel information of the static object into the radar coordinate system; Extracting point cloud features of the static object from the radar coordinate system.

6. The method according to claim 5, characterized in that The step of matching the point cloud features of the static object extracted from the radar image information with the target point cloud features of the static object in the target map to obtain a positioning result of the vehicle includes: Iteratively performing closest point matching on the point cloud features of the static object and the target point cloud features to obtain rotation error parameters and translation error parameters of the point cloud features of the static object and the target point cloud features of the static object in the target map; The position and initial posture of the vehicle in the radar coordinate system are rotated using a rotation error parameter, and the position and initial posture of the vehicle in the radar coordinate system are translated using a translation error parameter to obtain corrected vehicle position information, and the corrected vehicle position information is determined as the positioning result of the vehicle.

7. A vehicle positioning device, characterized in that: include: A map acquisition module, used to acquire a target map, wherein the target map includes point cloud features of static objects in a preset area; An image acquisition module, used to acquire image information of objects around the vehicle to be located, the image information including camera image information and radar image information; A semantic recognition module, used to perform semantic recognition on the camera image information to obtain pixel information of static objects in the camera image information; A feature extraction module, used for extracting point cloud features of the static object from the radar image information based on pixel information of the static object; A feature matching module, used to match the point cloud features of the static object extracted from the radar image information with the target point cloud features of the static object in the target map to obtain a vehicle positioning result; The feature matching module is specifically used to grid the space where the target point cloud feature is located to obtain a plurality of unit grids, and calculate the normal distribution state density parameters of the plurality of unit grids based on the distribution of the target point cloud corresponding to the target point cloud feature in the plurality of unit grids; The point cloud features of the static object are converted into the map coordinate system corresponding to the target point cloud features, and the distribution of the point clouds of the static object corresponding to the converted point cloud features of the static object in the multiple unit grids is determined; based on the distribution, a probability distribution function corresponding to the point cloud of the static object is calculated, and based on the probability distribution function, a position transformation relationship between the point cloud features of the static object and the target point cloud features is adjusted to obtain a transformation matrix between the point cloud features of the static object and the target point cloud features; Based on the transformation matrix, the position information of the vehicle is calculated to obtain the positioning result of the vehicle.

8. An electronic device, characterized in that: include: processor; A memory for storing executable instructions; The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the vehicle positioning method described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the vehicle positioning method according to any one of claims 1 to 6.

10. A vehicle, characterized in that: Comprising the electronic device as claimed in claim 8.

Images