Vehicle positioning method and device, vehicle and storage medium
By obtaining and processing the visual and historical positioning information of the vehicle's surrounding environment during the vehicle's driving process, the problem of inaccurate vehicle positioning in urban scenarios is solved, and the accurate positioning of the vehicle and a better driving experience are achieved.
Patent Information
- Application Number
- CN202510094567.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-21
- Publication Date
- 2025-05-16
AI Technical Summary
In urban scenarios, the visual-based vehicle positioning method is easily affected by factors such as vehicle positioning device failure, road obstacles, unclear road markings and light changes, resulting in inaccurate positioning and affecting the driving experience.
By obtaining the target environment image and historical positioning information of the vehicle's surrounding environment during the vehicle's driving process, semantic segmentation is performed to obtain the target semantic data, and a local high-precision map is determined based on the historical positioning information, and matching is performed to determine the target positioning information of the vehicle.
In urban scenarios, even if affected by various factors, the vehicle can be accurately positioned and the driving experience can be improved.
Smart Images
Figure CN120014046A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicle technology, and in particular to a vehicle positioning method, device, vehicle and storage medium. Background Art
[0002] Visual positioning is a key technical step in autonomous driving and assisted driving. In visual positioning, lane line images are usually collected by the vehicle's front-view camera, and then image processing is performed on the lane line images to obtain lane line information, and then the lane line information is integrated with the global high-precision map to locate the vehicle during driving.
[0003] However, currently, when vehicle positioning is performed through vision, the only visual information introduced is the lane line. In urban scenarios, vehicle positioning based on vision will be affected by factors such as failure of the positioning device in the vehicle, multiple obstacles on the road, unclear road markings and changes in lighting, which will make the vehicle positioning based on vision inaccurate, thus affecting the driving experience. Summary of the invention
[0004] One of the purposes of the present application is to provide a vehicle positioning method, which enables the vehicle to be accurately positioned at the current moment in an urban scenario even if it is affected by various factors, thereby improving the driving experience; the second purpose of the present application is to provide a vehicle positioning device; the third purpose of the present application is to provide a vehicle; the fourth purpose of the present application is to provide a storage medium.
[0005] In order to achieve the above objectives, in a first aspect, the present application provides a vehicle positioning method, comprising:
[0006] During vehicle driving, obtaining a target environment image of the surrounding environment of the vehicle at a first moment and obtaining historical positioning information of the vehicle at a second moment, wherein the first moment is a current moment and the second moment is a moment before the first moment;
[0007] Performing semantic segmentation on the target environment image to obtain target semantic data corresponding to the target environment image, and determining a first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment according to the historical positioning information;
[0008] Matching the target semantic data with the first local high-precision map to obtain a target matching result;
[0009] The target positioning information of the vehicle at the first moment is determined according to the target matching result.
[0010] Optionally, the acquiring a target environment image of the surrounding environment of the vehicle at a first moment and acquiring historical positioning information of the vehicle at a second moment include:
[0011] Acquire a target environment image of the surrounding environment of the vehicle at the first moment;
[0012] After acquiring the target environment image, determining whether the first moment is an initial moment, the initial moment being a moment corresponding to the first frame of the environment image of the vehicle surroundings acquired when the vehicle starts to travel;
[0013] When the first moment is not the initial moment, historical positioning information of the vehicle at the second moment is obtained.
[0014] Optionally, after executing the step of determining whether the first moment is an initial moment, the method further includes:
[0015] When the first moment is the initial moment, obtaining a target position of the vehicle at the first moment and obtaining a global high-precision map of the vehicle during driving, wherein the target position is obtained by a positioning device in the vehicle, and the global high-precision map is obtained by a map service server;
[0016] Determining, according to the target position and a first preset distance range, a first area corresponding to a first local high-precision map of the surrounding environment of the vehicle at the first moment to be determined;
[0017] Segmenting the global high-precision map according to the first area to obtain the first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment;
[0018] After obtaining the first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment, the step of matching the target semantic data with the first local high-precision map to obtain a target matching result is performed.
[0019] Optionally, determining, according to the historical positioning information, a first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment includes:
[0020] Determining a target track of the vehicle from the second moment to the first moment;
[0021] Determining predicted positioning information of the vehicle at the first moment according to the historical positioning information and the target track;
[0022] Acquire a second local high-precision map corresponding to the surrounding environment of the vehicle at the second moment;
[0023] A first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment is determined based on the predicted positioning information and the second local high-precision map.
[0024] Optionally, the predicted positioning information includes a predicted position;
[0025] The determining, according to the predicted positioning information and the second local high-precision map, a first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment includes:
[0026] Determine a second area corresponding to the second local high-precision map;
[0027] determining target distances between the predicted position and each edge in the second region to obtain a target distance set;
[0028] When there is a target distance less than a preset distance threshold in the target distance set, a global high-precision map of the vehicle during driving is obtained, the global high-precision map is obtained through a map service server, and the preset distance threshold is used to determine whether it is necessary to obtain the global high-precision map of the vehicle during driving;
[0029] Determining, according to the predicted position and the second preset distance range, a third area corresponding to a first local high-precision map of the surrounding environment of the vehicle at the first moment to be determined;
[0030] Segmenting the global high-precision map according to the third area to obtain the first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment;
[0031] When all the target distances in the target distance set are greater than or equal to the preset distance threshold, the second local high-precision map is determined as the first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment.
[0032] Optionally, the performing semantic segmentation on the target environment image to obtain target semantic data corresponding to the target environment image includes:
[0033] Segmenting an image of interest from the target environment image, wherein the image of interest is located in a middle area of the target environment image, a ratio between a first width corresponding to the image of interest and a second width corresponding to the target environment image is a first preset ratio, and a ratio between a first height corresponding to the image of interest and a second height corresponding to the target environment image is a second preset ratio;
[0034] The image of interest is semantically segmented to obtain target semantic data corresponding to the target environment image.
[0035] Optionally, matching the target semantic data with the first local high-precision map to obtain a target matching result includes:
[0036] Performing image processing on each first semantic element included in the target semantic data to obtain first distance data between each first semantic element;
[0037] Determining second distance data between each second semantic element in the first local high-precision map;
[0038] The obtained first distance data is matched with the second distance data to obtain a target matching result.
[0039] In order to achieve the above objectives, in a second aspect, the present application provides a vehicle positioning device, comprising:
[0040] An acquisition module is used to acquire a target environment image of the surrounding environment of the vehicle at a first moment and acquire historical positioning information of the vehicle at a second moment during the driving process of the vehicle, wherein the first moment is a current moment and the second moment is a moment before the first moment;
[0041] a determination module, configured to perform semantic segmentation on the target environment image to obtain target semantic data corresponding to the target environment image, and to determine a first local high-precision map of the surrounding environment of the vehicle at the first moment according to the historical positioning information;
[0042] A matching module, used for matching the target semantic data with the first local high-precision map to obtain a target matching result;
[0043] A positioning module is used to determine the target positioning information of the vehicle at the first moment according to the target matching result.
[0044] To achieve the above objectives, in a third aspect, the present application also provides a vehicle, including: a processor and a memory, the processor is used to execute a vehicle positioning program stored in the memory to implement the vehicle positioning method as described above.
[0045] To achieve the above objectives, in a fourth aspect, the present application also provides a storage medium, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the vehicle positioning method as described above.
[0046] Beneficial effects of the present application: The present application provides a vehicle positioning method, which, during vehicle driving, obtains a target environment image of the vehicle's surrounding environment at the current moment and obtains historical positioning information of the vehicle at the previous moment, performs semantic segmentation on the target image to obtain target semantic data corresponding to the target environment image, and determines a first local high-precision map corresponding to the vehicle's surrounding environment at the first moment based on the historical positioning information, matches the target semantic data with the first local high-precision map to obtain a target matching result, and thereby determines the target positioning information of the vehicle at the current moment based on the target matching result. In this way, the present application can obtain target semantic data including multiple road semantic elements by performing semantic segmentation on the target environment image of the vehicle's surrounding environment at the current moment, thereby matching the target semantic data with the first local high-precision map corresponding to the vehicle's surrounding environment at the previous moment, and thereby, in urban scenarios, even if affected by various factors, the vehicle can be accurately positioned at the current moment, thereby improving the driving experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 A schematic diagram showing a process flow of a vehicle positioning method provided by an embodiment of the present application is shown;
[0048] Figure 2 A schematic diagram showing a flow chart of another vehicle positioning method provided by an embodiment of the present application;
[0049] Figure 3 A schematic diagram showing a flow chart of another vehicle positioning method provided in an embodiment of the present application;
[0050] Figure 4 A schematic diagram showing the structure of a vehicle positioning device provided in an embodiment of the present application is shown;
[0051] Figure 5 A schematic structural diagram of a vehicle provided in an embodiment of the present application is shown.
[0052] in:
[0053] 10. Acquisition module; 20. Determination module; 30. Matching module; 40. Positioning module;
[0054] 500, vehicle; 501, processor; 502, memory; 5021, operating system; 5022, application; 503, user interface; 504, network interface; 505, bus system. DETAILED DESCRIPTION
[0055] The following will describe the implementation methods of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.
[0056] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.
[0057] To facilitate understanding of the embodiments of the present application, further explanation will be given below with reference to specific embodiments in conjunction with the accompanying drawings. The embodiments do not constitute a limitation on the embodiments of the present application.
[0058] refer to Figure 1 , Figure 1 A schematic flow chart of a vehicle positioning method provided in an embodiment of the present application. A vehicle positioning method provided in an embodiment of the present application comprises the following steps:
[0059] S101: During vehicle driving, a target environment image of the vehicle's surrounding environment at a first moment is obtained, and historical positioning information of the vehicle at a second moment is obtained.
[0060] In this embodiment, the first moment is the current moment, and the second moment is the moment before the first moment. When positioning a vehicle, the vehicle is usually positioned periodically, and the difference between the first moment and the second moment is the period. The period can be set according to actual needs, and the specific value of the period is not limited in this embodiment. For example, when the first moment is t1 and the second moment is t2, t1-t2 is equal to the period.
[0061] The vehicle is provided with a forward-looking camera and a plurality of surround-looking cameras. The image in front of the vehicle is obtained by shooting with the forward-looking camera, and the image around the vehicle is obtained by shooting with the surround-looking cameras. According to the image in front of the vehicle and the plurality of surround-looking images of the vehicle, a target environment image of the surrounding environment of the vehicle at the first moment can be obtained. When the positioning information of the vehicle at a moment is determined each time, the positioning information at each moment is stored in a memory in the vehicle. When the vehicle at the current moment needs to be positioned, the positioning information of the vehicle at the moment before the current moment is queried from the storage in the vehicle, and the positioning information is determined as the historical positioning information of the vehicle at the moment before the current moment.
[0062] S102: Perform semantic segmentation on the target environment image to obtain target semantic data corresponding to the target environment image, and determine a first local high-precision map of the vehicle's surrounding environment at a first moment based on historical positioning information.
[0063] In this embodiment, after obtaining the target environment image, the target environment image can be semantically segmented, so that multiple road semantic elements can be obtained from the target environment image, and then the target semantic data corresponding to the target environment image can be obtained. Among them, the multiple road semantic elements include lane lines, road signs, electric poles, signs, traffic lights, etc.
[0064] When the historical positioning information of the vehicle at the second moment is obtained, the first local high-precision map of the surrounding environment of the vehicle at the first moment can be determined based on the historical positioning information, so as to avoid the failure of the positioning device in the vehicle and still achieve accurate positioning of the vehicle based on vision. Specifically, the coverage of the first local high-precision map is consistent with the coverage of the target environment image, so as to achieve more accurate positioning of the vehicle.
[0065] S103: Match the target semantic data with the first local high-precision map to obtain a target matching result.
[0066] In this embodiment, after obtaining the target semantic data and the first local high-precision map, since the coverage range of the target environment image is consistent with the coverage range of the first local high-precision map, the target semantic data corresponding to the target environment image can be matched one by one with each element in the first local high-precision map, thereby obtaining a target matching result between the target semantic data and the first local high-precision map.
[0067] S104: Determine the target positioning information of the vehicle at the first moment according to the target matching result.
[0068] In this embodiment, after obtaining the target matching result, the transformation matrix between the target environment image and the first local high-precision map can be determined according to the target matching result, so as to obtain the target positioning information of the vehicle at the first moment according to the determined transformation matrix.
[0069] It should be noted that based on the target matching result, the transformation matrix between the target environment image and the first local high-precision map is determined, and based on the determined transformation matrix, the target positioning information of the vehicle at the first moment can be determined according to the existing technology, which will not be elaborated in this embodiment.
[0070] The present embodiment provides a vehicle positioning method. During vehicle driving, a target environment image of the vehicle's surrounding environment at the current moment and historical positioning information of the vehicle at the previous moment are obtained, the target image is semantically segmented to obtain target semantic data corresponding to the target environment image, and a first local high-precision map of the vehicle's surrounding environment at the first moment is determined based on the historical positioning information, and the target semantic data is matched with the first local high-precision map to obtain a target matching result, thereby determining the target positioning information of the vehicle at the current moment based on the target matching result. In this way, the present application can obtain target semantic data including multiple road semantic elements by semantically segmenting the target environment image of the vehicle's surrounding environment at the current moment, thereby matching the target semantic data with the first local high-precision map of the vehicle at the previous moment, and further, in urban scenarios, even if affected by various factors, the vehicle can be accurately positioned at the current moment, thereby improving the driving experience.
[0071] refer to Figure 2 , Figure 2 A flow chart of another vehicle positioning method provided in an embodiment of the present application. A vehicle positioning method provided in an embodiment of the present application comprises the following steps:
[0072] S201: During the driving process of the vehicle, a target environment image of the surrounding environment of the vehicle at a first moment is acquired.
[0073] In this embodiment, step S201 is consistent with the step of obtaining the target environment image of the surrounding environment of the vehicle at the first moment in the above step S101. For details, please refer to the above step S101, which will not be repeated in this embodiment.
[0074] S202: After acquiring the target environment image, determine whether the first moment is an initial moment.
[0075] S203: When the first moment is not the initial moment, obtaining historical positioning information of the vehicle at the second moment.
[0076] S204: Perform semantic segmentation on the target environment image to obtain target semantic data corresponding to the target environment image, and determine a first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment based on historical positioning information.
[0077] For the above steps S202 to S204, the initial moment is the moment corresponding to the first frame of the environmental image of the vehicle's surrounding environment obtained when the vehicle starts to travel. For example, when the vehicle starts to travel, the front-view camera and the surround-view camera in the vehicle start to work, and when the first environmental image of the vehicle's surrounding environment is obtained after the front-view camera and the surround-view camera work, the moment corresponding to the environmental image is determined as the initial moment.
[0078] Among them, when the target environment image is obtained, it is determined whether the first moment corresponding to the obtained target environment image is the initial moment. When the first moment is not the initial moment, in order to avoid the problem of large errors in determining the first local high-precision map of the vehicle's surrounding environment at the first moment by the positioning device in the vehicle, the historical positioning information of the vehicle at the second moment is obtained from the memory in the vehicle, and then the first local high-precision map corresponding to the vehicle's surrounding environment at the first moment is determined using the historical positioning information, and then the determined first local high-precision map is used to achieve more accurate positioning of the vehicle at the first moment.
[0079] In this embodiment, in step S204, determining a first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment according to the historical positioning information includes:
[0080] Determining a target track of the vehicle from the second moment to the first moment;
[0081] Determine the predicted positioning information of the vehicle at the first moment based on the historical positioning information and the target track;
[0082] Obtain a second local high-precision map corresponding to the surrounding environment of the vehicle at the second moment;
[0083] According to the predicted positioning information and the second local high-precision map, a first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment is determined.
[0084] Among them, the vehicle driving state can be measured by the odometer, gyroscope and other internal vehicle sensors in the vehicle, so as to obtain the target track of the vehicle from the second moment to the first moment, and the vehicle's posture increment is calculated according to the target track, and then the predicted position of the vehicle at the first moment is determined according to the posture increment and the historical positioning information of the vehicle at the second moment. It should be noted that the method for determining the target track and the predicted position of the vehicle at the first moment can refer to the existing technology, which will not be repeated in this embodiment.
[0085] Specifically, each time a local high-precision map corresponding to the surrounding environment of the vehicle at a moment is determined, the correspondence between the moment and the local high-precision map can be stored in a memory in the vehicle. When it is necessary to determine the target positioning information of the vehicle at the first moment and to obtain the second local high-precision map corresponding to the surrounding environment at the second moment, the second local high-precision map corresponding to the second moment is determined from the correspondence between the moment and the local high-precision map stored in the memory in the vehicle, so as to obtain the second local high-precision map corresponding to the surrounding environment of the vehicle at the second moment, thereby determining the first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment based on the predicted positioning information of the vehicle at the first moment and the second local high-precision map, and then realizing accurate determination of the target positioning information of the vehicle at the first moment based on the determined first local high-precision map, thereby avoiding inaccurate target positioning information of the vehicle at the first moment due to large errors in the positioning device in the vehicle.
[0086] In this embodiment, the predicted positioning information includes the predicted position. The first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment is determined based on the predicted positioning information and the second local high-precision map, including:
[0087] Determine a second area corresponding to the second local high-precision map;
[0088] determining target distances between the predicted position and each edge in the second region to obtain a target distance set;
[0089] When there is a target distance less than a preset distance threshold in the target distance set, a global high-precision map of the vehicle during driving is obtained;
[0090] Determine, according to the predicted position and the second preset distance range, a third area corresponding to a first local high-precision map of the surrounding environment of the vehicle to be determined at the first moment;
[0091] Segmenting the global high-precision map according to the third area to obtain a first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment;
[0092] When all target distances in the target distance set are greater than or equal to a preset distance threshold, the second local high-precision map is determined as the first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment.
[0093] The global high-precision map is obtained through the map service server, and the preset distance threshold is used to determine whether it is necessary to obtain the global high-precision map of the vehicle during driving. The preset distance threshold can be set according to actual needs, and the specific value of the preset distance threshold is not limited in this embodiment.
[0094] Specifically, the coverage of the second local high-precision map is consistent with the coverage of the target environment image. The coverage of the target environment image is usually determined by the forward-looking camera and the surround-looking camera in the vehicle. In this embodiment, the coverage of the target environment image is usually 120 meters in front of the vehicle and 60 meters behind the vehicle. Of course, the coverage of the target environment image can also be set according to actual needs. In this embodiment, the coverage of the target environment image is not specifically limited. The second area is usually a rectangular area. After determining the second local high-precision map, the target distance between the predicted position and each edge in the second area corresponding to the second local high-precision map is determined to obtain a target distance set. The target distance can be understood as the vertical distance between the predicted position and the edge in the second area.
[0095] More specifically, after obtaining the target distance set, each target distance in the target distance set can be compared with the preset distance threshold. When there is a target distance less than the preset distance threshold in the target distance set, it indicates that the vehicle is traveling at a high speed. At this time, the second local high-precision map corresponding to the surrounding environment of the vehicle at the second moment cannot be used as the first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment. In order to ensure the accuracy of the determination of the target positioning information of the subsequent vehicle at the first moment, the third area corresponding to the first local high-precision map of the surrounding environment of the vehicle to be determined at the first moment is determined according to the predicted position and the second preset distance range, so as to segment the global high-precision map using the third area to obtain the first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment. Among them, the coverage range of the third area is consistent with the coverage range of the target environment image. After obtaining the predicted position and the second preset distance range, the predicted position can be used as the origin and the second preset distance range can be used as the radiation range of the predicted position to obtain the third area. The second preset distance range can be set according to actual needs. In this embodiment, the specific value of the second preset distance range is not limited. It is only necessary to ensure that the coverage range of the third area determined by the predicted position and the second preset distance range is consistent with the coverage range of the target environment image.
[0096] In the above, when all target distances in the target distance set are greater than the preset distance range, it indicates that the vehicle is traveling slowly. At this time, in order to save computing resources, the second local high-precision map corresponding to the vehicle's surrounding environment at the second moment can be directly determined as the first local high-precision map corresponding to the vehicle's surrounding environment at the first moment, thereby using the determined first local high-precision map to accurately determine the target positioning information of subsequent vehicles at the first moment.
[0097] In this embodiment, the semantic segmentation of the target environment image in the above step S204 to obtain target semantic data corresponding to the target environment image includes:
[0098] Segmenting an image of interest from a target environment image;
[0099] The image of interest is semantically segmented to obtain the target semantic data corresponding to the target environment image.
[0100] Among them, the image of interest is located in the middle area of the target environment image, the ratio between the first width corresponding to the image of interest and the second width corresponding to the target environment image is a first preset ratio, and the ratio between the first height corresponding to the image of interest and the second height corresponding to the target environment image is a second preset ratio.
[0101] Specifically, the first preset ratio and the second preset ratio can be set according to actual needs, and the specific values of the first preset ratio and the second preset ratio are not limited in this embodiment. Since the target environment image is acquired by the forward-looking camera and the surround-looking camera in the vehicle, and the image acquired by the camera is usually distorted, the distortion of the image in the middle area of the target environment image is small, and the distortion is greater closer to the edge of the target environment image. Therefore, in order to improve the target positioning information of subsequent vehicles at the first moment, the image of interest located in the middle area of the target environment image is determined from the target environment image, so as to perform semantic segmentation on the image of interest to obtain the target semantic data corresponding to the target environment image. It should be noted that the semantic segmentation method can refer to the prior art, and this embodiment will not be repeated here.
[0102] More specifically, the image of interest being located in the middle area of the target environment image can be understood as follows: the distance between one width margin of the image of interest and one width edge of its adjacent target environment image is consistent with the distance between another width margin of the image of interest and another width edge of its adjacent target environment image, and the distance between one height edge of the image of interest and one height edge of its adjacent target environment image is consistent with the distance between another height margin of the image of interest and another height edge of its adjacent target environment image.
[0103] S205: When the first moment is the initial moment, obtain the target position of the vehicle at the first moment and obtain a global high-precision map of the vehicle during the driving process.
[0104] S206: Determine, based on the target position and the first preset distance range, a first area corresponding to a first local high-precision map of the surrounding environment of the vehicle to be determined at the first moment.
[0105] S207: Segment the global high-precision map according to the first area to obtain a first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment.
[0106] For the above-mentioned steps S205 to S207, the target position is obtained through the behavior device in the vehicle, and the global high-precision map is obtained through the map business server. After the target environment image is obtained, it is determined whether the first moment corresponding to the obtained target environment image is the initial moment. When the first moment is the initial moment, it indicates that the vehicle's memory does not store the positioning information of the vehicle at the moment before the first moment. At this time, the first local high-precision map corresponding to the vehicle's surrounding environment at the first moment cannot be determined through the positioning information of the vehicle at the second moment. It is necessary to obtain the target position of the vehicle at the first moment according to the positioning device in the vehicle, and obtain the global high-precision map of the vehicle during driving through the map business server, so as to determine the first area corresponding to the first local high-precision map of the surrounding environment of the vehicle at the first moment according to the target position and the first preset distance range, and then use the first area to wind the global high-precision map to obtain the first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment. Among them, the coverage range of the first area is consistent with the coverage range of the target environment image. After obtaining the target position and the first preset distance range, the target position can be used as the origin, and the radiation range of the first preset distance range and the target position can be used to obtain the first area. The first preset distance range can be set according to actual needs. In this embodiment, the specific value of the first preset distance range is not limited. It is only necessary to ensure that the coverage range of the first area determined by the target position and the first preset distance range is consistent with the coverage range of the target environment image.
[0107] S208: Match the target semantic data with the first local high-precision map to obtain a target matching result.
[0108] In this embodiment, in step S208, the target semantic data is matched with the first local high-precision map to obtain a target matching result, including:
[0109] Performing image processing on each first semantic element included in the target semantic data to obtain first distance data between each first semantic element;
[0110] Determine second distance data between each second semantic element in the first local high-precision map;
[0111] The obtained first distance data is matched with the second distance data to obtain a target matching result.
[0112] Among them, after obtaining the target semantic data, the first semantic elements of lane lines, road signs, telephone poles, signboards and traffic lights included in the target semantic data are subjected to image processing such as binarization, dilation and corrosion, and the first distance data between each first semantic element is obtained. Similarly, in order to match the target environment image with the first local high-precision map, the first local high-precision map is projected to the image plane to obtain the second distance data between each second semantic element in the first local high-precision map. After obtaining the first distance data and the second distance data, the first distance data and the second distance data are matched, and the correspondence between the pixel points projected onto the image plane of the first local high-precision map and the pixel points collected by the camera (i.e., the target matching result) can be obtained, so that according to the target matching result, the positioning information of the vehicle in the global high-precision map can be determined.
[0113] S209: Determine the target positioning information of the vehicle at the first moment according to the target matching result.
[0114] In this embodiment, according to the target matching result, the target positioning information of the vehicle at the first moment is determined by referring to the prior art, which is not described in detail in this embodiment.
[0115] The present embodiment provides a vehicle positioning method. During vehicle driving, a target environment image of the vehicle's surrounding environment at the current moment and historical positioning information of the vehicle at the previous moment are obtained, the target image is semantically segmented to obtain target semantic data corresponding to the target environment image, and a first local high-precision map of the vehicle's surrounding environment at the first moment is determined based on the historical positioning information, and the target semantic data is matched with the first local high-precision map to obtain a target matching result, thereby determining the target positioning information of the vehicle at the current moment based on the target matching result. In this way, the present application can obtain target semantic data including multiple road semantic elements by semantically segmenting the target environment image of the vehicle's surrounding environment at the current moment, thereby matching the target semantic data with the first local high-precision map of the vehicle at the previous moment, and further, in urban scenarios, even if affected by various factors, the vehicle can be accurately positioned at the current moment, thereby improving the driving experience.
[0116] As an example, refer to Figure 3 , let's introduce the whole vehicle positioning process in detail, as follows:
[0117] During the driving process of the vehicle, a target environment image of the surrounding environment of the vehicle at a first moment is obtained;
[0118] Determine whether the first moment is an initial moment;
[0119] When the first moment is the initial moment, the target position of the vehicle at the first moment is obtained and a global high-precision map of the vehicle during driving is obtained;
[0120] Determine, according to the target position and the first preset distance range, a first area corresponding to a first local high-precision map of the surrounding environment of the vehicle to be determined at the first moment;
[0121] Segmenting the global high-precision map according to the first region to obtain a first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment;
[0122] When the first moment is not the initial moment, obtaining historical positioning information of the vehicle at a second moment, where the second moment is a moment before the first moment;
[0123] Determining a target track of the vehicle from the second moment to the first moment;
[0124] Determine the predicted position of the vehicle at the first moment based on historical positioning information and target track;
[0125] Obtain a second local high-precision map corresponding to the surrounding environment of the vehicle at the second moment;
[0126] Determine a first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment according to the predicted position and the second local high-precision map;
[0127] Segmenting an image of interest from a target environment image;
[0128] Perform semantic segmentation on the image of interest to obtain target semantic data corresponding to the target environment image;
[0129] Matching the target semantic data with the first local high-precision map to obtain a target matching result;
[0130] According to the target matching result, the target positioning information of the vehicle at the first moment is determined.
[0131] refer to Figure 4 , Figure 4A schematic diagram of the structure of a vehicle positioning device provided in an embodiment of the present application. A vehicle positioning device provided in an embodiment of the present application includes: an acquisition module 10, a determination module 20, a matching module 30 and a positioning module 40. Among them, the acquisition module 10 is used to acquire a target environment image of the surrounding environment of the vehicle at a first moment and acquire historical positioning information of the vehicle at a second moment during the driving process of the vehicle, the first moment being the current moment and the second moment being the moment before the first moment; the determination module 20 is used to perform semantic segmentation on the target environment image to obtain target semantic data corresponding to the target environment image, and determine the first local high-precision map of the surrounding environment of the vehicle at the first moment according to the historical positioning information; the matching module 30 is used to match the target semantic data with the first local high-precision map to obtain a target matching result; the positioning module 40 is used to determine the target positioning information of the vehicle at the first moment according to the target matching result.
[0132] In this embodiment, the acquisition module 10 is further used for:
[0133] Acquire a target environment image of the surrounding environment of the vehicle at the first moment;
[0134] After acquiring the target environment image, determining whether the first moment is an initial moment, the initial moment being a moment corresponding to the first frame of the environment image of the vehicle surroundings acquired when the vehicle starts to travel;
[0135] When the first moment is not the initial moment, historical positioning information of the vehicle at the second moment is obtained.
[0136] In this embodiment, the acquisition module 10 is further used for:
[0137] When the first moment is the initial moment, obtaining a target position of the vehicle at the first moment and obtaining a global high-precision map of the vehicle during driving, wherein the target position is obtained by a positioning device in the vehicle, and the global high-precision map is obtained by a map service server;
[0138] Determining, according to the target position and a first preset distance range, a first area corresponding to a first local high-precision map of the surrounding environment of the vehicle at the first moment to be determined;
[0139] The global high-precision map is segmented according to the first area to obtain the first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment.
[0140] In this embodiment, the determination module 20 is further configured to:
[0141] Determining a target track of the vehicle from the second moment to the first moment;
[0142] Determining predicted positioning information of the vehicle at the first moment according to the historical positioning information and the target track;
[0143] Acquire a second local high-precision map corresponding to the surrounding environment of the vehicle at the second moment;
[0144] A first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment is determined based on the predicted positioning information and the second local high-precision map.
[0145] In this embodiment, the predicted positioning information includes a predicted position.
[0146] In this embodiment, the determination module 20 is further configured to:
[0147] Determine a second area corresponding to the second local high-precision map;
[0148] determining target distances between the predicted position and each edge in the second region to obtain a target distance set;
[0149] When there is a target distance less than a preset distance threshold in the target distance set, a global high-precision map of the vehicle during driving is obtained, the global high-precision map is obtained through a map service server, and the preset distance threshold is used to determine whether it is necessary to obtain the global high-precision map of the vehicle during driving;
[0150] Determining, according to the predicted position and the second preset distance range, a third area corresponding to a first local high-precision map of the surrounding environment of the vehicle at the first moment to be determined;
[0151] Segmenting the global high-precision map according to the third area to obtain the first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment;
[0152] When all the target distances in the target distance set are greater than or equal to the preset distance threshold, the second local high-precision map is determined as the first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment.
[0153] In this embodiment, the determination module 20 is further configured to:
[0154] Segmenting an image of interest from the target environment image, wherein the image of interest is located in a middle area of the target environment image, a ratio between a first width corresponding to the image of interest and a second width corresponding to the target environment image is a first preset ratio, and a ratio between a first height corresponding to the image of interest and a second height corresponding to the target environment image is a second preset ratio;
[0155] The image of interest is semantically segmented to obtain target semantic data corresponding to the target environment image.
[0156] In this embodiment, the target semantic data includes a plurality of first pixel values corresponding to the first road elements.
[0157] In this embodiment, the matching module 30 is further used for:
[0158] Performing image processing on each first semantic element included in the target semantic data to obtain first distance data between each first semantic element;
[0159] Determining second distance data between each second semantic element in the first local high-precision map;
[0160] The obtained first distance data is matched with the second distance data to obtain a target matching result.
[0161] A vehicle positioning device provided in this embodiment obtains a target environment image of the vehicle's surrounding environment at a current moment and obtains historical positioning information of the vehicle at a moment before the current moment during vehicle driving, performs semantic segmentation on the target image to obtain target semantic data corresponding to the target environment image, and determines a first local high-precision map of the vehicle's surrounding environment at a first moment based on the historical positioning information, matches the target semantic data with the first local high-precision map to obtain a target matching result, and thereby determines the target positioning information of the vehicle at the current moment based on the target matching result. In this way, the present application can obtain target semantic data including multiple road semantic elements by performing semantic segmentation on the target environment image of the vehicle's surrounding environment at the current moment, thereby matching the target semantic data with the first local high-precision map of the vehicle at a moment before the current moment, and thereby, in urban scenarios, even if affected by various factors, the vehicle can be accurately positioned at the current moment, thereby improving the driving experience.
[0162] refer to Figure 5 As shown, Figure 5A schematic diagram of the structure of a vehicle provided in an embodiment of the present application. The vehicle 500 in this embodiment may include: at least one processor 501, a vehicle memory 502, at least one network interface 504 and other user interfaces 503. The various components in the vehicle 500 are coupled together via a bus system 505. It can be understood that the bus system 505 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 505 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as bus system 505.
[0163] The user interface 503 may include a display, a keyboard, or a pointing device (eg, a mouse, a trackball, a touch pad, or a touch screen).
[0164] It can be understood that the memory 502 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 502 described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0165] In some implementations, the memory 502 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system 5021 and an application program 5022 .
[0166] Among them, the operating system 5021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 5022 includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present application can be included in the application 5022.
[0167] In the embodiment of the present application, by calling the program or instruction stored in the memory 502, specifically, the program or instruction stored in the application 5022, the processor 501 is used to execute the method provided by each method embodiment.
[0168] The method disclosed in the above embodiment of the present application can be applied to the processor 501, or implemented by the processor 501. The processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 501. The above processor 501 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software units in the decoding processor can be executed. The software unit can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502 and completes the above method in combination with its hardware.
[0169] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or at least one application specific integrated circuit (ASIC), digital signal processor (DSP), digital signal processing vehicle (DSPDevice, DSPD), programmable logic vehicle (PLD), field programmable gate array (FPGA), general purpose processor, controller, microcontroller, microprocessor, other electronic unit for performing the functions described in the present application, or a combination thereof.
[0170] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.
[0171] The embodiment of the present application also provides a storage medium (computer-readable storage medium). The storage medium here stores one or at least one program. The storage medium may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory may also include a combination of the above-mentioned types of memory.
[0172] When one or at least one program in the storage medium can be executed by one or at least one processor. When the storage medium is applied to a vehicle, the above-mentioned method executed in the vehicle can be implemented. The processor is used to execute the vehicle program stored in the memory to implement the above-mentioned method executed in the vehicle.
[0173] The professionals should also be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0174] It should be noted that the phrases "one implementation", "an embodiment", "an exemplary embodiment", "some embodiments", etc. mentioned in the specification indicate that the described embodiments may include certain features, structures or characteristics, but not every embodiment may include the certain features, structures or characteristics. In addition, such phrases do not necessarily refer to the same embodiment. In addition, when describing certain features, structures or characteristics in conjunction with an embodiment, it is within the knowledge of those skilled in the art to implement such features, structures or characteristics in conjunction with other embodiments, whether explicitly or not explicitly described.
[0175] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method or article including a series of elements may include not only those elements, but also other elements not explicitly listed, or may also include elements inherent to such process, method, article or vehicle. In the absence of more limitations, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or vehicle including the elements.
[0176] The above embodiments are only preferred embodiments for fully illustrating the present application, and the protection scope of the present application is not limited thereto. Any equivalent substitution or change made by a person skilled in the art based on the present application is within the protection scope of the present application.
Claims
1. A vehicle positioning method, characterized in that: include: During vehicle driving, obtaining a target environment image of the surrounding environment of the vehicle at a first moment and obtaining historical positioning information of the vehicle at a second moment, wherein the first moment is a current moment and the second moment is a moment before the first moment; Performing semantic segmentation on the target environment image to obtain target semantic data corresponding to the target environment image, and determining a first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment according to the historical positioning information; Matching the target semantic data with the first local high-precision map to obtain a target matching result; The target positioning information of the vehicle at the first moment is determined according to the target matching result.
2. The vehicle positioning method according to claim 1, characterized in that: The step of acquiring a target environment image of the surrounding environment of the vehicle at a first moment and acquiring historical positioning information of the vehicle at a second moment includes: Acquire a target environment image of the surrounding environment of the vehicle at the first moment; After acquiring the target environment image, determining whether the first moment is an initial moment, the initial moment being a moment corresponding to the first frame of the environment image of the vehicle surroundings acquired when the vehicle starts to travel; When the first moment is not the initial moment, historical positioning information of the vehicle at the second moment is obtained.
3. The vehicle positioning method according to claim 2, characterized in that: After executing the step of determining whether the first moment is an initial moment, the method further includes: When the first moment is the initial moment, obtaining a target position of the vehicle at the first moment and obtaining a global high-precision map of the vehicle during driving, wherein the target position is obtained by a positioning device in the vehicle, and the global high-precision map is obtained by a map service server; Determining, according to the target position and a first preset distance range, a first area corresponding to a first local high-precision map of the surrounding environment of the vehicle at the first moment to be determined; Segmenting the global high-precision map according to the first area to obtain the first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment; After obtaining the first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment, the step of matching the target semantic data with the first local high-precision map to obtain a target matching result is performed.
4. The vehicle positioning method according to claim 1, characterized in that: The determining, according to the historical positioning information, a first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment includes: Determining a target track of the vehicle from the second moment to the first moment; Determining predicted positioning information of the vehicle at the first moment according to the historical positioning information and the target track; Acquire a second local high-precision map corresponding to the surrounding environment of the vehicle at the second moment; A first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment is determined based on the predicted positioning information and the second local high-precision map.
5. The vehicle positioning method according to claim 4, characterized in that: The predicted positioning information includes a predicted position; The determining, according to the predicted positioning information and the second local high-precision map, a first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment includes: Determine a second area corresponding to the second local high-precision map; determining target distances between the predicted position and each edge in the second region to obtain a target distance set; When there is a target distance less than a preset distance threshold in the target distance set, a global high-precision map of the vehicle during driving is obtained, the global high-precision map is obtained through a map service server, and the preset distance threshold is used to determine whether it is necessary to obtain the global high-precision map of the vehicle during driving; Determining, according to the predicted position and the second preset distance range, a third area corresponding to a first local high-precision map of the surrounding environment of the vehicle at the first moment to be determined; Segmenting the global high-precision map according to the third area to obtain the first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment; When all the target distances in the target distance set are greater than or equal to the preset distance threshold, the second local high-precision map is determined as the first local high-precision map corresponding to the surrounding environment of the vehicle at the first moment.
6. The vehicle positioning method according to claim 1, characterized in that: The performing semantic segmentation on the target environment image to obtain target semantic data corresponding to the target environment image includes: Segmenting an image of interest from the target environment image, wherein the image of interest is located in a middle area of the target environment image, a ratio between a first width corresponding to the image of interest and a second width corresponding to the target environment image is a first preset ratio, and a ratio between a first height corresponding to the image of interest and a second height corresponding to the target environment image is a second preset ratio; The image of interest is semantically segmented to obtain target semantic data corresponding to the target environment image.
7. The vehicle positioning method according to claim 1, characterized in that: The matching the target semantic data with the first local high-precision map to obtain a target matching result includes: Performing image processing on each first semantic element included in the target semantic data to obtain first distance data between each first semantic element; Determining second distance data between each second semantic element in the first local high-precision map; The obtained first distance data is matched with the second distance data to obtain a target matching result.
8. A vehicle positioning device, characterized in that: include: An acquisition module is used to acquire a target environment image of the surrounding environment of the vehicle at a first moment and acquire historical positioning information of the vehicle at a second moment during the driving process of the vehicle, wherein the first moment is a current moment and the second moment is a moment before the first moment; a determination module, configured to perform semantic segmentation on the target environment image to obtain target semantic data corresponding to the target environment image, and to determine a first local high-precision map of the surrounding environment of the vehicle at the first moment according to the historical positioning information; A matching module, used for matching the target semantic data with the first local high-precision map to obtain a target matching result; A positioning module is used to determine the target positioning information of the vehicle at the first moment according to the target matching result.
9. A vehicle, characterized in that: include: A processor and a memory, wherein the processor is used to execute a vehicle positioning program stored in the memory to implement the vehicle positioning method according to any one of claims 1 to 7.
10. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the vehicle positioning method of any one of claims 1 to 7.