Distance prediction method, model training method, planning and control system, and related devices.

Abstract

This application discloses a distance prediction method, a model training method, a planning and control system, and related devices thereof, relating to the field of intelligent driving technology, and can solve the problem that the economic cost and time cost required for planning and control based on a high-precision map are high. The distance prediction method includes the steps of obtaining the first global pose, the first extended navigation route, and the first landmark information of the first vehicle, and processing the first global pose, the first extended navigation route, and the first landmark information based on a distance prediction model to obtain the farthest reachable distance on each lane in the first landmark information. The first global pose is the current global pose of the first vehicle, the first extended navigation route includes an extended route of the first in-vehicle navigation route determined based on the first global pose, and the first landmark information includes landmark information included in the first road environment image collected by the first vehicle at the first global pose. This application can be applied to a scenario where planning and control are realized based on image perception.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and specifically to a distance prediction method, a model training method, a planning control system and related devices thereof.

Background Art

[0002] An intelligent driving system is a complex system that combines hardware and software, and the system includes multiple modules such as sensor integration, environmental perception, prediction, and planning control. A high-precision map, as an important support for an intelligent driving system, can provide information with higher accuracy and richer details compared to a conventional navigation map, thereby realizing the commercial application of intelligent driving technology. For example, in the process of planning control, an intelligent driving system performs route planning based on a high-precision map and determines whether the vehicle continues to move forward based on the current lane or changes lanes. However, the collection and creation of high-precision maps are very complex. In order to ensure the "freshness" of high-precision maps and meet the needs of safe use of intelligent driving, high-precision maps need to be updated frequently, and the workload of data collection is huge, requiring a large amount of investment in time, personnel, and materials. Therefore, the economic cost and time cost required for planning control based on high-precision maps are high.

Summary of the Invention

Problems to be Solved by the Invention

[0003] This application provides a distance prediction method, a model training method, a planning control system and related devices thereof, which can solve the problem that the economic cost and time cost required for planning control based on high-precision maps are high.

Means for Solving the Problems

[0004] The specific technical solutions are as follows.

[0005] In a first aspect, an embodiment of the present application provides a distance prediction method, the method being: The steps include obtaining the first global pose, first extended navigation route, and first landmark information of the first vehicle, The process includes the steps of processing the first global pose, the first extended navigation route, and the first landmark information based on a distance prediction model to obtain the furthest achievable distance on each lane in the first landmark information, The first global pose is the current global pose of the first vehicle, the first extended navigation route includes an extended route of the first in-vehicle navigation route determined based on the first global pose, and the first landmark information includes landmark information contained in the first road environment image collected by the first vehicle in the first global pose. The training method for the distance prediction model is as follows: Steps to obtain a distance training sample set, The steps include: training using the distance training sample set to obtain the distance prediction model, Each training sample in the distance training sample set includes a second extended navigation route, second landmark information, a second global pose, and a truth value for the furthest achievable distance, wherein the second extended navigation route includes an extended route of the second in-vehicle navigation route, the second landmark information includes landmark information contained in a second road environment image collected by a second vehicle on the second extended navigation route, the second global pose includes the global pose at the time the second vehicle collected the second road environment image, and the truth value for the furthest achievable distance includes the truth value for the furthest achievable distance for each lane in the second landmark information.

[0006] According to the technical means described above, an embodiment of the present application first trains and acquires a distance prediction model based on a distance training sample set including a second extended navigation route, a second landmark information, a second global pose, and the truth value of the furthest reachable distance for each lane in the second landmark information. Then, the first global pose, the first extended navigation route, and the first landmark information are input into the distance prediction model to predict the furthest reachable distance on each lane in the first landmark information. Since the second global pose, second extended navigation route, and second landmark information required during model training, as well as the first global pose, first extended navigation route, and first landmark information required during model application, do not depend on high-precision maps, the embodiment of the present invention can automatically train a distance prediction model capable of predicting the furthest reachable distance in each lane ahead of the vehicle, while being free from the frequent updates of high-precision maps, and can use this distance prediction model to automatically predict the furthest reachable distance in each lane ahead of the vehicle. This makes it easy to quickly plan routes according to the furthest reachable distance in each lane ahead of the vehicle, and further enables intelligent driving planning control while saving economic and time costs.

[0007] In a first possible implementation of the first embodiment, where the target extended navigation route includes the first extended navigation route and / or the second extended navigation route, the method for acquiring the target extended navigation route is: Steps include obtaining a target in-vehicle navigation route and a target global pause when the target vehicle is traveling on the target in-vehicle navigation route, The steps include: extracting POI (Point of Interest) information corresponding to the aforementioned target global pose from the target data; The steps include adding the POI information to the target in-vehicle navigation route and obtaining the target extended navigation route, If the target extended navigation route is the first extended navigation route, the target in-vehicle navigation route is the first in-vehicle navigation route, the target vehicle is the first vehicle, and the target global pose is the first global pose; if the target extended navigation route is the second extended navigation route, the target in-vehicle navigation route is the second in-vehicle navigation route, and the target global pose is the second global pose. The target data includes navigation events and / or in-vehicle navigation maps, the POI information includes road attribute information related to the prediction of the furthest reachable distance, and the POI information corresponding to the target global pose includes the POI information within a predetermined distance range ahead of the target global pose on the target in-vehicle navigation route.

[0008] According to the above technical means, an embodiment of the present application can extract POI information corresponding to a second global pose from navigation events and / or an in-vehicle navigation map, add the extracted POI information to a second in-vehicle navigation route, and obtain a second extended navigation route that contributes to training a distance prediction model of higher quality than the second in-vehicle navigation route. Furthermore, an embodiment of the present application can extract POI information corresponding to a first global pose from navigation events and / or an in-vehicle navigation map, add the extracted POI information to a first in-vehicle navigation route, and contribute to predicting the furthest achievable distance than the first in-vehicle navigation route. 1 It is also possible to obtain extended navigation routes, and furthermore, the acquisition of the first and second extended navigation routes can be performed according to navigation events and / or in-vehicle navigation maps only, without relying on high-precision maps.

[0009] In a second possible embodiment of the first aspect, where the target landmark information includes the first landmark information and / or the second landmark information, the method for acquiring the target landmark information included in a target road environment image collected by the target vehicle at a target time is: Steps include obtaining input data for a landmark detection model, The step of processing the input data based on the landmark detection model and obtaining the target landmark information included in the target road environment image at the target time, The input data includes the target road environment image at the target time and the target local pose at the target time, wherein the target local pose includes an offset amount between the global pose of the target vehicle at the time the target road environment image is collected and the global pose at the target start point, the target time is any time when the target vehicle is collecting the target road environment image on the target extended navigation route, and if the target landmark information is the first landmark information, the target vehicle is the first vehicle, the target road environment image is the first road environment image, the target extended navigation route is the first extended navigation route, and the target local pose is the first local pose; if the target landmark information is the second landmark information, the target vehicle is the second vehicle, the target road environment image is the second road environment image, the target extended navigation route is the second extended navigation route, and the target local pose is the second local pose.

[0010] According to the above technical means, the embodiment of the present invention can not only automatically sense target landmark information contained in a target road environment image at any given time based on a pre-trained landmark sensing model, but also, since the input data for the landmark sensing model is independent of high-precision maps and includes only the target road environment image and target local poses, it is possible to acquire target landmark information even when the system is not based on high-precision maps.

[0011] In a third possible implementation of the first embodiment, the target time is the time when the target road environment image is being collected for the Nth time on the target extended navigation route, the input data further includes the output result of the landmark sensing model at a time adjacent to the target time, the output result at the time prior to the target time includes the target landmark information contained in the target road environment image collected at the time prior, and N is a positive integer of 2 or more.

[0012] According to the above technical means, when the target time is not the time when the target road environment image is collected for the first time on the target extended navigation route, the embodiment of the present invention can use the output result of the landmark detection model at the previous time as one of the input data for the landmark detection model corresponding to the target time, thereby improving the accuracy of detection by the landmark detection model.

[0013] In the fourth possible implementation of the first embodiment, the method for generating the landmark sensing model is: Steps to obtain a landmark training sample set, The steps include: training using the aforementioned landmark training sample set to obtain a landmark sensing model, Each training sample in the aforementioned landmark training sample set includes a road environment sample image group, a third local pose corresponding to the road environment sample image for each frame in the road environment sample image group, and a landmark truth value corresponding to the road environment sample image for each frame. sample The image group includes a series of consecutive frames of sample images of the road environment. The landmark detection model is for detecting and outputting landmark information in the road environment sample image.

[0014] According to the above technical means, the landmark detection model is trained on multiple road environment sample image groups. In the machine learning process, when detecting landmark information for each frame of a road environment sample image, the system refers to the adjacent frames before and after the frame in the road environment sample image group (either one adjacent frame or multiple adjacent frames). Therefore, when detecting landmarks based on the landmark detection model for a single frame of a road environment image, it is possible to detect invisible landmark information in the road environment image, including landmark information that is not included in the current image or cannot be clearly displayed, such as landmark information outside the sensor's detection range, obstructed landmark information, or landmark information that cannot be clearly displayed due to poor image quality (such as at night or in bright scenes). This achieves a detection effect that allows for the acquisition of even invisible landmarks. In addition, since both the acquisition of the third local pose and landmark truth value in the landmark training sample set are independent of high-precision maps, a distance prediction model capable of detecting landmark information in road environment sample images can be automatically trained, free from the frequent updates of high-precision maps.

[0015] In the fifth possible implementation of the first embodiment, the method for obtaining the landmark truth value is: For each of the road environment sample image groups awaiting processing, the steps include: obtaining the corresponding road measurement time-series data for the road environment sample image group based on the third local pose of the road environment sample image for each frame in the road environment sample image group; The steps include constructing a first local map based on the aforementioned road measurement time-series data, The method includes the step of obtaining the landmark truth values ​​of each road environment sample image for each frame by projecting the first local map onto each of the road environment sample images in the road environment sample image group.

[0016] According to the above technical means, the embodiment of the present application constructs a first local map based on the corresponding road measurement time series data of the road environment sample image group, and projects the first local map onto the road environment sample image, thereby obtaining the landmark truth value of the road environment sample image for each frame. As a result, on the premise of extracting from the high-precision map, it is possible to automatically mark the existence of landmark information invisible in the road environment sample image, providing a technical basis for bringing the technical effect that even invisible information can be obtained to the landmark perception model.

[0017] In the sixth possible implementation form of the first aspect, the method for obtaining the truth value of the farthest reachable distance is as follows: Generating a second local map including the second extended navigation route based on a continuous video stream, and driving according to the second extended navigation route in the second local map; Calculating the farthest reachable distance on each lane in front of each of the second global poses on the second extended navigation route based on the second local map; Taking the calculated farthest reachable distance as the truth value, performing truth value marking on the corresponding second landmark information, and obtaining the truth value of the farthest reachable distance for each lane in the second landmark information.

[0018] According to the above technical means, the embodiment of the present application can, without manual intervention, generated realize the truth value marking of the farthest reachable distance in the second landmark information by means of a second local map including the second extended navigation route and the second extended navigation route by based on a continuous video stream, thereby improving the efficiency of truth value marking of the farthest reachable distance.

[0019] In a second aspect, an embodiment of the present application provides a method for training a distance prediction model, the method comprising: obtaining a distance training sample set; training using the distance training sample set to obtain a distance prediction model, each training sample in the distance training sample set includes a second extended navigation route, second landmark information, a second global pose, and a ground truth value of the farthest reachable distance, the second extended navigation route includes an extended route of a second vehicle-mounted navigation route, the second landmark information includes landmark information included in a second road environment image collected by a second vehicle on the second extended navigation route, and the second global pose includes the global pose when the second vehicle collects the second road environment image; the distance prediction model is for predicting the farthest reachable distance on each lane in front of an arbitrary vehicle.

[0020] According to the above technical means, the embodiment of the present application can train and obtain a distance prediction model based on a distance training sample set including a second extended navigation route, second landmark information, a second global pose, and a ground truth value of the farthest reachable distance for each lane in the second landmark information. Of course, the acquisition of these sample information does not depend on a high-precision map, so it is possible to automatically train a distance prediction model that can predict the farthest reachable distance on each lane in front of a vehicle while getting out of the frequent update of the high-precision map, and use the distance prediction model to automatically predict the farthest reachable distance on each lane in front of the vehicle. As a result, it becomes easy to quickly perform route planning according to the farthest reachable distance on each lane in front of the vehicle. Furthermore, it is possible to realize intelligent driving plan control while saving economic cost and time cost.

[0021] In a first possible implementation form of the second aspect, the method for obtaining the second extended navigation route is Steps include obtaining a second in-vehicle navigation route and a second global pause when the second vehicle is traveling on the second in-vehicle navigation route, The steps include extracting points of interest (POI) information corresponding to the second global pose from the target data, The steps include adding the POI information to the second in-vehicle navigation route and obtaining the second extended navigation route, The target data includes navigation events and / or in-vehicle navigation maps, the POI information includes road attribute information related to the prediction of the furthest reachable distance, and the POI information corresponding to the second global pose includes the POI information within a predetermined distance range ahead of the second global pose on the second in-vehicle navigation route.

[0022] According to the above technical means, the embodiment of the present application can extract POI information corresponding to a second global pose from navigation events and / or an in-vehicle navigation map, add the extracted POI information to a second in-vehicle navigation route, and obtain a second extended navigation route that contributes to training a distance prediction model of higher quality than the second in-vehicle navigation route. Therefore, the acquisition of the second extended navigation route can be performed according to navigation events and / or an in-vehicle navigation map, without relying on a high-precision map.

[0023] In a second possible embodiment of the second aspect, the method for acquiring the second landmark information contained in the second road environment image collected by the second vehicle at a target time is: Steps include obtaining input data for a landmark detection model, The step includes processing the input data based on the landmark sensing model and obtaining the second landmark information included in the second road environment image at the target time, The input data includes the second road environment image at the target time and the second local pose at the target time, wherein the second local pose includes an offset amount from the global pose at the target start point of the second vehicle's global pose when the second road environment image is being collected, and the target time is any point in time when the second vehicle is collecting the second road environment image on the second extended navigation route.

[0024] According to the above technical means, the embodiment of the present invention can not only automatically detect second landmark information contained in a second road environment image at any point in time based on a landmark detection model that has been trained in advance, but also, since the input data for the landmark detection model is unrelated to high-precision maps and includes only the second road environment image and the second local pose, the second landmark information can be acquired while being outside the realm of high-precision maps.

[0025] In a third possible implementation of the second embodiment, the target time is the time when the second road environment image is being collected for the Nth time on the second extended navigation route, the input data further includes the output result of the landmark sensing model at a time adjacent to the target time, the output result at the time prior to the target time includes the second landmark information contained in the second road environment image collected at the time prior to the target time, and N is a positive integer of 2 or more.

[0026] According to the above technical means, when the target time is not the time when the second road environment image is collected for the first time on the second extended navigation route, the embodiment of the present invention can use the output result of the landmark detection model at the previous time as one of the input data for the landmark detection model corresponding to the target time, thereby improving the accuracy of detection by the landmark detection model.

[0027] In the fourth possible implementation of the second embodiment, the method for generating the landmark sensing model is: Steps to obtain a landmark training sample set, The steps include: training using the aforementioned landmark training sample set to obtain a landmark sensing model, Each training sample in the aforementioned landmark training sample set includes a road environment sample image group, a third local pose corresponding to the road environment sample image for each frame in the road environment sample image group, and a landmark truth value corresponding to the road environment sample image for each frame. sample The image group includes a series of consecutive frames of sample images of the road environment. The landmark detection model is for detecting and outputting landmark information in the road environment sample image.

[0028] According to the above technical means, the landmark detection model is trained on multiple road environment sample image groups. In the machine learning process, when detecting landmark information for each frame of a road environment sample image, the system refers to the adjacent frames before and after the frame in the road environment sample image group (either one adjacent frame or multiple adjacent frames). Therefore, when detecting landmarks based on the landmark detection model for a single frame of a road environment image, it is possible to detect invisible landmark information in the road environment image, including landmark information that is not included in the current image or cannot be clearly displayed, such as landmark information outside the sensor's detection range, obstructed landmark information, or landmark information that cannot be clearly displayed due to poor image quality (such as at night or in bright scenes). This achieves a detection effect that allows for the acquisition of even invisible landmarks. In addition, since both the acquisition of the third local pose and landmark truth value in the landmark training sample set are independent of high-precision maps, a distance prediction model capable of detecting landmark information in road environment sample images can be automatically trained, escaping the frequent updates of high-precision maps.

[0029] In the fifth possible implementation of the second aspect, the method for obtaining the landmark truth value is: For each of the road environment sample image groups awaiting processing, the steps include: obtaining the corresponding road measurement time-series data for the road environment sample image group based on the third local pose of the road environment sample image for each frame in the road environment sample image group; The steps include constructing a first local map based on the aforementioned road measurement time-series data, The method includes the step of obtaining the landmark truth values ​​of each road environment sample image for each frame by projecting the first local map onto each of the road environment sample images in the road environment sample image group.

[0030] According to the above technical means, the embodiment of the present application constructs a first local map based on corresponding road measurement time-series data of a road environment sample image group, and projects the first local map onto the road environment sample image, thereby enabling the acquisition of landmark truth values ​​for each frame of the road environment sample image. This allows for the automatic marking of invisible landmark information in the road environment sample image, assuming it is removed from the high-precision map, and provides a technical foundation for bringing the technical effect of acquiring even invisible information to the landmark sensing model.

[0031] In the sixth possible implementation of the second embodiment, the method for obtaining the truth value of the furthest reachable distance is: The steps include generating a second local map including the second extended navigation route based on a continuous video stream, and driving in the second local map according to the second extended navigation route, A step of calculating the furthest reachable distance on each lane ahead of each of the second global pauses on the second extended navigation route, based on the second local map, The process includes the steps of: using the calculated furthest reachable distance as a truth value, marking the corresponding second landmark information with a truth value, and obtaining the truth value of the furthest reachable distance for each lane in the second landmark information.

[0032] According to the above technical means, the embodiment of the present application is based on a continuous video stream without human intervention. generated Second local map including second extended navigation route and second extended navigation route by This enables the realization of truth value marking of the furthest reachable distance in the second landmark information, thereby improving the efficiency of truth value marking of the furthest reachable distance.

[0033] In a third aspect, an embodiment of the present application provides a vehicle planning control system comprising a positioning module, an extended route module, a sensing module, a distance prediction module, and a planning control module. The positioning module is used to acquire a first global pose and a first local pose of a first vehicle, the first local pose including an offset amount relative to the global pose of the first vehicle at the target starting point when the first road environment image is being collected. The extended route module is used to determine a first extended navigation route based on the first global pause, and the first extended navigation route includes an extended route of the first in-vehicle navigation route. The sensing module is used to sense first landmark information and target information around the first vehicle, wherein the first landmark information includes landmark information included in a first road environment image collected by the first vehicle in the first global pause, and the target information includes at least one of the following: traffic signal information in front of the first vehicle, static object information around the first vehicle, and dynamic object information around the first vehicle. The distance prediction module is used to obtain the furthest reachable distance on each lane in the first landmark information, based on the method described in any embodiment of the first aspect. The planning control module determines the planned trajectory of the first vehicle based on user prediction information, the furthest distance that can be reached on each lane in the first landmark information, the first local pause, and the target information, and is used to control the driving of the first vehicle based on the planned trajectory.

[0034] According to the above technical means, the planning control system provided by the embodiment of the present application, comprising a positioning module, an extended route module, a sensing module, a distance prediction module, and a planning control module, automatically predicts the furthest reachable distance in each lane in first landmark information using an automatically trained distance prediction model that does not rely on high-precision maps, and determines the planned trajectory of the first vehicle based on user prediction information, the furthest reachable distance in each lane in first landmark information, first local pause, and target information, and controls the driving of the first vehicle based on the planned trajectory. This not only enables the planning control of intelligent driving while saving economic and time costs, but also employs a decoupling design for global positioning and local positioning, meaning that when each module processes data, global positioning and local positioning do not affect each other, and only one of the positioning results is required. Furthermore, even if the accuracy of global positioning deteriorates, it does not affect local positioning, thus not affecting real-time sensing and planning control results, and preventing false emergency braking.

[0035] In a first possible implementation of the third embodiment, the extended route module is used to obtain the first global pose and a first in-vehicle navigation route determined based on the first global pose, to extract point of interest (POI) information corresponding to the first global pose from target data, and to add the POI information to the first in-vehicle navigation route to obtain the first extended navigation route. The target data includes navigation events and / or in-vehicle navigation maps, the POI information includes road attribute information related to the prediction of the furthest reachable distance, and the POI information corresponding to the first global pose includes the POI information within a predetermined distance range ahead of the first global pose on the first in-vehicle navigation route.

[0036] In a second possible implementation of the third embodiment, the sensing module is used to acquire input data for a landmark sensing model, and to process the input data based on the landmark sensing model to acquire the first landmark information included in the first road environment image at the target time. The input data includes the first road environment image at the target time and the first local pose at the target time, where the target time is any point in time when the first vehicle is collecting the first road environment image on the first extended navigation route.

[0037] In a third possible implementation of the third embodiment, the target time is the time when the first road environment image is being collected for the Nth time on the first extended navigation route, the input data acquired by the sensing module further includes the output result of the landmark sensing model at a time adjacent to the target time, the output result at the time prior to the target time includes the first landmark information contained in the first road environment image collected at the time prior to the target time, and N is a positive integer of 2 or more.

[0038] In a fourth aspect, an embodiment of the present application provides a distance prediction device, the device including an acquisition unit and a distance prediction unit. The acquisition unit is used to acquire a first global pose of a first vehicle, a first extended navigation route, and first landmark information, wherein the first global pose is the current global pose of the first vehicle, the first extended navigation route includes an extended route of a first in-vehicle navigation route determined based on the first global pose, and the first landmark information includes landmark information contained in a first road environment image collected by the first vehicle in the first global pose. The distance prediction unit is used to process the first global pose, the first extended navigation route, and the first landmark information based on a distance prediction model, and to obtain the furthest distance that can be reached on each lane in the first landmark information. The acquisition unit is further used to acquire a distance training sample set before processing the first global pose, the first extended navigation route, and the first landmark information based on the distance prediction model, wherein each training sample in the distance training sample set includes a second extended navigation route, a second landmark information, a second global pose, and a truth value for the furthest reachable distance, the second extended navigation route includes an extended route of the second in-vehicle navigation route, the second landmark information includes landmark information contained in a second road environment image collected by a second vehicle on the second extended navigation route, the second global pose includes the global pose when the second vehicle collected the second road environment image, and the truth value for the furthest reachable distance includes the truth value for the furthest reachable distance per lane in the second landmark information. The aforementioned device is The system further includes a training unit for training using the aforementioned distance training sample set to obtain the aforementioned distance prediction model.

[0039] In the first possible implementation of the fourth aspect, the acquisition unit includes a first acquisition module, an extraction module, and an additional module, The first acquisition module is used to acquire the target in-vehicle navigation route and the target global pose when the target vehicle is traveling on the target in-vehicle navigation route, where the target extended navigation route includes the first extended navigation route and / or the second extended navigation route, wherein if the target extended navigation route is the first extended navigation route, the target in-vehicle navigation route is the first in-vehicle navigation route, the target vehicle is the first vehicle, and the target global pose is the first global pose; if the target extended navigation route is the second extended navigation route, the target in-vehicle navigation route is the second in-vehicle navigation route, and the target global pose is the second global pose. The extraction module is used to extract points of interest (POI) information corresponding to the target global pose from the target data, the target data includes navigation events and / or in-vehicle navigation maps, the POI information includes road attribute information related to the prediction of the furthest reachable distance, and the POI information corresponding to the target global pose includes the POI information within a predetermined distance range ahead of the target global pose on the target in-vehicle navigation route. The additional module is used to add the POI information to the target in-vehicle navigation route and to obtain the target extended navigation route.

[0040] In the second possible implementation of the fourth aspect, the acquisition unit includes a second acquisition module and a processing module, The second acquisition module is used to acquire input data for a landmark sensing model when the target landmark information includes the first landmark information and / or the second landmark information, the input data includes the target road environment image at the target time and the target local pose at the target time, the target local pose includes an offset amount of the global pose of the target vehicle at the time the target road environment image is collected relative to the global pose at the target start point, the target time is any time when the target vehicle is collecting the target road environment image on the target extended navigation route, if the target landmark information is the first landmark information, the target vehicle is the first vehicle, the target road environment image is the first road environment image, the target extended navigation route is the first extended navigation route, and the target local pose is the first local pose, if the target landmark information is the second landmark information, the target vehicle is the second vehicle, the target road environment image is the second road environment image, the target extended navigation route is the second extended navigation route, and the target local pose is the second local pose, The processing module is used to process the input data based on the landmark detection model and to obtain the target landmark information included in the target road environment image at the target time.

[0041] In a third possible implementation of the fourth embodiment, where the target time is the time when the target road environment image is being collected for the Nth time on the target extended navigation route, the input data further includes the output result of the landmark sensing model at a time adjacent to the target time, the output result at the time prior to the target time includes the target landmark information contained in the target road environment image collected at the time prior, and N is a positive integer of 2 or more.

[0042] In the fourth possible embodiment of the fourth aspect, the acquisition unit is: The processing module further includes a first generation module for generating the landmark sensing model before processing the input data based on the landmark sensing model and obtaining the target landmark information included in the target road environment image at the target time, The first generation module includes an acquisition submodule and a training submodule, The acquisition submodule is used to acquire a landmark training sample set, and each training sample in the landmark training sample set includes a road environment sample image group, and a third local pose corresponding to the road environment sample image for each frame in the road environment sample image group, and a landmark truth value corresponding to the road environment sample image for each frame, and the road environment sample The image group includes a series of consecutive frames of sample images of the road environment. The training submodule is used to train using the landmark training sample set and to acquire a landmark sensing model for sensing and outputting landmark information in the road environment sample image.

[0043] In the fifth possible implementation of the fourth embodiment, the acquisition submodule is used for each of the road environment sample image groups awaiting processing to acquire the corresponding road measurement time series data for the road environment sample image group based on the third local pose of the road environment sample image for each frame in the road environment sample image group; to construct a first local map based on the road measurement time series data; and to acquire the landmark truth values ​​of the road environment sample image for each frame by projecting the first local map onto each of the road environment sample images in the road environment sample image group.

[0044] In the sixth possible embodiment of the fourth aspect, the acquisition unit is: A second generation module for generating a second local map including the second extended navigation route based on a continuous video stream, A driving module for driving according to the second extended navigation route in the second local map, A calculation module for calculating the furthest reachable distance on each lane ahead of each of the second global pauses on the second extended navigation route, based on the second local map, The system includes a truth value marking module for obtaining the truth value of the furthest reachable distance for each lane in the second landmark information, by using the calculated furthest reachable distance as the truth value and marking the corresponding second landmark information with truth value.

[0045] According to the technical means described above, an embodiment of the present application first trains and acquires a distance prediction model based on a distance training sample set including a second extended navigation route, a second landmark information, a second global pose, and the truth value of the furthest reachable distance for each lane in the second landmark information. Then, the first global pose, the first extended navigation route, and the first landmark information are input into the distance prediction model to predict the furthest reachable distance on each lane in the first landmark information. Since the second global pose, second extended navigation route, and second landmark information required during model training, as well as the first global pose, first extended navigation route, and first landmark information required during model application, do not depend on high-precision maps, the embodiment of the present invention can automatically train a distance prediction model capable of predicting the furthest reachable distance in each lane ahead of the vehicle, while being free from the frequent updates of high-precision maps, and can use this distance prediction model to automatically predict the furthest reachable distance in each lane ahead of the vehicle. This makes it easy to quickly plan routes according to the furthest reachable distance in each lane ahead of the vehicle, and further enables intelligent driving planning control while saving economic and time costs.

[0046] In a fifth aspect, an embodiment of the present application provides a training device for a distance prediction model, the device comprising an acquisition unit and a training unit, The acquisition unit is used to acquire a distance training sample set, each training sample in the distance training sample set includes a second extended navigation route, second landmark information, a second global pose, and a truth value for the furthest achievable distance, the second extended navigation route includes an extended route of the second in-vehicle navigation route, the second landmark information includes landmark information contained in a second road environment image collected by a second vehicle on the second extended navigation route, and the second global pose includes the global pose when the second vehicle collected the second road environment image. The training unit is used to train using the distance training sample set to obtain a distance prediction model for predicting the furthest distance that can be reached in each lane ahead of any given vehicle.

[0047] In the first possible implementation of the fifth aspect, the acquisition unit includes a first acquisition module, an extraction module, and an additional module, The first acquisition module is used to acquire a second in-vehicle navigation route and a second global pause when the second vehicle is traveling along the second in-vehicle navigation route. The extraction module is used to extract point of interest (POI) information corresponding to the second global pose from target data, the target data includes navigation events and / or in-vehicle navigation maps, the POI information includes road attribute information related to the prediction of the furthest reachable distance, and the POI information corresponding to the second global pose includes the POI information within a predetermined distance range ahead of the second global pose on the second in-vehicle navigation route. The additional module is used to add the POI information to the second in-vehicle navigation route and to obtain the second extended navigation route.

[0048] In the second possible implementation of the fifth aspect, the acquisition unit includes a second acquisition module and a processing module, The second acquisition module is used to acquire input data for a landmark sensing model, the input data including the second road environment image at the target time and the second local pose at the target time, the second local pose including an offset amount from the global pose of the second vehicle at the target start point to the global pose when the second road environment image is being collected, and the target time is any point in time when the second vehicle is collecting the second road environment image on the second extended navigation route. The processing module is used to process the input data based on the landmark detection model and to obtain the second landmark information contained in the second road environment image at the target time.

[0049] In a third possible implementation of the fifth embodiment, the target time is the time when the second road environment image is being collected for the Nth time on the second extended navigation route, the input data further includes the output result of the landmark sensing model at a time adjacent to the target time, the output result at the time prior to the target time includes the second landmark information contained in the second road environment image collected at the time prior to the target time, and N is a positive integer of 2 or more.

[0050] In the fourth possible embodiment of the fifth aspect, the acquisition unit is: The system further includes a first generation module for generating the landmark detection model, before processing the input data based on the landmark detection model and obtaining the second landmark information contained in the second road environment image at the target time, The first generation module includes an acquisition submodule and a training submodule, The acquisition submodule is used to acquire a landmark training sample set, and each training sample in the landmark training sample set includes a road environment sample image group, and a third local pose corresponding to the road environment sample image for each frame in the road environment sample image group, and a landmark truth value corresponding to the road environment sample image for each frame, and the road environment sample The image group includes a series of consecutive frames of sample images of the road environment. The training submodule is used to train using the landmark training sample set and to acquire a landmark sensing model for sensing and outputting landmark information in the road environment sample image.

[0051] In the fifth possible implementation of the fifth aspect, the acquisition submodule is used for each of the road environment sample image groups awaiting processing to acquire the corresponding road measurement time series data for the road environment sample image group based on the third local pose of the road environment sample image for each frame in the road environment sample image group; to construct a first local map based on the road measurement time series data; and to acquire the landmark truth values ​​of the road environment sample image for each frame by projecting the first local map onto each of the road environment sample images in the road environment sample image group.

[0052] In the sixth possible embodiment of the fifth aspect, the acquisition unit is: A second generation module for generating a second local map including the second extended navigation route based on a continuous video stream, A driving module for driving according to the second extended navigation route in the second local map, A calculation module for calculating the furthest reachable distance on each lane ahead of each of the second global pauses on the second extended navigation route, based on the second local map, The system includes a truth value marking module for obtaining the truth value of the furthest reachable distance for each lane in the second landmark information, by using the calculated furthest reachable distance as the truth value and marking the corresponding second landmark information with truth value.

[0053] According to the above technical means, embodiments of the present invention can train and acquire a distance prediction model based on a distance training sample set including a second extended navigation route, second landmark information, second global pose, and truth values ​​of the furthest reachable distance for each lane in the second landmark information. Naturally, since the acquisition of this sample information does not depend on high-precision maps, a distance prediction model capable of predicting the furthest reachable distance in each lane ahead of the vehicle can be automatically trained, free from the frequent updates of high-precision maps, and the furthest reachable distance in each lane ahead of the vehicle can be automatically predicted using this distance prediction model. This makes it easier to quickly plan routes according to the furthest reachable distance in each lane ahead of the vehicle, and further enables intelligent driving planning control while saving economic and time costs.

[0054] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium in which a computer program is stored, and when the program is executed by a processor, the method of any one possible implementation of the first aspect or any one possible implementation of the second aspect is implemented.

[0055] In the seventh aspect, an embodiment of the present application provides an electronic device, the electronic device is One or more processors, The processor includes a memory device for storing one or more programs, When one or more programs are executed by one or more processors, the electronic device implements a method according to any one possible implementation of the first embodiment or any one possible implementation of the second embodiment.

[0056] In the eighth aspect, an embodiment of the present application provides a vehicle which is a system described in any one possible embodiment of the third aspect, or the 4 Any one possible implementation of the form or the 5 An apparatus according to any one possible embodiment of the embodiments, or the 7 The electronic device is provided as described in the embodiment.

[0057] In the ninth aspect, an embodiment of the present application provides a computer program product which includes instructions, and when the instructions are executed by a computer or processor, the computer or processor performs a method according to any one possible implementation of the first aspect or any one possible implementation of the second aspect. [Brief explanation of the drawing]

[0058] To more clearly describe the embodiments of the present application or the solutions of the prior art, the drawings that may be used in the description of the embodiments or the prior art are briefly described below. Naturally, the drawings described below are some embodiments of the present application, and those skilled in the art will be able to conceive of other drawings based on these without requiring any creative effort. [Figure 1] This is a flowchart of the distance prediction method provided by the embodiment of the present invention. [Figure 2] This figure shows an example of the furthest distance that can be reached in landmark information provided by the embodiment of the present invention. [Figure 3] This is a flowchart of the training method for the distance prediction model provided by the embodiment of the present invention. [Figure 4]This figure shows an example of an in-vehicle navigation route and an extended navigation route provided by an embodiment of the present invention. [Figure 5] This is a structural diagram of a vehicle planning control system provided by an embodiment of the present invention. [Figure 6] This figure shows an example of planning control by the vehicle planning control system provided by the embodiment of the present invention. [Figure 7] This is a block diagram of the configuration of the distance prediction device provided by the embodiment of the present application. [Figure 8] This is a block diagram of the configuration of a training device for a distance prediction model provided by an embodiment of the present invention. [Figure 9] This is a schematic diagram of the structure of an electronic device or computer device provided by an embodiment of the present application. [Modes for carrying out the invention]

[0059] The technical solutions described below will be clearly and completely explained with reference to the drawings relating to the embodiments of this application. Naturally, the embodiments described are only a part of the embodiments of this application, not all of them. A person skilled in the art will not need to perform any creative work based on the embodiments of this application to obtain all other embodiments which fall within the scope of protection of this application.

[0060] The embodiments and features described herein can be combined with each other, provided they do not contradict each other. The terms “includes” and “has,” and their variations, relating to the embodiments and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a set of steps or units may, but is not limited to, further selectively include steps or units not listed, or further selectively include other steps or units specific to those processes, methods, products, or apparatus.

[0061] Figure 1 is a flowchart of a distance prediction method, which can be applied to electronic devices or computer equipment, specifically vehicles or servers. Specifically, the method may include steps S110 to S120.

[0062] In S110, the first global pause, first extended navigation route, and first landmark information of the first vehicle are obtained.

[0063] The first global pose is the current global pose of the first vehicle, the first extended navigation route includes an extended route of the first in-vehicle navigation route determined based on the first global pose, and the first landmark information includes landmark information contained in the first road environment image collected by the first vehicle in the first global pose.

[0064] The pose includes the vehicle's position and attitude. The global pose includes the vehicle's position and attitude based on global positioning, and acquiring the global pose does not require high-precision positioning such as RTK (Real-time kinematic, carrier phase difference technology); it can be acquired with only road-level positioning, such as navigation positioning. Global positioning can perform positioning using the Earth as the reference system, meaning that the coordinate system for global positioning can be a geodetic coordinate system.

[0065] After the user inputs a starting point (which may be the default current location) and an ending point into the in-vehicle navigation software, the in-vehicle navigation software generates a first in-vehicle navigation route according to the starting point and ending point, and the navigation positioning system determines the first global position of the first vehicle in real time. Here, the in-vehicle navigation software may be the software of the in-vehicle unit or the software of a mobile terminal communicating with the in-vehicle unit, and the in-vehicle navigation route may be the route generated by the in-vehicle navigation software of the in-vehicle unit or the route generated by the in-vehicle navigation software of a mobile terminal communicating with the in-vehicle unit, and the embodiment of the present application does not limit the source of the in-vehicle navigation route.

[0066] The specific methods for obtaining the first extended navigation route and the first landmark information can be found in the method for obtaining the target extended navigation route and target landmark information described later, and will not be explained here.

[0067] In S120, the first global pose, the first extended navigation route, and the first landmark information are processed based on a distance prediction model to obtain the furthest achievable distance on each lane in the first landmark information.

[0068] After obtaining the first global pose, the first extended navigation route, and the first landmark information, the first global pose, the first extended navigation route, and the first landmark information are input into a pre-trained distance prediction model for calculation, and the furthest reachable distance on each lane in the first landmark information output from the distance prediction model is obtained. The furthest reachable distance obtained on each lane in the first landmark information may be expressed as an actual distance, such as 2000 meters, or 0 is 0 meters, 1 is (0, 200) meterThe symbols may be mapped to actual distances, such that 2 represents (200,400) meters, 3 represents (400,600) meters, 4 represents (600,800) meters, 5 represents (800,1000) meters, 6 represents (1000,2000) meters, and 7 represents 2000 meters or more. As shown in Figure 2, the first landmark information has four lanes, and if the furthest distances that can be reached from left to right are 7, 7, 0, and 7 respectively, then there are three lanes that can be driven in.

[0069] After obtaining the furthest reachable distance in each lane based on the first landmark information, the furthest reachable distance in each lane based on the first landmark information, user prediction information, first local pause, and target object information around the first vehicle are input to the planning control module, and planning control results such as whether or not to change lanes, when to change lanes, whether or not to accelerate or decelerate, when to accelerate or decelerate, and the future planned route trajectory can be obtained. User prediction information includes user operations on the vehicle, such as lever operation for lane changes, activation of lane change lights, and brake operation, and target object information includes at least one of the following: surrounding vehicles, pedestrians, obstacles, traffic lights, traffic signal information in front of the first vehicle, static object information around the first vehicle, and dynamic object information around the first vehicle. As the furthest reachable distance in the lane the vehicle is currently traveling in decreases, the planning control module refers to the user prediction information, first local pause, and target object information to determine whether or not to change lanes and when to change lanes.

[0070] The distance prediction method according to the embodiment of the present invention first trains and acquires a distance prediction model based on a distance training sample set that includes a second extended navigation route, a second landmark information, a second global pause, and the truth value of the furthest distance that can be reached for each lane in the second landmark information. Next, the first global pause, the first extended navigation route, and the first landmark information are input into the distance prediction model to predict the furthest distance that can be reached on each lane in the first landmark information. Since the second global pose, second extended navigation route, and second landmark information required during model training, as well as the first global pose, first extended navigation route, and first landmark information required during model application, do not depend on high-precision maps, the embodiment of the present invention can automatically train a distance prediction model capable of predicting the furthest reachable distance in each lane ahead of the vehicle, while being free from the frequent updates of high-precision maps, and can use this distance prediction model to automatically predict the furthest reachable distance in each lane ahead of the vehicle. This makes it easy to quickly plan routes according to the furthest reachable distance in each lane ahead of the vehicle, and further enables intelligent driving planning control while saving economic and time costs.

[0071] Figure 3 is a flowchart of a distance prediction model training method, which can be applied to electronic devices or computer equipment, specifically vehicles or servers. Specifically, the method may include steps S210 to S220.

[0072] In S210, a distance training sample set is obtained, and each training sample in the distance training sample set contains a second extended navigation route, second landmark information, second global pause, and a truth value for the furthest achievable distance.

[0073] Of these, the second extended navigation route includes an extended route of the second in-vehicle navigation route, the second landmark information includes landmark information contained in the second road environment image collected by the second vehicle on the second extended navigation route, the second global pause includes the global pause when the second vehicle is collecting the second road environment image, and the truth value of the furthest reachable distance includes the truth value of the furthest reachable distance for each lane in the second landmark information. The specific methods for acquiring the second extended navigation route and the second landmark information can be found in the methods for acquiring the target extended navigation route and target landmark information described later, and are omitted here. The navigation positioning system positions the second global pause of the second vehicle in real time.

[0074] The following explains how to obtain the truth value for the furthest distance that can be reached.

[0075] The truth value of the furthest reachable distance may be marked manually or automatically. An automatic marking method may include the steps of: generating a second local map including a second extended navigation route based on a continuous video stream and driving along the second extended navigation route in the second local map; calculating the furthest reachable distance on each lane ahead of each second global pause on the second extended navigation route based on the second local map; and marking the corresponding second landmark information with the calculated furthest reachable distance as a truth value to obtain the truth value of the furthest reachable distance for each lane in the second landmark information.

[0076] A continuous video stream can be generated by a device such as an image acquisition device or dashcam of a second vehicle, as long as it is a video stream capable of recording road environment information in front of or around a second vehicle while it is in motion. Here, "continuous" includes not only literal continuity but also the fact that the total number of frames of the video stream collected within a predetermined time exceeds a certain predetermined value.

[0077] As an additional explanation, the furthest reachable distance may be the furthest reachable distance within a certain distance range, and this distance range is usually larger than the distance range included in the road environment image. For example, if the road environment image covers a range of 100 meters, the furthest reachable distance can be limited to a distance range of 2 kilometers.

[0078] In S220, a distance prediction model is obtained by training it using a distance training sample set.

[0079] The distance prediction model is for predicting the furthest distance a vehicle can reach in each lane ahead. In the embodiment of the present invention, the distance prediction model can be obtained by multiple training iterations. Thus, each time training is performed based on the distance training sample set, the distance prediction model obtained in the current training iteration is acquired, and then the second road environment image of at least one frame is processed based on the distance prediction model obtained in the current training iteration. The predicted value of the furthest distance that can be reached for each frame of the second road environment image of at least one frame is acquired, and a loss value is calculated based on the difference between the predicted value of the furthest distance that can be reached and the corresponding truth value of the furthest distance that can be reached. If the loss value is greater than a second loss threshold, the training iteration is continued, and training is stopped until the loss value becomes less than or equal to the second loss threshold, and the distance prediction model finally obtained through training is the final required distance prediction model.

[0080] Furthermore, the first vehicle may be the vehicle used when training the distance prediction model, or it may not be the vehicle used when training the distance prediction model. Therefore, the first vehicle may be the second vehicle, or it may not be the second vehicle. Distance prediction model training The number of second vehicles participating may be one or more.

[0081] When a distance prediction model is first trained and converged on the second road environment image for the first time, and then the information in the first road environment image is processed using the distance prediction model, the second road environment image is acquired earlier than the first road environment image, meaning that the second road environment image belongs to the history of road environment images with respect to the first road environment image. When a distance prediction model is first trained and converged on one group of second road environment images, and then an additional group of second road environment images, including road scenes awaiting capture, is acquired to further train the distance prediction model in order to improve its quality, the second road environment images for further training of the distance prediction model may be acquired later than the first road environment images, meaning that in this case, the second road environment image may not belong to the history of road environment images with respect to the first road environment image.

[0082] Accordingly, the second global pause may belong to the history of the first global pause, or it may not belong to the history of the first global pause. Similarly, the second extended navigation route may belong to the history of the first extended navigation route, or it may not belong to the history of the first extended navigation route. The second landmark information may belong to the history of the first landmark information, or it may not belong to the history of the first landmark information.

[0083] The distance prediction model training method provided by the embodiment of the present invention trains and acquires a distance prediction model based on a distance training sample set including a second extended navigation route, second landmark information, second global pose, and truth values ​​of the furthest reachable distance for each lane in the second landmark information. Naturally, since the acquisition of this sample information does not depend on high-precision maps, a distance prediction model capable of predicting the furthest reachable distance in each lane ahead of the vehicle can be automatically trained, free from the frequent updates of high-precision maps, and the furthest reachable distance in each lane ahead of the vehicle can be automatically predicted using this distance prediction model. This makes it easy to quickly plan routes according to the furthest reachable distance in each lane ahead of the vehicle, and further enables intelligent driving planning control while saving economic and time costs.

[0084] In one embodiment, where the target extended navigation route includes a first extended navigation route and / or a second extended navigation route, the method for obtaining the target extended navigation route is described below.

[0085] After the user enters a starting point (which may be the default current location) and an ending point into the in-vehicle navigation software, the software generates at least one target in-vehicle navigation route according to the starting and ending points, including road names, road types, and road geometry (consisting of a global positioning point set). To enable the distance prediction model to accurately predict the furthest achievable distance for each lane, some auxiliary information can be added to the target in-vehicle navigation route to help the distance prediction model make its predictions.

[0086] An electronic device or computer can first acquire the target in-vehicle navigation route and the target global pose when the vehicle is traveling on the target in-vehicle navigation route, then extract POI information corresponding to the target global pose from the target data, and finally add the POI information to the target in-vehicle navigation route to acquire the target extended navigation route.

[0087] Here, the target data includes navigation events and / or in-vehicle navigation maps, where navigation events refer to events played back by the in-vehicle navigation system, such as "Enter a ramp 1 kilometer ahead," "There is a junction 1 kilometer ahead," "There is a fork in the road ahead," "Take the left fork in the road," and "You will soon enter a tunnel." POI information includes road attribute information related to the prediction of the furthest achievable distance, such as the number of lanes, junctions, speed limit information, and long solid lines. POI information corresponding to the target global pose is POI information within a predetermined distance range ahead of the target global pose on the target in-vehicle navigation route. The predetermined distance range is greater than or equal to the length of the route included in the target road environment image collected by the vehicle at the target global pose.

[0088] As shown in Figure 4, (a) is an in-vehicle navigation route that includes basic information such as road geometry, road type, and road name, and (b) is an extended navigation route after adding POI information to (a). To save storage space, the road geometry can be removed and a simpler shape combining lines and attribute icons can be retained. Specifically, as shown in (c), roads can be shown as straight lines, and merging and branching routes can be shown as dots at their corresponding locations, with corresponding attribute information assigned to the dots.

[0089] Furthermore, if the target extended navigation route is the first extended navigation route, the target in-vehicle navigation route is the first in-vehicle navigation route, the target vehicle is the first vehicle, and the target global pause is the first global pause. If the target extended navigation route is the second extended navigation route, the target in-vehicle navigation route is the second in-vehicle navigation route, and the target global pause is the second global pause.

[0090] Embodiments of the present invention can extract POI information corresponding to a second global pose from navigation events and / or in-vehicle navigation maps, add the extracted POI information to a second in-vehicle navigation route, and obtain a second extended navigation route that contributes to training a distance prediction model of higher quality than the second in-vehicle navigation route. Embodiments of the present invention can also extract POI information corresponding to a first global pose from navigation events and / or in-vehicle navigation maps, add the extracted POI information to a first in-vehicle navigation route, and contribute to predicting the furthest achievable distance beyond the first in-vehicle navigation route. 1 It is also possible to obtain extended navigation routes, and furthermore, the acquisition of the first and second extended navigation routes can be performed according to navigation events and / or in-vehicle navigation maps only, without relying on high-precision maps.

[0091] In one embodiment, where the target landmark information includes first landmark information and / or second landmark information, the method for acquiring the target landmark information is described below.

[0092] The target landmark information obtained from the target road environment image is not landmark information marked on the target road environment image, but rather target landmark information extracted from the target road environment image. Furthermore, the obtained target landmark information retains the geometric shape and the relative positional relationships between each landmark, and its display effect is similar to that of a simple map.

[0093] A method for obtaining target landmark information contained in a target road environment image collected by a vehicle at a target time includes the steps of: obtaining input data for a landmark sensing model; and processing the input data based on the landmark sensing model to obtain target landmark information contained in the target road environment image at the target time. The input data includes the target road environment image at the target time and the target local pose at the target time, the target local pose includes the offset amount of the global pose of the target vehicle at the time the target road environment image is being collected relative to the global pose at the target starting point, and the target time is any point in time when the target vehicle is collecting the target road environment image on the target extended navigation route.

[0094] Here, the coordinate system in which the local pause is located can be the boot coordinate system, and in the boot coordinate system, the point where power is supplied to the vehicle is the origin of the coordinate system, so this coordinate system is also called the start coordinate system. Furthermore, the local pause may perform positioning based on a navigation positioning system without requiring high-precision positioning. The target starting point can be any designated position, for example, it may be the position when power is supplied to the target vehicle, or it may be any designated position while the target vehicle is in motion.

[0095] When there is a difference between the image acquisition cycle of the image collector and the positioning cycle of the navigation positioning system, for example, if the target road environment image is collected at the target time but positioning is not performed at the target time, the target local pose at the most recent positioning time can be used as the target local pose at the target time. However, because the order of magnitude of the difference between the acquisition cycle and the positioning cycle is relatively small, the error that occurs when collecting the target road environment image and target local pose for the same target time when there is a time difference is negligible.

[0096] Furthermore, if the target landmark information is the first landmark information, the target vehicle will be the first vehicle, the target road environment image will be the first road environment image, the target extended navigation route will be the first extended navigation route, and the target local pose will be the first local pose. If the target landmark information is the second landmark information, the target vehicle will be the second vehicle, the target road environment image will be the second road environment image, the target extended navigation route will be the second extended navigation route, and the target local pose will be the second local pose.

[0097] The embodiment of the present invention can automatically detect target landmark information contained in a target road environment image at any given time based on a pre-trained landmark detection model. Furthermore, since the input data for the landmark detection model is independent of high-precision maps and includes only the target road environment image and target local poses, it is possible to acquire target landmark information while detached from high-precision maps.

[0098] In one embodiment, the landmark sensing model may be a generative model or another neural network model, and the embodiments of the present application do not limit the model algorithm specifically used for the landmark sensing model. A method for generating a landmark sensing model includes the steps of obtaining a landmark training sample set, and training using the landmark training sample set to obtain a landmark sensing model, wherein each training sample in the landmark training sample set includes a road environment sample image group, a local pose corresponding to each road environment sample image in the road environment sample image group, and a landmark truth value corresponding to each road environment sample image, for the landmark sensing model to sense and output landmark information in the road environment sample images.

[0099] Here, the road environment sampleThe image group includes multiple consecutive frames of road environment sample images, where "consecutive" includes not only literal continuity but also the total number of road environment sample images collected within a given time period exceeding a specific predetermined value.

[0100] According to the above technical means, the landmark detection model is trained on multiple road environment sample image groups. In the machine learning process, when detecting landmark information for each frame of a road environment sample image, the model refers to the adjacent frames before and after the frame in the road environment sample image group (either one adjacent frame or multiple adjacent frames). Therefore, when detecting landmarks based on the landmark detection model for a single frame of a road environment image, it is possible to detect invisible landmark information in the road environment image, including landmark information that is not included in the current image or cannot be clearly displayed, such as landmark information outside the sensor's detection range, obstructed landmark information, or landmark information that cannot be clearly displayed due to poor image quality (such as at night or in bright scenes). This achieves a detection effect that allows for the acquisition of even invisible landmarks. In addition, since the acquisition of local poses and landmark truth values ​​in the landmark training sample set does not depend on high-precision maps, a distance prediction model capable of detecting landmark information in road environment sample images can be automatically trained, escaping the frequent updates of high-precision maps.

[0101] In one embodiment, the landmark truth values ​​may be marked manually or automatically. An automatic marking method includes, for each road environment sample image group awaiting processing, the steps of: obtaining the corresponding road measurement time series data for the road environment sample image group based on a third local pose of the road environment sample image for each frame in the road environment sample image group; constructing a first local map based on the road measurement time series data; and obtaining the landmark truth values ​​for each frame of the road environment sample image by projecting the first local map onto each frame of the road environment sample image in the road environment sample image group.

[0102] The road measurement time-series data includes the vehicle's trajectory and road environment images collected during the vehicle's journey. The road measurement time-series data may be actual time-series data, including data recorded by a drive recorder, data collected by on-board sensors, data recorded when the server and the vehicle interact, or data recorded by other devices.

[0103] When acquiring corresponding road measurement time series data for a road environment sample image group based on the third local pose of each frame of the road environment sample image in the road environment sample image group, it is possible to first acquire the third local pose of each frame of the road environment sample image in the road environment sample image group, and then search for road measurement time series data that includes these third local poses, and use the road measurement time series data that includes these third local poses as the corresponding road measurement time series data for the road environment sample image group.

[0104] A method for projecting a first local map onto a road environment sample image and obtaining the landmark truth values ​​of the road environment sample image includes the steps of converting the first local map from a map coordinate system to an image coordinate system, and using the landmark information in the road environment sample image of the first local map converted to the image coordinate system as the landmark truth values ​​of the road environment sample image.

[0105] The embodiment of the present invention constructs a first local map based on corresponding road measurement time-series data of a road environment sample image group, and projects the first local map onto the road environment sample image. This makes it possible to obtain the truth value of the landmarks in the road environment sample image for each frame. This allows for the automatic marking of the presence of invisible landmark information in the road environment sample image, assuming that the image is not in high-precision, and provides a technical foundation for bringing the technical effect of being able to acquire even invisible landmarks to the landmark sensing model.

[0106] Each time training is iterated based on the landmark training sample set, the landmark detection model obtained in the current training iteration is acquired. Then, based on the landmark detection model obtained in the current training iteration, at least one group of road environment sample images is processed, and the landmark prediction values ​​for each frame of the road environment sample image in that at least one group of road environment sample images are acquired. A loss value is calculated based on the difference between the landmark prediction value and the corresponding landmark truth value. If the loss value is greater than a first loss threshold, the training iteration is continued, and training is stopped until the loss value becomes less than or equal to the first loss threshold. The landmark detection model finally obtained through training is then considered the final required landmark detection model.

[0107] After training the final landmark detection model, the road environment images for each frame collected by the second vehicle on the second extended navigation route can be processed by directly inputting the road environment image for that frame and a third local pose corresponding to that frame into the landmark detection model, thereby obtaining the landmark information contained in the road environment image for that frame output from the landmark detection model.

[0108] To further improve the accuracy of target landmark information and enhance the sensing effect to acquire even invisible landmarks, the embodiment of the present invention can also use target landmark information, after sensing target landmark information contained in a target road environment image of one frame based on the landmark sensing model, as input information for the landmark sensing model when sensing the road environment image of the next frame. In other words, if the target time is the time when the target road environment image is being collected for the Nth time on the target extended navigation route, the input information for the landmark sensing model can include the target road environment image and target local pose at the target time, as well as the output result of the landmark sensing model at the previous time adjacent to the target time. The output result at the previous time includes target landmark information contained in the target road environment image collected at the previous time. This allows the landmark sensing model to refer to the sensing result of the previous frame as an intermediate value when performing landmark sensing on the road environment image of the current frame, where N is a positive integer of 2 or more.

[0109] Based on the embodiments of the above method, another embodiment of the present application provides a vehicle planning control system, which, as shown in Figures 5 and 6, comprises a positioning module 310, an extended route module 320, a sensing module 330, a distance prediction module 340, and a planning control module 350. The positioning module 310 is used to acquire a first global pose and a first local pose of the first vehicle, the first local pose including an offset amount relative to the global pose at the target starting point of the global pose of the first vehicle when the first road environment image is being collected. The extended route module 320 is used to determine a first extended navigation route based on the first global pause, and the first extended navigation route includes an extended route of the first in-vehicle navigation route. The sensing module 330 is used to sense first landmark information and target information around the first vehicle, wherein the first landmark information includes landmark information included in a first road environment image collected by the first vehicle in the first global pause, and the target information includes at least one of the following: traffic signal information in front of the first vehicle, static object information around the first vehicle, and dynamic object information around the first vehicle. The distance prediction module 340 is used to obtain the furthest reachable distance on each lane in the first landmark information, based on the method described in any embodiment of the above-described distance prediction method. The planning control module 350 determines the planned trajectory of the first vehicle based on user prediction information, the furthest distance that can be reached on each lane in the first landmark information, the first local pause, and the target information, and is used to control the driving of the first vehicle based on the planned trajectory.

[0110] According to the above technical means, the planning control system provided by the embodiment of the present application, comprising a positioning module, an extended route module, a sensing module, a distance prediction module, and a planning control module, does not rely on high-precision maps and automatically predicts the furthest reachable distance on each lane in the first landmark information using an automatically trained distance prediction model. It also determines the planned trajectory of the first vehicle based on the user's prediction information, the furthest reachable distance on each lane in the first landmark information, the first local pause, and target information, and controls the driving of the first vehicle based on the planned trajectory. This not only enables intelligent driving planning control while saving economic and time costs, but also employs a decoupling design for global positioning and local positioning. That is, when each module processes data, global positioning and local positioning do not affect each other, and only one of the positioning results is required. Furthermore, even if the accuracy of global positioning deteriorates, it does not affect local positioning, thus not affecting real-time sensing and planning control results, and preventing false emergency braking.

[0111] In one possible implementation, as shown in Figure 6, the system further comprises an in-vehicle navigation module 360. The in-vehicle navigation module 360, after obtaining the start and end points (i.e., the start and end points) entered by the user, is used to generate a first in-vehicle navigation route and target data according to the start and end points. The start point can be the default current position, i.e., the first global pose at the present time. Figure 6 shows an example for user 1.

[0112] In one possible implementation, the extended route module 320 is used to obtain a first global pose and a first in-vehicle navigation route determined based on the first global pose, to extract point of interest (POI) information corresponding to the first global pose from target data, and to add the POI information to the first in-vehicle navigation route to obtain the first extended navigation route, wherein the target data includes navigation events and / or an in-vehicle navigation map, the POI information includes road attribute information related to the prediction of the furthest reachable distance, and the POI information corresponding to the first global pose includes the POI information within a predetermined distance range ahead of the first global pose on the first in-vehicle navigation route.

[0113] In one possible implementation, the sensing module 330 is used to acquire input data for a landmark sensing model, and to process the input data based on the landmark sensing model to acquire first landmark information contained in the first road environment image at the target time, wherein the input data includes the first road environment image at the target time and a first local pose at the target time, and the target time is any point in time when the first vehicle is collecting the first road environment image on a first extended navigation route.

[0114] In one possible implementation, as shown in Figure 6, the system is: The system further comprises an image collector 370 for collecting a first road environment image.

[0115] In one possible implementation, the target time is the time when the first road environment image is being collected for the Nth time on the first extended navigation route, and the input data acquired by the sensing module 330 further includes the output result of the landmark sensing model at a time adjacent to the target time, the output result at the time prior to the target time includes the first landmark information contained in the first road environment image collected at the time prior to the target time, and N is a positive integer of 2 or more.

[0116] In accordance with the embodiments of the above method, another embodiment of the present application provides a distance prediction device, which, as shown in Figure 7, includes an acquisition unit 410 and a distance prediction unit 420. The acquisition unit 410 is used to acquire a first global pose of a first vehicle, a first extended navigation route, and first landmark information, wherein the first global pose is the current global pose of the first vehicle, the first extended navigation route includes an extended route of a first in-vehicle navigation route determined based on the first global pose, and the first landmark information includes landmark information contained in a first road environment image collected by the first vehicle in the first global pose. The distance prediction unit 420 is used to process the first global pause, the first extended navigation route, and the first landmark information based on a distance prediction model, and to obtain the furthest achievable distance on each lane in the first landmark information. The acquisition unit 410 is further used to acquire a distance training sample set before processing the first global pose, the first extended navigation route, and the first landmark information based on the distance prediction model, wherein each training sample in the distance training sample set includes a second extended navigation route, a second landmark information, a second global pose, and a truth value for the furthest reachable distance, the second extended navigation route includes an extended route of the second in-vehicle navigation route, the second landmark information includes landmark information contained in a second road environment image collected by a second vehicle on the second extended navigation route, the second global pose includes the global pose when the second vehicle collected the second road environment image, and the truth value for the furthest reachable distance includes the truth value for the furthest reachable distance per lane in the second landmark information. The aforementioned device is The system further includes a training unit 430 for training using the distance training sample set to obtain the distance prediction model.

[0117] In one possible implementation, the acquisition unit 410 includes a first acquisition module, an extraction module, and an additional module. The first acquisition module is used to acquire the target in-vehicle navigation route and the target global pose when the target vehicle is traveling on the target in-vehicle navigation route, where the target extended navigation route includes the first extended navigation route and / or the second extended navigation route, wherein if the target extended navigation route is the first extended navigation route, the target in-vehicle navigation route is the first in-vehicle navigation route, the target vehicle is the first vehicle, and the target global pose is the first global pose; if the target extended navigation route is the second extended navigation route, the target in-vehicle navigation route is the second in-vehicle navigation route, and the target global pose is the second global pose. The extraction module is used to extract points of interest (POI) information corresponding to the target global pose from the target data, the target data includes navigation events and / or in-vehicle navigation maps, the POI information includes road attribute information related to the prediction of the furthest reachable distance, and the POI information corresponding to the target global pose includes the POI information within a predetermined distance range ahead of the target global pose on the target in-vehicle navigation route. The additional module is used to add the POI information to the target in-vehicle navigation route and to obtain the target extended navigation route.

[0118] In one possible implementation, the acquisition unit 410 includes a second acquisition module and a processing module. The second acquisition module is used to acquire input data for a landmark sensing model when the target landmark information includes the first landmark information and / or the second landmark information, the input data includes the target road environment image at the target time and the target local pose at the target time, the target local pose includes an offset amount of the global pose of the target vehicle at the time the target road environment image is collected relative to the global pose at the target start point, the target time is any time when the target vehicle is collecting the target road environment image on the target extended navigation route, if the target landmark information is the first landmark information, the target vehicle is the first vehicle, the target road environment image is the first road environment image, the target extended navigation route is the first extended navigation route, and the target local pose is the first local pose, if the target landmark information is the second landmark information, the target vehicle is the second vehicle, the target road environment image is the second road environment image, the target extended navigation route is the second extended navigation route, and the target local pose is the second local pose, The processing module is used to process the input data based on the landmark detection model and to obtain the target landmark information included in the target road environment image at the target time.

[0119] In one possible implementation, the target time is the time when the target road environment image is being collected for the Nth time on the target extended navigation route, the input data further includes the output result of the landmark sensing model at a time adjacent to the target time, the output result at the time adjacent includes the target landmark information contained in the target road environment image collected at the time adjacent, and N is a positive integer of 2 or more.

[0120] In one possible implementation, the acquisition unit 410 is The processing module further includes a first generation module for generating the landmark sensing model before processing the input data based on the landmark sensing model and obtaining the target landmark information included in the target road environment image at the target time, The first generation module includes an acquisition submodule and a training submodule, The acquisition submodule is used to acquire a landmark training sample set, and each training sample in the landmark training sample set includes a road environment sample image group, and a third local pose corresponding to the road environment sample image for each frame in the road environment sample image group, and a landmark truth value corresponding to the road environment sample image for each frame, and the road environment sample The image group includes a series of consecutive frames of sample images of the road environment. The training submodule is used to train using the landmark training sample set and to acquire a landmark sensing model for sensing and outputting landmark information in the road environment sample image.

[0121] In one possible implementation, the acquisition submodule is used to acquire the corresponding road measurement time-series data for each of the road environment sample image groups awaiting processing, based on the third local pose of the road environment sample image in each frame of the road environment sample image group; to construct a first local map based on the road measurement time-series data; and to acquire the landmark truth values ​​of the road environment sample image for each frame by projecting the first local map onto each of the road environment sample images in the road environment sample image group.

[0122] In one possible implementation, the acquisition unit 410 is A second generation module for generating a second local map including the second extended navigation route based on a continuous video stream, A driving module for driving according to the second extended navigation route in the second local map, A calculation module for calculating the furthest reachable distance on each lane ahead of each of the second global pauses on the second extended navigation route, based on the second local map, The system includes a truth value marking module for obtaining the truth value of the furthest reachable distance for each lane in the second landmark information, by using the calculated furthest reachable distance as the truth value and marking the corresponding second landmark information with truth value.

[0123] The distance prediction device provided by the embodiment of the present invention first trains and acquires a distance prediction model based on a distance training sample set including a second extended navigation route, a second landmark information, a second global pause, and the truth value of the furthest reachable distance for each lane in the second landmark information, and then inputs the first global pause, a first extended navigation route, and the first landmark information into the distance prediction model to predict the furthest reachable distance on each lane in the first landmark information. Since the second global pose, second extended navigation route, and second landmark information required during model training, as well as the first global pose, first extended navigation route, and first landmark information required during model application, do not depend on high-precision maps, the embodiment of the present invention can automatically train a distance prediction model capable of predicting the furthest reachable distance in each lane ahead of the vehicle, while being free from the frequent updates of high-precision maps, and can use this distance prediction model to automatically predict the furthest reachable distance in each lane ahead of the vehicle. This makes it easy to quickly plan routes according to the furthest reachable distance in each lane ahead of the vehicle, and further enables intelligent driving planning control while saving economic and time costs.

[0124] Depending on the embodiment of the above method, another embodiment of the present application provides a training apparatus for a distance prediction model, which, as shown in Figure 8, includes an acquisition unit 510 and a training unit 520. The acquisition unit 510 is used to acquire a distance training sample set, each training sample in the distance training sample set includes a second extended navigation route, second landmark information, a second global pose, and a truth value of the furthest achievable distance, the second extended navigation route includes an extended route of the second in-vehicle navigation route, the second landmark information includes landmark information contained in a second road environment image collected by a second vehicle on the second extended navigation route, and the second global pose includes the global pose when the second vehicle collected the second road environment image. The training unit 520 is used to train using the distance training sample set to acquire a distance prediction model for predicting the furthest distance that can be reached in each lane ahead of any given vehicle.

[0125] In one possible implementation, the acquisition unit 510 includes a first acquisition module, an extraction module, and an additional module. The first acquisition module is used to acquire a second in-vehicle navigation route and a second global pause when the second vehicle is traveling along the second in-vehicle navigation route. The extraction module is used to extract point of interest (POI) information corresponding to the second global pose from target data, the target data includes navigation events and / or in-vehicle navigation maps, the POI information includes road attribute information related to the prediction of the furthest reachable distance, and the POI information corresponding to the second global pose includes the POI information within a predetermined distance range ahead of the second global pose on the second in-vehicle navigation route. The additional module is used to add the POI information to the second in-vehicle navigation route and to obtain the second extended navigation route.

[0126] In one possible implementation, the acquisition unit 510 includes a second acquisition module and a processing module. The second acquisition module is used to acquire input data for a landmark sensing model, the input data including the second road environment image at the target time and the second local pose at the target time, the second local pose including an offset amount from the global pose of the second vehicle at the target start point to the global pose when the second road environment image is being collected, and the target time is any point in time when the second vehicle is collecting the second road environment image on the second extended navigation route. The processing module is used to process the input data based on the landmark detection model and to obtain the second landmark information contained in the second road environment image at the target time.

[0127] In one possible implementation, the target time is the time when the second road environment image is being collected for the Nth time on the second extended navigation route, the input data further includes the output result of the landmark sensing model at a time adjacent to the target time, the output result at the time prior to the target time includes the second landmark information contained in the second road environment image collected at the time prior to the target time, and N is a positive integer of 2 or more.

[0128] In one possible implementation, the acquisition unit 510 is The system further includes a first generation module for generating the landmark detection model, before processing the input data based on the landmark detection model and obtaining the second landmark information contained in the second road environment image at the target time, The first generation module includes an acquisition submodule and a training submodule, The acquisition submodule is used to acquire a landmark training sample set, and each training sample in the landmark training sample set includes a road environment sample image group, and a third local pose corresponding to the road environment sample image for each frame in the road environment sample image group, and a landmark truth value corresponding to the road environment sample image for each frame, and the road environment sample The image group includes a series of consecutive frames of sample images of the road environment. The training submodule is used to train using the landmark training sample set and to acquire a landmark sensing model for sensing and outputting landmark information in the road environment sample image.

[0129] In one possible implementation, the acquisition submodule is used to acquire the corresponding road measurement time-series data for each of the road environment sample image groups awaiting processing, based on the third local pose of the road environment sample image in each frame of the road environment sample image group; to construct a first local map based on the road measurement time-series data; and to acquire the landmark truth values ​​of the road environment sample image for each frame by projecting the first local map onto each of the road environment sample images in the road environment sample image group.

[0130] In one possible implementation, the acquisition unit 510 is A second generation module for generating a second local map including the second extended navigation route based on a continuous video stream, A driving module for driving according to the second extended navigation route in the second local map, A calculation module for calculating the furthest reachable distance on each lane ahead of each of the second global pauses on the second extended navigation route, based on the second local map, The system includes a truth value marking module for obtaining the truth value of the furthest reachable distance for each lane in the second landmark information, by using the calculated furthest reachable distance as the truth value and marking the corresponding second landmark information with truth value.

[0131] The distance prediction model training device provided by the embodiment of the present invention can train and acquire a distance prediction model based on a distance training sample set including a second extended navigation route, second landmark information, second global pose, and truth values ​​of the furthest reachable distance for each lane in the second landmark information. Naturally, since the acquisition of this sample information does not depend on high-precision maps, it is possible to automatically train a distance prediction model capable of predicting the furthest reachable distance in each lane ahead of the vehicle, while being free from the frequent updates of high-precision maps, and to automatically predict the furthest reachable distance in each lane ahead of the vehicle using this distance prediction model. This makes it easier to quickly plan routes according to the furthest reachable distance in each lane ahead of the vehicle, and furthermore, it is possible to realize intelligent driving planning control while saving economic and time costs.

[0132] Based on embodiments of the above-described methods, another embodiment of the present application provides a computer-readable storage medium in which a computer program is stored, and the method of any one of the above embodiments is realized when the program is executed by a processor.

[0133] Based on the embodiments of the above method, another embodiment of the present application provides an electronic device or computer device, as shown in Figure 9, the computer device is One or more processors 610, The processor 610 is coupled to a storage device 620 for storing one or more programs, When the one or more programs are executed by the one or more processors 610, the electronic device or computer device implements the method according to any one of the embodiments described above.

[0134] Based on embodiments of the above-described methods, another embodiment of the present application provides a vehicle comprising the system described in any one of the above embodiments, or the apparatus described in any one of the above embodiments, or the electronic equipment described above.

[0135] The vehicle is equipped with a CPU (Central Processing Unit), a T-Box (Telematics Box, remote information processor), an image collector, and a navigation positioning device. The image collector is used to collect road environment images, the navigation positioning device is used to position the vehicle and obtain its global and local poses, and the CPU acquires road environment images, global poses, and local poses, and also acquires a distance prediction module using an embodiment of the distance prediction model training method described above, and uses an embodiment of the distance prediction method described above to predict the furthest reachable distance on each lane in the current landmark information. The CPU can also transmit road environment images, global poses, and local poses to a server via the T-Box, and the server can acquire a distance prediction module using an embodiment of the distance prediction model training method described above, and use an embodiment of the distance prediction method described above to predict the furthest reachable distance on each lane in the current landmark information.

[0136] Based on the embodiments described above, another embodiment of the present application provides a computer program product that includes instructions, and when the instructions are executed by a computer or processor, the computer or processor performs the method described in any one of the embodiments described above.

[0137] The above-described embodiment of the apparatus corresponds to the embodiment of the method and has the same technical effects as the embodiment of the method; a detailed description thereof should be found in the embodiment of the method. The embodiment of the apparatus is derived from the embodiment of the method, and a detailed description can be found in the part of the embodiment of the method, which will not be repeated here. Those skilled in the art will understand that the drawings are merely schematic diagrams of one embodiment, and that the modules or flows shown in the drawings are not necessarily necessary for carrying out the present application.

[0138] Those skilled in the art will understand that the modules in the apparatus according to the embodiment may be distributed in the apparatus according to the embodiment as described in the embodiment, or they may be arranged in one or more different apparatuses with appropriate modifications. The modules according to the above embodiment may be combined as a single module, or they may be further divided into multiple submodules.

[0139] Finally, it should be noted that the above embodiments are for illustrating the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art can still modify the technical solutions described in the above embodiments or make equivalent substitutions to some of their technical features. These modifications or substitutions should be understood as not departing the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A distance prediction method, Steps include obtaining the first global pose, first extended navigation route, and first landmark information of the first vehicle, The process includes processing the first global pose, the first extended navigation route, and the first landmark information based on a distance prediction model to obtain the furthest distance achievable on each lane in the first landmark information, The first global pose is the current global pose of the first vehicle, the first extended navigation route includes an extended route of the first in-vehicle navigation route determined based on the first global pose, and the first landmark information includes landmark information contained in the first road environment image collected by the first vehicle in the first global pose. The aforementioned distance prediction model, Obtaining a distance training sample set, The distance prediction model is obtained by training using the aforementioned distance training sample set, and the model is obtained by doing so. A distance prediction method characterized in that each training sample in the distance training sample set includes a second extended navigation route, second landmark information, a second global pose, and a truth value for the furthest achievable distance, wherein the second extended navigation route includes an extended route of the second in-vehicle navigation route, the second landmark information includes landmark information contained in a second road environment image collected by a second vehicle on the second extended navigation route, the second global pose includes the global pose when the second vehicle collected the second road environment image, and the truth value for the furthest achievable distance includes the truth value for the furthest achievable distance for each lane in the second landmark information.

2. When the target extended navigation route includes the first extended navigation route and / or the second extended navigation route, the acquisition of the target extended navigation route is: To acquire the target in-vehicle navigation route and the target global pause when the target vehicle is traveling on the said target in-vehicle navigation route, Extracting points of interest (POI) information corresponding to the aforementioned target global pose from the target data, This includes adding the POI information to the target in-vehicle navigation route and obtaining the target extended navigation route, If the target extended navigation route is the first extended navigation route, the target in-vehicle navigation route is the first in-vehicle navigation route, the target vehicle is the first vehicle, and the target global pose is the first global pose; if the target extended navigation route is the second extended navigation route, the target in-vehicle navigation route is the second in-vehicle navigation route, and the target global pose is the second global pose. The distance prediction method according to claim 1, wherein the target data includes navigation events and / or an in-vehicle navigation map, the POI information includes road attribute information related to the prediction of the furthest reachable distance, and the POI information corresponding to the target global pose includes the POI information within a predetermined distance range ahead of the target global pose on the target in-vehicle navigation route.

3. When the target landmark information includes the first landmark information and / or the second landmark information, the acquisition of the target landmark information included in the target road environment image collected by the target vehicle at the target time is: To obtain input data for the landmark detection model, This includes processing the input data based on the landmark detection model and obtaining the target landmark information included in the target road environment image at the target time, The distance prediction method according to claim 1, characterized in that the input data includes the target road environment image at the target time and the target local pose at the target time, the target local pose includes an offset amount of the global pose of the target vehicle at the time the target road environment image is collected relative to the global pose at the target start point, the target time is any time when the target vehicle is collecting the target road environment image on the target extended navigation route, if the target landmark information is the first landmark information, the target vehicle is the first vehicle, the target road environment image is the first road environment image, the target extended navigation route is the first extended navigation route, and the target local pose is the first local pose, and if the target landmark information is the second landmark information, the target vehicle is the second vehicle, the target road environment image is the second road environment image, the target extended navigation route is the second extended navigation route, and the target local pose is the second local pose.

4. The distance prediction method according to claim 3, wherein the target time is the time when the target road environment image is collected for the Nth time on the target extended navigation route, the input data further includes the output result of the landmark sensing model at a time adjacent to the target time, the output result at the time before includes the target landmark information contained in the target road environment image collected at the time before, and N is a positive integer of 2 or more.

5. The generation of the aforementioned landmark sensing model is performed by Obtaining a landmark training sample set, This includes training using the aforementioned landmark training sample set to obtain a landmark sensing model, Each training sample in the landmark training sample set includes a road environment sample image group, a third local pose corresponding to each frame of the road environment sample image in the road environment sample image group, and a landmark truth value corresponding to each frame of the road environment sample image, the road environment sample image group includes road environment sample images of multiple consecutive frames, The distance prediction method according to claim 3, characterized in that the landmark detection model is for detecting and outputting landmark information in the road environment sample image.

6. The acquisition of the aforementioned landmark truth value is performed by, For each of the road environment sample image groups awaiting processing, the corresponding road measurement time-series data for the road environment sample image group is obtained based on the third local pose of the road environment sample image for each frame in the road environment sample image group. A first local map is constructed based on the aforementioned road measurement time-series data, The distance prediction method according to claim 5, characterized in that it includes obtaining the truth values ​​of the landmarks in each frame of the road environment sample image by projecting the first local map onto each of the road environment sample images in the road environment sample image group.

7. Obtaining the truth value of the furthest distance that can be reached is: A second local map is generated based on a continuous video stream, including the second extended navigation route, and the user drives according to the second extended navigation route in the second local map. Based on the second local map, the furthest distance that can be reached on each lane ahead of each of the second global pauses on the second extended navigation route is calculated, A distance prediction method according to any one of claims 1 to 6, characterized in that it includes: using the calculated furthest reachable distance as a truth value, marking the corresponding second landmark information with a truth value, and obtaining the truth value of the furthest reachable distance for each lane in the second landmark information.

8. Steps to obtain a distance training sample set, The steps include: training using the aforementioned distance training sample set to obtain a distance prediction model for predicting the furthest distance that can be reached in each lane ahead of any vehicle; A method for training a distance prediction model, characterized in that each training sample in the distance training sample set includes a second extended navigation route, second landmark information, a second global pose, and a truth value for the furthest achievable distance, the second extended navigation route includes an extended route of a second in-vehicle navigation route, the second landmark information includes landmark information contained in a second road environment image collected by a second vehicle on the second extended navigation route, and the second global pose includes the global pose when the second vehicle collected the second road environment image.

9. A vehicle planning control system comprising a positioning module, an extended route module, a sensing module, a distance prediction module, and a planning control module, The positioning module is used to acquire a first global pose and a first local pose of the first vehicle, the first local pose including an offset amount relative to the global pose at the target starting point of the global pose of the first vehicle when the first road environment image is being collected. The extended route module is used to determine a first extended navigation route based on the first global pause, and the first extended navigation route includes an extended route of the first in-vehicle navigation route. The sensing module is used to sense first landmark information and target information around the first vehicle, wherein the first landmark information includes landmark information included in a first road environment image collected by the first vehicle in the first global pause, and the target information includes at least one of the following: traffic signal information in front of the first vehicle, static object information around the first vehicle, and dynamic object information around the first vehicle. The distance prediction module is used to obtain the furthest distance that can be reached on each lane in the first landmark information, based on the distance prediction method described in any one of claims 1 to 6. A vehicle planning control system characterized in that the planning control module determines the planned trajectory of the first vehicle based on user prediction information, the furthest distance that can be reached on each lane in the first landmark information, the first local pause, and the target information, and is used to control the driving of the first vehicle based on the planned trajectory.

10. A distance prediction device, wherein the distance prediction device includes an acquisition unit and a distance prediction unit. The acquisition unit is used to acquire a first global pose of a first vehicle, a first extended navigation route, and first landmark information, wherein the first global pose is the current global pose of the first vehicle, the first extended navigation route includes an extended route of a first in-vehicle navigation route determined based on the first global pose, and the first landmark information includes landmark information contained in a first road environment image collected by the first vehicle in the first global pose. The distance prediction unit is used to process the first global pose, the first extended navigation route, and the first landmark information based on a distance prediction model, and to obtain the furthest distance that can be reached on each lane in the first landmark information. The acquisition unit is further used to acquire a distance training sample set before processing the first global pose, the first extended navigation route, and the first landmark information based on the distance prediction model, wherein each training sample in the distance training sample set includes a second extended navigation route, a second landmark information, a second global pose, and a truth value for the furthest reachable distance, the second extended navigation route includes an extended route of the second in-vehicle navigation route, the second landmark information includes landmark information contained in a second road environment image collected by a second vehicle on the second extended navigation route, the second global pose includes the global pose when the second vehicle collected the second road environment image, and the truth value for the furthest reachable distance includes the truth value for the furthest reachable distance per lane in the second landmark information. The distance prediction device is, A distance prediction device further comprising a training unit for training using the distance training sample set and obtaining the distance prediction model.

11. The acquisition unit includes a first acquisition module, an extraction module, and an additional module. The first acquisition module is used to acquire the target in-vehicle navigation route and the target global pose when the target vehicle is traveling on the target in-vehicle navigation route, when the target extended navigation route includes the first extended navigation route and / or the second extended navigation route, wherein if the target extended navigation route is the first extended navigation route, the target in-vehicle navigation route is the first in-vehicle navigation route, the target vehicle is the first vehicle, and the target global pose is the first global pose; if the target extended navigation route is the second extended navigation route, the target in-vehicle navigation route is the second in-vehicle navigation route, and the target global pose is the second global pose. The extraction module is used to extract points of interest (POI) information corresponding to the target global pose from the target data, the target data includes navigation events and / or in-vehicle navigation maps, the POI information includes road attribute information related to the prediction of the furthest reachable distance, and the POI information corresponding to the target global pose includes the POI information within a predetermined distance range ahead of the target global pose on the target in-vehicle navigation route. The distance prediction device according to claim 10, characterized in that the additional module is used to add the POI information to the target in-vehicle navigation route and to obtain the target extended navigation route.

12. The acquisition unit includes a second acquisition module and a processing module. The second acquisition module is used to acquire input data for a landmark sensing model when the target landmark information includes the first landmark information and / or the second landmark information, the input data includes a target road environment image at a target time and a target local pose at the target time, the target local pose includes an offset amount of the global pose of the target vehicle at the time the target road environment image is collected relative to the global pose at the target start point, the target time is any time when the target vehicle is collecting the target road environment image on the target extended navigation route, if the target landmark information is the first landmark information, the target vehicle is the first vehicle, the target road environment image is the first road environment image, the target extended navigation route is the first extended navigation route, and the target local pose is the first local pose, if the target landmark information is the second landmark information, the target vehicle is the second vehicle, the target road environment image is the second road environment image, the target extended navigation route is the second extended navigation route, and the target local pose is the second local pose, The distance prediction device according to claim 10, characterized in that the processing module is used to process the input data based on the landmark sensing model and to acquire the target landmark information included in the target road environment image at the target time.

13. The distance prediction device according to claim 12, wherein the target time is the time when the target road environment image is collected for the Nth time on the target extended navigation route, the input data further includes the output result of the landmark sensing model at a time adjacent to the target time, the output result at the time before includes the target landmark information contained in the target road environment image collected at the time before, and N is a positive integer of 2 or more.

14. The aforementioned acquisition unit is The processing module further includes a first generation module for generating the landmark sensing model before processing the input data based on the landmark sensing model and obtaining the target landmark information included in the target road environment image at the target time, The first generation module includes an acquisition submodule and a training submodule, The acquisition submodule is used to acquire a landmark training sample set, each training sample in the landmark training sample set includes a road environment sample image group, a third local pose corresponding to the road environment sample image for each frame in the road environment sample image group, and a landmark truth value corresponding to the road environment sample image for each frame, the road environment sample image group includes road environment sample images for multiple consecutive frames, The distance prediction device according to claim 12, characterized in that the training submodule is used to train using the landmark training sample set and to acquire a landmark sensing model for sensing and outputting landmark information in the road environment sample image.

15. The distance prediction device according to claim 14, wherein the acquisition submodule is used for each of the road environment sample image groups awaiting processing to acquire corresponding road measurement time series data for the road environment sample image group based on the third local pose of the road environment sample image for each frame in the road environment sample image group, construct a first local map based on the road measurement time series data, and acquire the landmark truth values ​​of the road environment sample image for each frame by projecting the first local map onto each of the road environment sample images in the road environment sample image group.

16. The aforementioned acquisition unit is A second generation module for generating a second local map including the second extended navigation route based on a continuous video stream, A driving module for driving according to the second extended navigation route in the second local map, A calculation module for calculating the furthest reachable distance on each lane ahead of each of the second global pauses on the second extended navigation route, based on the second local map, A distance prediction device according to any one of claims 10 to 15, comprising: a truth value marking module for obtaining truth values ​​of the furthest reachable distance for each lane in the second landmark information, using the calculated furthest reachable distance as a truth value; and performing truth value marking on the corresponding second landmark information.

17. A training device for a distance prediction model, wherein the training device for the distance prediction model includes an acquisition unit and a training unit. The acquisition unit is used to acquire a distance training sample set, each training sample in the distance training sample set includes a second extended navigation route, second landmark information, a second global pose, and a truth value of the furthest achievable distance, the second extended navigation route includes an extended route of the second in-vehicle navigation route, the second landmark information includes landmark information contained in a second road environment image collected by a second vehicle on the second extended navigation route, and the second global pose includes the global pose when the second vehicle collected the second road environment image. A distance prediction model training device characterized in that the training unit is used to train using the distance training sample set to acquire a distance prediction model for predicting the furthest distance that can be reached in each lane ahead of any vehicle.

18. A computer-readable storage medium in which a computer program is stored, wherein when the program is executed by a processor, the distance prediction method described in any one of claims 1 to 6 or the distance prediction model training method described in claim 8 is realized.

19. It is an electronic device, One or more processors, The processor includes a memory device for storing one or more programs, The electronic device is characterized in that, when the one or more programs are executed by the one or more processors, the electronic device implements the distance prediction method described in any one of claims 1 to 6 or the distance prediction model training method described in claim 8.

20. A vehicle characterized by comprising a distance prediction device according to any one of claims 10 to 15 or a distance prediction model training device according to claim 17.

Images