Multi-vehicle Cooperative Perception Information Spatiotemporal Unification Method and Device, Storage Medium, and Terminal

Abstract

A multi-vehicle joint perception information spatio-temporal unification method, device, storage medium, and terminal based on a map-aware container. The method includes: extracting sensor information of one or more sensors, where the sensor information includes spatial information in the coordinate system of the sensor observation data and time information at the time of observation; providing a map, and converting the spatial information in the coordinate system of the sensor observation data of each sensor information to spatial information in the map coordinate system based on the map; converting the time information at the time of observation of each sensor information to time information in the driving environment state space; and determining the spatio-temporal reference of the perception result of the vehicle according to the spatial information in the map coordinate system and the time information in the driving environment state space. The present invention can effectively improve the spatio-temporal consistency of the perception result of the fused autonomous vehicle.

Description

Technical Field

[0001] The present invention relates to the field of autonomous vehicle perception, and particularly to a method and device for spatio-temporal unification of multi-vehicle joint perception information based on a map perception container, a storage medium, and a terminal. Background Art

[0002] There are bottleneck problems in single-vehicle environmental perception, such as difficulty in eliminating blind spots and low perception accuracy. V2X (Vehicle To Everything) vehicle networking can achieve communication between vehicles, between vehicles and people, between vehicles and roadside units, and between vehicles and cloud platforms, realizing information sharing and enabling autonomous driving to bid farewell to the era of single-vehicle intelligence.

[0003] In recent years, with the development of vehicle networking technology, the range of information that a single vehicle can obtain has increased significantly. By fusing the perception information of other vehicles in the vehicle network, vehicle perception occlusion and out-of-field-of-view blind spots can be effectively eliminated, and the perception ability of a single vehicle can be improved. In addition, prior information about the road environment can be obtained through an electronic map, which provides a technical basis for further improving the perception ability of a single vehicle by fusing external perception resources.

[0004] However, in existing autonomous vehicle perception technologies based on vehicle networking technology, the local coordinate system of the host vehicle is mostly used as the information fusion benchmark, which depends on the degree of overlap of the fields of view between multiple vehicles and fails to form an effective constraint on the vehicle pose. As a result, the joint perception accuracy based on mass-produced vehicle sensors is difficult to meet the requirements of intelligent connected vehicles, and the consistency of the fusion results between the multiple sensors is insufficient.

[0005] There is an urgent need for a method for spatio-temporal unification of multi-vehicle joint perception information based on a map perception container, which can fuse the information of different sensors in a vehicle networking environment, solve the problems of unified state quantity benchmark and consistent state estimation in multi-vehicle joint perception, and effectively unify the spatio-temporal benchmark. Summary of the Invention

[0006] The technical problem solved by the present invention is to provide a method and device for spatio-temporal unification of multi-vehicle joint perception information based on a map perception container, a storage medium, and a terminal, which can effectively improve the spatio-temporal consistency of the perception results of the fused autonomous vehicle.

[0007] To solve the above technical problems, an embodiment of the present invention provides a method for spatio-temporal unification of multi-vehicle joint perception information based on a map-aware container, which is characterized by including the following steps: extracting sensor information of one or more sensors, the spatial information in the coordinate system of the sensor observation data of the sensors, and the time information of the observation time; providing a map, and converting the spatial information in the coordinate system of the sensor observation data of each sensor information to the spatial information in the map coordinate system based on the map; converting the time information of the observation time of each sensor information to the time information in the driving environment state space; and determining the spatio-temporal reference of the perception result of the vehicle according to the spatial information in the map coordinate system and the time information in the driving environment state space.

[0008] Optionally, the sensor information includes self-detection information for detecting the vehicle itself and external detection information for detecting other objects except the vehicle.

[0009] Optionally, the external detection information includes one or more of the following: connected vehicle detection information for detecting other connected vehicles in the vehicle network; dynamic obstacle information for detecting dynamic obstacles except the other connected vehicles; map detection information for detecting static obstacles and road information on the map; new static obstacle detection information for detecting static obstacles not recorded on the map; and new road detection information for detecting road information not recorded on the map.

[0010] Optionally, converting the spatial information in the coordinate system of the sensor observation data of each sensor information to the spatial information in the map coordinate system based on the map includes: extracting the sensor observation data coordinates of the target in the spatial information in the coordinate system of the sensor observation data of the sensor information; performing a first mapping process on the sensor observation data coordinates of the target to obtain the data coordinates of the target in the observation data coordinate system; and performing a first inverse mapping process on the data coordinates of the target to obtain the map coordinates of the target in the map coordinate system.

[0011] Optionally, performing a first mapping process on the sensor observation data coordinates of the target to obtain the data coordinates of the target in the observation data coordinate system includes: converting the sensor observation data coordinates of the target to the body coordinates in the body coordinate system based on the geographical location of the autonomous vehicle itself; converting the body coordinates of the target to the sensor coordinates in the sensor coordinate system based on the geographical location of the sensor to which the sensor information belongs; and converting the sensor coordinates of the target to the data coordinates in the observation data coordinate system based on the detection parameters of the sensor.

[0012] Optionally, meet one or more of the following: Use the following formula to convert the coordinate of the sensor observation data of the target to the body coordinate in the body coordinate system based on the geographical location of the vehicle itself:

[0013]

[0014] where v is used to represent the body coordinate system, w is used to represent the sensor observation data coordinate system, and x v is used to represent the position of the target in the body coordinate system, is used to represent the transformation matrix from the sensor observation data coordinate system to the body coordinate system, is used to represent the coordinate of the sensor observation data of the target in the set of driving environment space states of the vehicle, k is used to represent the moment when the coordinate of the sensor observation data of the target is obtained, and D is used to represent the set of elements in the driving environment space, including dynamic targets, static targets, and road information;

[0015] Use the following formula to convert the target from the body coordinate system v to the sensor coordinate based on the sensor position coordinate system a:

[0016]

[0017] where v is used to represent the body coordinate system, a is used to represent the sensor coordinate system, and x a is used to represent the coordinate of the target in the sensor coordinate system, is used to represent the transformation matrix from the body coordinate system to the sensor coordinate system, and x v is used to represent the position coordinate of the target in the body coordinate system;

[0018] Use the following formula to convert the target from the sensor coordinate to the data coordinate based on the sensor observation data coordinate system d;

[0019] h k = f a (x a )

[0020] where a is used to represent the sensor coordinate system, d is used to represent the observation data coordinate system, and h k is used to represent the detection data coordinate of the target, f a is used to represent the conversion function from the sensor coordinate system to the observation data coordinate system, and x a is used to represent the data coordinate of the target; In summary, the unified space mapping coordinate conversion process of the target position from the driving environment space to the perception space is expressed as:

[0021]

[0022] Optionally, performing a first inverse mapping process on the data coordinates of the target to obtain the map coordinates of the target in the map coordinate system includes: converting the data coordinates of the target to the sensor coordinates; converting the sensor coordinates of the target to the vehicle body coordinates; and converting the vehicle body coordinates of the target to the map coordinates in the map coordinate system.

[0023] Optionally, satisfying one or more of the following: using the following formula to convert the data coordinates of the target to the sensor coordinates:

[0024]

[0025] where a is used to represent the sensor coordinate system, d is used to represent the observation data coordinate system, h k is used to represent the detection data coordinates of the target, is used to represent the conversion function from the observation data coordinate system to the sensor coordinate system, x a is used to represent the data coordinates of the target; using the following formula to convert the sensor coordinates of the target to the vehicle body coordinates:

[0026]

[0027] where v is used to represent the vehicle body coordinate system, a is used to represent the sensor coordinate system, x a is used to represent the sensor coordinates of the target, is used to represent the transformation matrix from the sensor coordinate system to the vehicle body coordinate system, x v is used to represent the vehicle body coordinates of the target;

[0028] Using the following formula to convert the vehicle body coordinates of the target to the map coordinates in the map coordinate system:

[0029]

[0030] where v is used to represent the vehicle body coordinate system, m is used to represent the map coordinate system, x m is used to represent the map coordinates of the target, is used to represent the transformation matrix from the vehicle body coordinate system to the map coordinate system, x v is used to represent the vehicle body coordinates of the target.

[0031] Optionally, converting the time information of the observation acquisition moment of each sensor information to the time information of the driving environment state space includes: extracting the sensor information of the target at the observation acquisition moment; performing a second mapping process on the sensor information of the target using the following formula to predict the sensor information of the target at the detection moment, so as to obtain the sensor information of the target at the detection moment:

[0032]

[0033]

[0034] where, π t () is used to represent the unified time mapping of the vehicle, is used to represent the set of time of existing state quantities in the driving environment space, is used to represent the set of time corresponding to the state quantity at the predicted observation moment, and → is used to represent the mapping relationship of the set of time corresponding to the state quantity;

[0035] is used to represent the sensor information of the target at the detection moment, τ:T is used to represent the duration of the observation acquisition moment, is used to represent the sensor information of the target at τ:T, D is used to represent the set of elements in the driving environment space, including dynamic targets, static targets and road information, is used to represent the data sample based on τ: The fitting of the data obtained at time k.

[0036] Optionally, determining the spatio-temporal reference of the vehicle's perception result according to the spatial information in the map coordinate system and the time information of the driving environment state space includes: determining the converted sensor information of each target at the detection moment according to the spatial information of each target in the map coordinate system and the time information of the driving environment state space; determining the converted sensor information of each target at the same moment according to the converted sensor information of each target at the detection moment; predicting the perception result of the vehicle at the current moment or a preset future moment according to the converted sensor information at the same moment.

[0037] Optionally, determining the converted sensor information of each target at the same moment according to the converted sensor information of each target at the detection moment includes: performing a second inverse mapping process on the converted sensor information of the target at the detection moment using the following formula to obtain the converted sensor information of the target at the same moment:

[0038]

[0039] Use the following formula to determine the converted sensor information of each target at the same moment:

[0040]

[0041] where, is used to represent the inverse mapping of the unified time of the vehicle, is used to represent the set of times of the existing state quantities in the driving environment space, is used to represent the set of times corresponding to the state quantities at the predicted observation moment, and → is used to represent the mapping relationship of the set of times corresponding to the state quantities;

[0042] is used to represent the converted sensor information of the i-th target at the same moment;

[0043] is used to represent the data set of the converted sensor information of N targets at the same moment.

[0044] Optionally, according to the converted sensor information at the same moment, estimating the perception result of the vehicle at the current moment or a future preset moment includes: using the maximum likelihood estimation algorithm to estimate the predicted perception result of the vehicle in a future preset moment or a preset duration range.

[0045] Optionally, using the following formula to estimate the predicted perception result of the vehicle in a future preset moment or a preset duration range includes:

[0046]

[0047] where, is used to represent the predicted perception result in a future preset moment or a preset duration range, X D is used to represent the perception result at the current moment, is used to represent the data set of the sensor information of N targets at the same moment, is the conditional probability under X D under conditions.

[0048] To solve the above technical problems, an embodiment of the present invention provides a multi-vehicle joint perception information spatio-temporal unification device based on a map perception container, which is characterized by including: a sensor information extraction module for extracting sensor information of one or more sensors, where the sensor information includes spatial information in the coordinate system of sensor observation data and time information at the time of observation; a spatial information conversion module for providing a map and converting the spatial information in the coordinate system of sensor observation data of each sensor information to spatial information in the map coordinate system based on the map; a time information conversion module for converting the time information at the time of observation of each sensor information to time information in the driving environment state space; and a perception result determination module for determining the spatio-temporal reference of the perception result of the vehicle according to the spatial information in the map coordinate system and the time information in the driving environment state space.

[0049] To solve the above technical problems, an embodiment of the present invention provides a storage medium with a computer program stored thereon, and when the computer program is run by a processor, it executes the steps of the above multi-vehicle joint perception information spatio-temporal unification method based on a map perception container.

[0050] To solve the above technical problems, an embodiment of the present invention provides a terminal, including a memory and a processor, with a computer program stored on the memory that can run on the processor, and when the processor runs the computer program, it executes the steps of the above multi-vehicle joint perception information spatio-temporal unification method based on a map perception container.

[0051] Compared with the prior art, the technical solution of the embodiment of the present invention has the following beneficial effects:

[0052] In the embodiment of the present invention, by setting to convert the spatial information in the coordinate system of sensor observation data of each sensor information to spatial information in the map coordinate system based on the map, the actual positions of the targets detected by sensors installed at different positions on the vehicle can be unified, and by setting the step of converting the time information at the time of observation of each sensor information to time information in the driving environment state space, the actual detection times of the data detected by sensors with uneven data processing speeds can be unified. Furthermore, based on the unified time and space reference, the spatio-temporal consistency of the perception result of the vehicle after fusion can be effectively improved.

[0053] Further, in the embodiment of the present invention, by setting a first mapping process on the coordinate of the sensor observation data of the target to obtain the data coordinate of the target in the observation data coordinate system, and then performing a first inverse mapping process on the data coordinate of the target to obtain the map coordinate of the target in the map coordinate system, the actual positions of the targets detected by sensors installed at different positions of the vehicle can be unified through the first mapping process, and then the positions of the targets can be mapped onto the map through the first inverse mapping process, so as to achieve the spatial consistency of the same target detected by multiple sensors on the map.

[0054] Further, in the embodiment of the present invention, by setting a second mapping process on the sensor information of the target extracted at the observation acquisition moment, the time information of the observation acquisition moment of each sensor information can be converted into the time information of the driving environment state space, so as to unify the actual detection time of the data detected by sensors with uneven processing speeds, and achieve the time consistency of the actual detection times of multiple sensors.

[0055] Further, in the embodiment of the present invention, by setting a second inverse mapping process on the converted sensor information of the target at the detection moment, the converted sensor information of each target at the same moment can be determined, the sensor information of multiple targets can be time-calibrated, the time consistency of multiple targets can be achieved, and the accuracy of the perception result of the vehicle at the current moment or a future preset moment can be further improved.

[0056] Further, by adopting the maximum likelihood estimation algorithm to estimate the predicted perception result of the vehicle in a future preset moment or a preset duration range, the accuracy of the prediction result can be effectively improved. Description of the Drawings

[0057] Figure 1 is a flowchart of a multi-vehicle joint perception information spatio-temporal unification method based on a map perception container in the embodiment of the present invention;

[0058] Figure 2 is a schematic diagram of the working scenario of a first mapping process in the embodiment of the present invention;

[0059] Figure 3 is a schematic diagram of the working scenario of a second inverse mapping process in the embodiment of the present invention;

[0060] Figure 4 is a schematic diagram of the processing path of a first inverse mapping process and a second inverse mapping process in the embodiment of the present invention;

[0061] Figure 5 is a schematic diagram of the structure of a multi-vehicle joint perception information spatio-temporal unification device based on a map perception container in the embodiment of the present invention;

[0062] Figure 6 It is a schematic structural diagram of a perception system of a multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container in an embodiment of the present invention. Detailed implementation manners

[0063] As mentioned above, in the prior art, by fusing the perception information of other vehicles in the vehicle network, the vehicle perception occlusion and the ultra-field-of-view blind area can be effectively eliminated, and the perception ability of a single vehicle can be improved. However, in the existing perception technology of autonomous vehicles based on vehicle network technology, the local coordinate system of the host vehicle is mostly used as the information fusion benchmark, which depends on the degree of overlap of the fields of view between multiple vehicles, and fails to form an effective constraint on the vehicle pose, resulting in the difficulty for the joint perception accuracy based on the sensors of mass-produced vehicle models to meet the requirements of intelligent networked vehicles, and the lack of consistency in the fusion results between multiple sensors.

[0064] The inventors of the present invention have found through research that in the prior art, since the data comes from multiple sensors, the multiple sensors are often installed at different positions on the vehicle, resulting in a large error in the true position of the detected target; and the processing speeds of different types of sensors are uneven, resulting in different time differences between the observation acquisition time and the detection time, and the data results obtained at the same moment often correspond to different detection times. Due to the lack of a unified time and space benchmark, the consistency of the fusion results is insufficient.

[0065] In an embodiment of the present invention, by setting steps to convert the spatial information in the sensor observation data coordinate system of each sensor information to the spatial information in the map coordinate system based on the map, the actual positions of the targets detected by the sensors installed at different positions on the vehicle can be unified, and by setting steps to convert the time information of the observation acquisition time of each sensor information to the time information of the driving environment state space, the actual detection times of the data detected by the sensors with uneven processing speeds can be unified. Furthermore, based on the unified time and space benchmark, the spatio-temporal consistency of the perception results of the fused autonomous vehicle can be effectively improved.

[0066] To make the above objects, features, and beneficial effects of the present invention more obvious and understandable, the following detailed description of the specific embodiments of the present invention will be given with reference to the accompanying drawings.

[0067] Referring to Figure 1 , Figure 1 It is a flowchart of a multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container. The multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container may include steps S11 to S14:

[0068] Step S11: Extract the sensor information of one or more sensors, the spatial information in the sensor observation data coordinate system, and the time information at the time of observation.

[0069] Step S12: Provide a map, and convert the spatial information in the sensor observation data coordinate system of each sensor information to the spatial information in the map coordinate system based on the map.

[0070] Step S13: Convert the time information at the time of observation of each sensor information to the time information in the driving environment state space.

[0071] Step S14: Determine the spatio-temporal reference of the perception result of the vehicle according to the spatial information in the map coordinate system and the time information in the driving environment state space.

[0072] In the specific implementation of step S11, one or more sensors can be set on the autonomous vehicle, and the sensor information extracted from the sensors can be the measurement data of one or more targets.

[0073] Specifically, the sensor information can include the spatial information in the sensor observation data coordinate system and the time information at the time of observation. Among them, the spatial information can include the position information of the target detected by the sensor for the target, and the time information can include the time when the sensor obtains the measurement data after data processing.

[0074] It should be noted that since the spatio-temporal unification involved in this application is completed under the high-precision map container, and the high-precision map records the target pose in the geographic coordinate system, so it is converted from the sensor observation data coordinate system to the sensor position coordinate system, then to the vehicle body coordinate system, and finally to the map coordinate system.

[0075] Further, the sensor information can include the self-detection information for detecting the vehicle itself and the external detection information for detecting other objects except the vehicle.

[0076] Specifically, the sensor information can include the integrated navigation observation of the vehicle's own state and the external sensor observation of other targets. The objects of the external observation can be other intelligent connected vehicles, or dynamic obstacles, or the targets included or to be updated in the map data.

[0077] Furthermore, in the step of detecting the vehicle itself to obtain the self-detection information, the observation data generated by the combined positioning system of the i-th connected vehicle on its own vehicle state at time k i can be described as The set of the combined positioning data of the i-th connected vehicle at time is Then, the combined positioning observations directly observing the vehicle's own state in the perception space can be modeled as:

[0078]

[0079] Among them, the perception space is used to represent the environmental space where the sensor can detect the target.

[0080] Furthermore, the external detection information may include one or more of the following: connected vehicle detection information for detecting other connected vehicles in the vehicle network; dynamic obstacle information for detecting dynamic obstacles other than the other connected vehicles; map detection information for detecting static obstacles and road information on the map; new static obstacle detection information for detecting static obstacles not recorded on the map; new road detection information for detecting road information not recorded on the map.

[0081] Specifically, data of external target sensors of other vehicles are obtained through the vehicle network to make up for the deficiency of the single-vehicle perception ability. Considering the actual scenario, in addition to the observations of dynamic obstacles and static non-map targets in the on-vehicle external target sensor data, it also includes the observations of other connected vehicles, positioning features, road information, etc.

[0082] Furthermore, in the step of detecting other connected vehicles in the vehicle network to obtain the connected vehicle detection information, the observation of other vehicles by the connected vehicle can be called vehicle-to-vehicle observation. Let v be the index of the static non-map target. For the observation data of the i-th connected vehicle at time k i of another connected vehicle v, the corresponding time set is The corresponding set is defined as:

[0083]

[0084] In the step of detecting dynamic obstacles other than the other connected vehicles to obtain the dynamic obstacle information, it can be assumed that the sensor of a connected vehicle i makes an observation of a dynamic obstacle o at time k i and its data description is The corresponding time set is Then, the vehicle-obstacle observation set is:

[0085]

[0086] In the step of detecting static obstacles and road information on the map to obtain the map detection information, the observation of the positioning feature by the connected vehicle can be called vehicle-feature observation. Let f be the index of the static non-map target. Observation data of the positioning feature f for the i-th connected vehicle at time k i The set of times corresponding to this observation The corresponding set is defined as:

[0087]

[0088] In the step of detecting static obstacles not recorded in the map to obtain new static obstacle detection information, the observation of static non-map targets can be called vehicle-static non-map target observation. Denote s as the index of the static non-map target. Observation data of the static non-map target s for the i-th connected vehicle at time k i The set of times corresponding to this observation Then the corresponding observation set is defined as:

[0089]

[0090] In the step of detecting road information not recorded in the map to obtain new road detection information, the observation of road information by the connected vehicle is called vehicle-road information observation. Denote r as the index of the road information. Observation data of the road information r for the i-th connected vehicle at time k i The set of times corresponding to this observation The corresponding set is defined as:

[0091]

[0092] Based on the above information, the sensor observations of external targets in the perception space can be modeled as:

[0093] Z V ={Z V-O ,Z V-S ,Z V-V ,Z V-F ,Z V-R}

[0094] It can be understood that in the embodiments of the present invention, the detection data obtained by the sensor for detecting the target is based on the spatial information (such as geographical location) in the coordinate system of the sensor observation data and the time point of the observation time. There is a problem of inconsistency in space and time for different sensors.

[0095] In the specific implementation of step S12, a map is provided, and the spatial information in the coordinate system of the sensor observation data of each sensor information is converted into the spatial information in the map coordinate system based on the map.

[0096] Further, the step of converting the spatial information in the sensor observation data coordinate system of each sensor information to the spatial information in the map coordinate system based on the map may include: extracting the sensor observation data coordinates of the target from the spatial information in the sensor observation data coordinate system of the sensor information; performing a first mapping process on the sensor observation data coordinates of the target to obtain the data coordinates of the target in the observation data coordinate system; and performing a first inverse mapping process on the data coordinates of the target to obtain the map coordinates of the target in the map coordinate system.

[0097] In an embodiment of the present invention, by setting a first mapping process on the sensor observation data coordinates of the target to obtain the data coordinates of the target in the observation data coordinate system, and then performing a first inverse mapping process on the data coordinates of the target to obtain the map coordinates of the target in the map coordinate system, the actual positions of the targets detected by sensors installed at different positions of the vehicle can be unified through the first mapping process, and then the positions of the targets can be mapped onto the map through the first inverse mapping process, thereby achieving the spatial consistency of the same target detected by multiple sensors on the map.

[0098] Furthermore, the step of performing a first mapping process on the sensor observation data coordinates of the target to obtain the data coordinates of the target in the observation data coordinate system may include: converting the sensor observation data coordinates of the target to the body coordinates in the body coordinate system based on the geographical location of the autonomous driving vehicle itself; converting the body coordinates of the target to the sensor coordinates in the sensor coordinate system based on the geographical location of the sensor to which the sensor information belongs; and converting the sensor coordinates of the target to the data coordinates in the observation data coordinate system based on the detection parameters of the sensor.

[0099] Refer to Figure 2 , Figure 2 which is a schematic diagram of the working scenario of a first mapping process in an embodiment of the present invention.

[0100] Assume that at time k, there is a target M in the driving environment space, and its position coordinates in the sensor observation data coordinate system are m w , and this coordinate value exists in the set of states of the intelligent vehicle driving environment space, that is used to represent the sensor observation data coordinates of the target.

[0101] The following formula can be used to convert the sensor observation data coordinates of the target to the body coordinates in the body coordinate system based on the geographical location of the autonomous driving vehicle itself:

[0102]

[0103] Among them, v is used to represent the vehicle body coordinate system, w is used to represent the sensor observation data coordinate system, and x v is used to represent the vehicle body coordinates of the target, is used to represent the transformation matrix from the sensor observation data coordinate system to the vehicle body coordinate system, is used to represent the sensor observation data coordinates of the target.

[0104] Furthermore, the target can be transformed from the vehicle body coordinate system v to the sensor a coordinate system.

[0105] The following formula can be used to transform the vehicle body coordinates of the target to the sensor coordinates in the sensor coordinate system based on the geographical location of the sensor to which the sensor information belongs:

[0106]

[0107] Among them, v is used to represent the vehicle body coordinate system, a is used to represent the sensor coordinate system, and x a is used to represent the sensor coordinates of the target, is used to represent the transformation matrix from the vehicle body coordinate system to the sensor coordinate system, and x v is used to represent the vehicle body coordinates of the target.

[0108] Furthermore, according to the sensor model and internal parameters, it can be transformed from the sensor coordinate system to the sensor observation data coordinate system (also called the data coordinate system). For example, it can be the coordinates in the image captured by an image sensor, such as Figure 2 shown as (ρ, θ).

[0109] The following formula can be used to transform the sensor coordinates of the target to the data coordinates in the observation data coordinate system based on the detection parameters of the sensor;

[0110] h k = f a (x a )

[0111] Among them, a is used to represent the sensor coordinate system, d is used to represent the observation data coordinate system, and h k is used to represent the detection data coordinates of the target, and f a is used to represent the transformation function from the sensor coordinate system to the observation data coordinate system, and x a is used to represent the data coordinates of the target.

[0112] It should be noted that Figure 2The shown v←w is used to represent the conversion of the sensor observation data coordinates of the target to the body coordinates in the body coordinate system based on the geographical location of the autonomous vehicle itself, a←v is used to represent the conversion of the body coordinates of the target to the sensor coordinates in the sensor coordinate system based on the geographical location of the sensor to which the sensor information belongs, and d←a is used to represent the conversion of the sensor coordinates of the target to the data coordinates in the observation data coordinate system based on the detection parameters of the sensor.

[0113] Combining the above processes, the unified space mapping coordinate conversion process of the target position from the driving environment space to the perception space can be expressed as:

[0114]

[0115] Among them, f a can be determined by the sensor model and internal parameters, can be determined by the installation position of the sensor on the vehicle body. Generally, it is considered that both are constant and are determined by the calibration of the internal and external parameters of the sensor. Therefore, the generation of the predicted value is only related to the system state quantity in the driving environment space.

[0116] Furthermore, the steps of performing a first inverse mapping process on the data coordinates of the target to obtain the map coordinates of the target in the map coordinate system may include: converting the data coordinates of the target to the sensor coordinates; converting the sensor coordinates of the target to the body coordinates; and converting the body coordinates of the target to the map coordinates in the map coordinate system.

[0117] Continue to refer to Figure 2 , assuming there is a sensor a, the coordinates of its observation of a certain target in the sensor data coordinate system (i.e., data coordinates) are h k .

[0118] The following formula can be used to convert the data coordinates of the target to the sensor coordinates:

[0119]

[0120] Among them, a is used to represent the sensor coordinate system, d is used to represent the observation data coordinate system, h k is used to represent the detection data coordinates of the target, is used to represent the conversion function from the observation data coordinate system to the sensor coordinate system, and x a is used to represent the data coordinates of the target;

[0121] Specifically, for 3D sensors such as lidar and stereo vision, this process generally involves the conversion from polar coordinates to Cartesian coordinates; for sensors with data dimensionality reduction such as monocular cameras, depth estimation is generally required, and then the 3D position information of the target is restored by the sensor model.

[0122] Furthermore, the target can be transformed into the vehicle body coordinate system.

[0123] The following formula can be used to convert the sensor coordinates of the target to the vehicle body coordinates:

[0124]

[0125] where v is used to represent the vehicle body coordinate system, a is used to represent the sensor coordinate system, x a is used to represent the sensor coordinates of the target, is used to represent the transformation matrix from the sensor coordinate system to the vehicle body coordinate system, x v is used to represent the vehicle body coordinates of the target;

[0126] Furthermore, the target can be transformed from the vehicle body coordinate system to the map coordinate system.

[0127] The following formula is used to convert the vehicle body coordinates of the target to the map coordinates in the map coordinate system:

[0128]

[0129] where v is used to represent the vehicle body coordinate system, m is used to represent the map coordinate system, x m is used to represent the map coordinates of the target, is used to represent the transformation matrix from the vehicle body coordinate system to the map coordinate system, x v is used to represent the vehicle body coordinates of the target.

[0130] It should be noted that a←d in the above content is used to represent converting the data coordinates of the target to the sensor coordinates, v←a is used to represent converting the sensor coordinates of the target to the vehicle body coordinates, and m←v is used to represent converting the vehicle body coordinates of the target to the map coordinates in the map coordinate system.

[0131] In the embodiments of the present invention, through the first mapping process and the first inverse mapping process, the position of the target is mapped onto the map, thereby achieving the spatial consistency of the same target detected by multiple sensors on the map.

[0132] In the specific implementation of step S13, the time information of the observation acquisition time of each sensor information is converted into the time information of the driving environment state space.

[0133] Further, the step of converting the time information of the observation acquisition time of each sensor information to the time information of the driving environment state space may include: extracting the sensor information of the target at the observation acquisition time; performing a second mapping process on the sensor information of the target by using the following formula, and predicting the sensor information of the target at the observation acquisition time to obtain the sensor information of the target at the detection time:

[0134]

[0135]

[0136] where, π t () is used to represent the unified time mapping of the vehicle, is used to represent the set of existing state quantity times in the driving environment space, is used to represent the set of times corresponding to the state quantity at the predicted observation time, and → is used to represent the mapping relationship of the set of times corresponding to the state quantity;

[0137] is used to represent the sensor information of the target at the detection time, τ:T is used to represent the duration of the observation acquisition time, is used to represent the sensor information of the target at τ:T, and D is used to represent the set of elements in the driving environment space, including dynamic targets, static targets, and road information, is used to represent the data sample based on τ: The fitting of the data obtained at time k.

[0138] where, π t () is used to represent the unified time mapping of the intelligent vehicle, that is, based on a certain motion hypothesis, according to the state of the joint perception system at the state time τ∶T Predict the state at time k of the observation time process.

[0139] D is used to represent the set of elements in the driving environment space, which can be composed of dynamic targets, static targets, and road information, and the mathematical definition is: D = {H, Z, R}.

[0140] In the embodiment of the present invention, by setting a second mapping process for the sensor information of the target extracted at the observation acquisition time, the time information of the observation acquisition time of each sensor information can be converted to the time information of the driving environment state space, so as to unify the actual detection time of the data detected by the sensors with uneven processing speeds, and achieve the time consistency of the actual detection times of multiple sensors.

[0141] In the specific implementation of step S14, based on the spatial information in the map coordinate system and the time information in the driving environment state space, determine the spatio-temporal reference of the perception result of the vehicle.

[0142] Further, the step of determining the spatio-temporal reference of the perception result of the vehicle according to the spatial information in the map coordinate system and the time information in the driving environment state space may include: determining the converted sensor information of each target at the detection moment according to the spatial information of each target in the map coordinate system and the time information in the driving environment state space; determining the converted sensor information of each target at the same moment according to the converted sensor information of each target at the detection moment; predicting the perception result of the vehicle at the current moment or a preset future moment according to the converted sensor information at the same moment.

[0143] Wherein, the same moment may be a preset moment determined based on the current moment or the detection moment.

[0144] Specifically, the converted sensor information of each target at the detection moment includes the sensor information after the first mapping process, the first inverse mapping process, and the second mapping process.

[0145] Furthermore, the step of determining the converted sensor information of each target at the same moment according to the converted sensor information of each target at the detection moment may include: performing a second inverse mapping process on the converted sensor information of the target at the detection moment by using the following formula to obtain the converted sensor information of the target at the same moment:

[0146]

[0147] Use the following formula to determine the converted sensor information of each target at the same moment:

[0148]

[0149] Wherein, Used to represent the unified time inverse mapping of the vehicle, Used to represent the set of time of existing state quantities in the driving environment space, Used to represent the set of time corresponding to the state quantity at the predicted observation moment, → is used to represent the mapping relationship of the set of time corresponding to the state quantity; wherein, the For example, it may be the process of transforming the target from the moment of observation generation to the moment of the state quantity in the driving environment space.

[0150] It is used to represent the transformed sensor information of the i-th target at the same moment, that is, the observation value of the target in the sensor observation data coordinate system and the synchronized time system; i ranges from 1 to N, representing N sensors; the superscript I indicates that it is represented in the sensor observation data coordinate system and the synchronized time system after spatio-temporal synchronization mapping.

[0151] A data set used to represent the transformed sensor information of N targets at the same moment.

[0152] Refer to Figure 3 , Figure 3 is a schematic diagram of the working scenario of a second inverse mapping process in an embodiment of the present invention. During the duration of τ: T, the transformed sensor information of the target at the same moment at different detection times (such as τ: T)

[0153] In an embodiment of the present invention, by setting a second inverse mapping process for the transformed sensor information of the target at the detection time, the transformed sensor information of each target at the same moment can be determined, the sensor information of multiple targets can be time-calibrated, the time consistency of multiple targets can be achieved, and the accuracy of the perception result of the vehicle at the current moment or a future preset moment can be further improved.

[0154] Refer to Figure 4 , Figure 4 is a schematic diagram of the processing path of a first inverse mapping process and a second inverse mapping process in an embodiment of the present invention.

[0155] First, based on multiple sensors S 1 , S 2 , S 3 to S 4 determine the information that needs to be subjected to the first inverse mapping process, such as the data coordinates of the target in the embodiment of the present application.

[0156] Then perform a first inverse mapping process on the data coordinates of the target to to obtain the map coordinates of the target.

[0157] Then perform a second inverse mapping process on the sensor information based on the data coordinates of the target to obtain the transformed sensor information of the target at the same moment:

[0158]

[0159]

[0160] Used to represent the transformed sensor information of the i-th target at the same moment, that is, the observation value of the target in the sensor observation data coordinate system and the synchronized time system; i ranges from 1 to N, representing N sensors; the superscript I indicates that it is represented in the sensor observation data coordinate system and the synchronized time system after spatio-temporal synchronization mapping.

[0161] Then based on to determine the transformed sensor information of each target at the same moment:

[0162] Furthermore, according to the transformed sensor information at the same moment, the steps of estimating the perception result of the vehicle at the current moment or a future preset moment may include: adopting the maximum likelihood estimation algorithm to estimate the predicted perception result of the vehicle within a future preset moment or a preset time range.

[0163] Specifically, in the embodiment of the present invention, it is necessary to generate observation quantities based on map, own vehicle, and other vehicle data, that is, the transformed sensor information of each target at the same moment Solve the perception result of the driving vehicle, and the perception result can be represented by the state quantity X D representation.

[0164] Among them, according to the perception method in the embodiment of the present application, the perception information of dynamic and static targets and road traffic information can be determined according to the perception requirements of intelligent vehicles. Among them, dynamic targets are divided into connected vehicles and other dynamic obstacles For dynamic targets, construct a state set X H , the dynamic target state set can be expressed as:

[0165] X H ={X V , X O}

[0166] Traffic signs, lane lines, cones and other static targets in the traffic scene provide reference information for vehicle autonomous positioning. In addition, static targets within the road edge need to be considered in the decision-making system. Static targets in the environment include positioning features already included in the map and static non-map targets That is, the map targets to be updated. For static targets, construct a state set X Z , the static target state set can be expressed as:

[0167] X Z ={X F , X S}

[0168] Set of road information status including road network information lane network information dynamic traffic information and decision - making assistance information For the construction of the status set X of road information R , the set of road information status can be expressed as:

[0169] X R ={X WL ,X w ,X B ,X P}

[0170] In summary, the targets in the driving environment space can be composed of dynamic targets, static targets, and road information, and its mathematical definition is:

[0171]

[0172] The corresponding perception result of the driving vehicle, that is, the status set can be expressed as:

[0173] X D ={X H ,X Z ,X R}

[0174] Furthermore, among various perception inputs of the joint perception system, there are generally observations of the same target from different sources, and there are contradictions between them. In order to obtain a consistent perception result, maximum likelihood estimation is performed. Since the joint perception problem needs to describe the state under a unified map spatio - temporal benchmark, it can be reduced to maximizing the likelihood probability of all system observables with respect to the state quantity. According to the Maximum Likelihood Estimation (MLE) theory, the driving environment space state estimator maximizes the probability of the observables conditional on the state quantity set.

[0175] Furthermore, the following formula can be used to estimate the predicted perception result of the vehicle in the future preset time or preset time range, including:

[0176]

[0177] where is used to represent the predicted perception result in the future preset time or preset time range, X D is used to represent the perception result at the current moment, is used to represent the predicted perception result in the sense of maximum likelihood, A data set for representing sensor information of N targets at the same moment, is the conditional probability under X D below .

[0178] Assuming that the likelihood probabilities of different sensors at different times are independent of each other, the conditional probability can be further written as the joint product of the likelihood probabilities of all observations over a period of time.

[0179]

[0180] Assuming that the noise follows a Gaussian distribution, the maximum likelihood problem can be transformed into the following least squares optimization problem. In this formula, the residual is the difference between the actual observation and the prediction of the system state on the perception result. In the state estimator of the perception container, by optimizing the values of the state variables, the weighted sum of various observation residuals is minimized.

[0181]

[0182] Observation residual:

[0183]

[0184] The above formula gives the mathematical method for the state estimator in the map perception container to estimate the state X D in the driving environment space. It should be noted that this formula adjusts the weights of sensor data with different precisions in the optimization problem through the inverse matrix (CovarianceMatrix) of the covariance matrix of each sensor noise, so as to organically integrate the data of each vehicle-mounted sensor and map sensor in the entire joint perception system, and finally form a consistent perception result.

[0185] In the embodiment of the present invention, by using the maximum likelihood estimation algorithm to estimate the predicted perception result of the vehicle at a future preset moment or within a preset duration range, the accuracy of the prediction result can be effectively improved.

[0186] In the embodiment of the present invention, by setting steps to convert the spatial information in the sensor observation data coordinate system of each sensor information to the spatial information in the map coordinate system based on the map, the actual positions of the targets detected by sensors installed at different positions of the vehicle can be unified, and by setting steps to convert the time information of the observation acquisition moment of each sensor information to the time information of the driving environment state space, the actual detection times of the data detected by sensors with uneven data processing speeds can be unified. Furthermore, based on the unified time and space benchmarks, the spatio-temporal consistency of the perception result of the fused vehicle can be effectively improved.

[0187] Referring to Figure 5 , Figure 5It is a schematic structural diagram of a multi-vehicle joint perception information spatio-temporal unification device based on a map perception container in an embodiment of the present invention. The multi-vehicle joint perception information spatio-temporal unification device based on a map perception container may include:

[0188] A sensor information extraction module 51, configured to extract sensor information of one or more sensors, where the sensor information includes spatial information in the coordinate system of sensor observation data and time information at the time of observation;

[0189] A spatial information conversion module 52, configured to provide a map and convert the spatial information in the coordinate system of sensor observation data of each sensor information to spatial information in the map coordinate system based on the map;

[0190] A time information conversion module 53, configured to convert the time information at the time of observation of each sensor information to time information in the driving environment state space;

[0191] A perception result determination module 54, configured to determine the spatio-temporal reference of the perception result of the vehicle according to the spatial information in the map coordinate system and the time information in the driving environment state space.

[0192] For the principle, specific implementation and beneficial effects of the multi-vehicle joint perception information spatio-temporal unification device based on a map perception container, please refer to the relevant description of the multi-vehicle joint perception information spatio-temporal unification method described above, which will not be elaborated here.

[0193] Refer to Figure 6 , Figure 6 It is a schematic structural diagram of a perception system of a multi-vehicle joint perception information spatio-temporal unification method in an embodiment of the present invention.

[0194] Specifically, based on the sensor information determined by multiple sensors S 1 , S 2 , S 3 to S 4 , using an inverse mapper T I , perform a first mapping process, a first inverse mapping process, a second mapping process, and a second inverse mapping process to determine a data set of the converted sensor information of each target at the same moment based on the multi-vehicle joint perception information spatio-temporal unification method based on a map perception container In other words, the is the spatio-temporal consistency expression result of different types of sensor data in the map coordinate system.

[0195] Collect the map information of the map, denoted as D M , and then input D M together with into DC .

[0196] Among them, D C is a driving environment space constructor, which allocates state variables in the driving environment space according to the expression results of multi-source sensors in the map coordinate system; M E is a driving environment space state estimator, which realizes the estimation of state variables in the driving environment space based on multi-source heterogeneous and asynchronous redundant data

[0197] In an embodiment of the present invention, a storage medium is further provided, on which a computer program is stored. When the computer program is run by a processor, the steps of the above method are executed. The storage medium may be a computer-readable storage medium, for example, it may include a non-volatile memory or a non-transitory memory, and may also include an optical disc, a mechanical hard disk, a solid-state drive, etc.

[0198] Specifically, in an embodiment of the present invention, the processor may be a central processing unit (CPU for short), and this processor may also be other general-purpose processors, digital signal processors (DSP for short), application specific integrated circuits (ASIC for short), field programmable gate arrays (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc.

[0199] It should also be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink dynamic random access memory (SLDRAM), and direct rambus random access memory (DR RAM).

[0200] In an embodiment of the present invention, a terminal is further provided, including a memory and a processor. A computer program capable of running on the processor is stored on the memory. When the processor runs the computer program, it executes the steps of the above method. The terminal includes, but is not limited to, a vehicle, a vehicle center control device, a terminal device externally connected to or integrated in the vehicle, a mobile phone, a computer, a tablet computer, and other terminal devices.

[0201] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention should be subject to the scope defined by the claims.

Claims

1. A multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container, characterized in that, it includes the following steps: Extract sensor information of one or more sensors, where the sensor information includes spatial information in the coordinate system of sensor observation data and time information at the time of observation; Provide a map, and convert the spatial information in the coordinate system of sensor observation data of each sensor information to spatial information in the map coordinate system based on the map; Convert the time information at the time of observation of each sensor information to the time information in the driving environment state space; Determine the spatio-temporal reference of the perception result of the vehicle according to the spatial information in the map coordinate system and the time information in the driving environment state space; The conversion of the spatial information in the coordinate system of sensor observation data of each sensor information to the spatial information in the map coordinate system based on the map includes: Extract the sensor observation data coordinates of the target in the spatial information in the coordinate system of sensor observation data of the sensor information; Perform a first mapping process on the sensor observation data coordinates of the target to obtain the data coordinates of the target in the observation data coordinate system; Perform a first inverse mapping process on the data coordinates of the target to obtain the map coordinates of the target in the map coordinate system; During the first mapping process, the unified space mapping coordinate conversion process of the target position from the driving environment space to the perception space is expressed as: Among them, h k is used to represent the detection data coordinates of the target, a is used to represent the sensor coordinate system, v is used to represent the vehicle body coordinate system, w is used to represent the sensor observation data coordinate system, and f a is used to represent the conversion function from the sensor coordinate system to the observation data coordinate system, is used to represent the conversion matrix from the vehicle body coordinate system to the sensor coordinate system, is used to represent the conversion matrix from the sensor observation data coordinate system to the vehicle body coordinate system, is used to represent the sensor observation data coordinates of the target under the set of driving environment space states of the vehicle, k is used to represent the moment of obtaining the sensor observation data coordinates of the target, D is used to represent the set of elements in the driving environment space, including dynamic targets, static targets and road information, and the perception space is used to represent the environmental space where the sensor can detect the target.

2. The multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container according to claim 1, characterized in that, The sensor information includes self-detection information for detecting the vehicle itself and external detection information for detecting other objects except the vehicle.

3. The multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container according to claim 2, characterized in that, The external detection information includes one or more of the following: Networked vehicle detection information for detecting other networked vehicles in the vehicle network; Dynamic obstacle information for detecting dynamic obstacles other than the other networked vehicles; Map detection information for detecting static obstacles and road information on the map; New static obstacle detection information for detecting static obstacles not recorded on the map; New road detection information for detecting road information not recorded on the map.

4. The multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container according to claim 1, characterized in that, Performing a first mapping process on the sensor observation data coordinates of the target to obtain the data coordinates of the target in the observation data coordinate system includes: Converting the sensor observation data coordinates of the target to the body coordinates in the body coordinate system based on the geographical location of the vehicle itself; Converting the body coordinates of the target to the sensor coordinates in the sensor coordinate system based on the geographical location of the sensor to which the sensor information belongs; Converting the sensor coordinates of the target to the data coordinates in the observation data coordinate system based on the detection parameters of the sensor.

5. The multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container according to claim 4, characterized in that, the following formula is used to convert the sensor observation data coordinates of the target to the vehicle body coordinates in the vehicle body coordinate system based on the geographical location of the vehicle itself: where x v is used to represent the position of the target in the vehicle body coordinate system; the following formula is used to convert the target from the vehicle body coordinate system v to the sensor coordinates in the sensor position coordinate system a; where x a is used to represent the coordinates of the target in the sensor coordinate system, and x v is used to represent the position coordinates of the target in the vehicle body coordinate system; the following formula is used to convert the target from the sensor coordinates to the data coordinates in the sensor observation data coordinate system d; h k = f a (x a ) where x a is used to represent the data coordinates of the target.

6. The multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container according to claim 4, characterized in that, performing a first inverse mapping process on the data coordinates of the target to obtain the map coordinates of the target in the map coordinate system, including: converting the data coordinates of the target to the sensor coordinates; converting the sensor coordinates of the target to the vehicle body coordinates; converting the vehicle body coordinates of the target to the map coordinates in the map coordinate system.

7. The multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container according to claim 6, characterized in that, the following formula is used to convert the data coordinates of the target to the sensor coordinates: Among them, a is used to represent the sensor coordinate system, h k is used to represent the detection data coordinates of the target, is used to represent the conversion function from the observation data coordinate system to the sensor coordinate system, x a is used to represent the data coordinates of the target; the following formula is used to convert the sensor coordinates of the target to the vehicle body coordinates: Among them, v is used to represent the vehicle body coordinate system, a is used to represent the sensor coordinate system, and x a is used to represent the sensor coordinates of the target, is used to represent the transformation matrix from the sensor coordinate system to the vehicle body coordinate system, and x v is used to represent the vehicle body coordinates of the target; the following formula is used to convert the vehicle body coordinates of the target to the map coordinates in the map coordinate system: Among them, v is used to represent the vehicle body coordinate system, m is used to represent the map coordinate system, and x m is used to represent the map coordinates of the target, is used to represent the transformation matrix from the vehicle body coordinate system to the map coordinate system, and x v is used to represent the vehicle body coordinates of the target.

8. The multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container according to claim 1, characterized in that, converting the time information of the observation acquisition moment of each sensor information to the time information of the driving environment state space includes: extracting the sensor information of the target at the observation acquisition moment; using the following formula to perform a second mapping process on the sensor information of the target, and predicting the sensor information of the target at the detection moment to obtain the sensor information of the target at the detection moment: Among them, π t () is used to represent the unified time mapping of the vehicle, and T is used to represent the set of times of existing state quantities in the driving environment space. is used to represent the set of times corresponding to the state quantities at the predicted observation moment, and → is used to represent the mapping relationship of the set of times corresponding to the state quantities. The sensor information used to represent the target at the detection moment, and τ: T is used to represent the duration of the observation acquisition moment. The sensor information used to represent the target in τ: T, and D is used to represent the set of elements in the driving environment space, including dynamic targets, static targets, and road information. For representing data samples based on τ:T The fitting of the data obtained at time k.

9. The multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container according to claim 1, characterized in that, determining the spatio-temporal reference of the perception result of the vehicle according to the spatial information in the map coordinate system and the time information of the driving environment state space includes: determining the converted sensor information of each target at the detection moment according to the spatial information of each target in the map coordinate system and the time information of the driving environment state space; determining the converted sensor information of each target at the same moment according to the converted sensor information of each target at the detection moment; estimating the perception result of the vehicle at the current moment or a preset future moment according to the converted sensor information at the same moment.

10. The multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container according to claim 9, characterized in that, determining the converted sensor information of each target at the same moment according to the converted sensor information of each target at the detection moment includes: The following formula is used to perform a second inverse mapping process on the converted sensor information of the target at the detection moment to obtain the converted sensor information of the target at the same moment: The following formula is used to determine the converted sensor information of each target at the same moment: Among them, used to represent the unified time inverse mapping of the vehicle, used to represent the set of times of the existing state quantities in the driving environment space, used to represent the set of times corresponding to the state quantities at the predicted observation moment, and → is used to represent the mapping relationship of the set of times corresponding to the state quantities; For representing the converted sensor information of the i-th target at the same moment; A data set for representing the converted sensor information of N targets at the same moment.

11. The multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container according to claim 9, wherein, Estimating the perception result of the vehicle at the current moment or a future preset moment according to the converted sensor information at the same moment includes: Using the maximum likelihood estimation algorithm to estimate the predicted perception result of the vehicle within a future preset moment or a preset time range.

12. The multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container according to claim 11, wherein, Using the following formula to estimate the predicted perception result of the vehicle within a future preset moment or a preset time range includes: Among them, used to represent the estimated perception result at a future preset moment or within a preset time duration range, X D used to represent the perception result at the current moment, a data set used to represent the sensor information of N targets at the same moment, is at X D under the conditional probability.

13. A perception device for multi-vehicle joint perception information spatio-temporal unification based on a map-aware container, wherein, comprising: A sensor information extraction module for extracting sensor information of one or more sensors, where the sensor information includes spatial information in the sensor observation data coordinate system of the sensor and time information of the observation acquisition moment; A spatial information conversion module for providing a map and converting the spatial information in the sensor observation data coordinate system of each sensor information to spatial information in the map coordinate system based on the map; A time information conversion module for converting the time information of the observation acquisition moment of each sensor information to the time information of the driving environment state space; A perception result determination module for determining the perception result of the vehicle according to the spatial information in the map coordinate system and the time information of the driving environment state space; The spatial information conversion module is further configured to: Extract the sensor observation data coordinates of the target from the spatial information in the sensor observation data coordinate system of the sensor information; Perform a first mapping process on the sensor observation data coordinates of the target to obtain the data coordinates of the target in the observation data coordinate system; Perform a first inverse mapping process on the data coordinates of the target to obtain the map coordinates of the target in the map coordinate system; During the first mapping process, the unified space mapping coordinate conversion process of the target position from the driving environment space to the perception space is expressed as: where h k is used to represent the detection data coordinates of the target, a is used to represent the sensor coordinate system, v is used to represent the vehicle body coordinate system, w is used to represent the sensor observation data coordinate system, f a is used to represent the conversion function from the sensor coordinate system to the observation data coordinate system, is used to represent the transformation matrix from the vehicle body coordinate system to the sensor coordinate system, is used to represent the transformation matrix from the sensor observation data coordinate system to the vehicle body coordinate system, is used to represent the sensor observation data coordinates of the target in the set of driving environment space states of the vehicle, k is used to represent the moment of obtaining the sensor observation data coordinates of the target, D is used to represent the set of elements in the driving environment space, including dynamic targets, static targets and road information, and the perception space is used to represent the environmental space where the sensor can detect the target.

14. A storage medium, on which a computer program is stored, wherein, When the computer program is run by a processor, it executes the steps of the multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container according to any one of claims 1 to 12.

15. A terminal, including a memory and a processor, and a computer program capable of running on the processor is stored on the memory, wherein, When the processor runs the computer program, it executes the steps of the multi-vehicle joint perception information spatio-temporal unification method based on a map-aware container according to any one of claims 1 to 12.

Images