Vehicle perception information fusion method, device, equipment and storage medium

Abstract

Embodiments of the present invention disclose a vehicle perception information fusion method, apparatus, device, and storage medium. The method includes: generating local grid map information for the perception information fusion vehicle based on sensor detection data from each perception information fusion vehicle; fusing the local grid map information of each perception information fusion vehicle to obtain target grid map information; and transmitting the target grid map information to each perception information fusion vehicle so that each perception information fusion vehicle determines environmental information based on the target grid map information. By generating grid map information based on sensor detection data and fusing it, the information generated based on the sensor detection data is continuous, ensuring data accuracy, and thereby improving the accuracy of the perception information fusion results.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular to a vehicle perception information fusion method, apparatus, device, and storage medium. Background Art

[0002] With the development of next-generation communications technology, autonomous vehicles are increasingly becoming connected, and collaborative perception has become an emerging technology field. Powered by communications technology, vehicles can interact with each other to perceive information. Multi-perspective perception data not only provides vehicles with more environmental information, but also provides more basis for subsequent decision-making and execution modules.

[0003] In the process of realizing the present invention, the inventors discovered that there are at least the following technical problems in the prior art: the detection data collected by vehicle sensors is discretely distributed, resulting in a certain degree of uncertainty in the collected data, which in turn affects the fusion results of the perception information. Summary of the Invention

[0004] Embodiments of the present invention provide a vehicle perception information fusion method, apparatus, device, and storage medium to improve the accuracy of perception information fusion.

[0005] In a first aspect, an embodiment of the present invention provides a vehicle perception information fusion method, comprising:

[0006] Generate local grid map information of the perception information fusion vehicle based on sensor detection data of each perception information fusion vehicle;

[0007] The perception information is integrated with the local grid map information of the vehicle to obtain the target grid map information;

[0008] The target grid map information is sent to each perception information fusion vehicle, so that each perception information fusion vehicle determines the environmental information according to the target grid map information.

[0009] In a second aspect, an embodiment of the present invention further provides a vehicle perception information fusion device, comprising:

[0010] A raster map information acquisition module is used to generate local raster map information of the perception information fusion vehicle based on the sensor detection data of each perception information fusion vehicle;

[0011] The grid map information fusion module is used to fuse the local grid map information of each perception information with the vehicle to obtain the target grid map information;

[0012] The grid map information sending module is used to send the target grid map information to each perception information fusion vehicle, so that each perception information fusion vehicle determines the environmental information according to the target grid map information.

[0013] In a third aspect, an embodiment of the present invention further provides a computer device, comprising:

[0014] one or more processors;

[0015] a storage device for storing one or more programs;

[0016] When one or more programs are executed by one or more processors, the one or more processors implement the vehicle perception information fusion method provided by any embodiment of the present invention.

[0017] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle perception information fusion method provided in any embodiment of the present invention.

[0018] The vehicle perception information fusion method provided by an embodiment of the present invention generates local grid map information for the perception information fusion vehicle based on sensor detection data from each perception information fusion vehicle; fuses the local grid map information of each perception information fusion vehicle to obtain target grid map information; and transmits the target grid map information to each perception information fusion vehicle so that each perception information fusion vehicle determines environmental information based on the target grid map information. By generating grid map information based on sensor detection data and then fusing it, the information generated based on the sensor detection data is continuous, ensuring data accuracy and thereby improving the accuracy of the perception information fusion results. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1a This is a flow chart of a vehicle perception information fusion method provided in the first embodiment of the present invention;

[0020] Figure 1b This is a schematic diagram of a sensor beam model provided by the second embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of a multi-vehicle collaborative perception process provided by the second embodiment of the present invention;

[0022] Figure 3 This is a schematic structural diagram of a vehicle perception information fusion device provided by the third embodiment of the present invention;

[0023] Figure 4 It is a structural diagram of a computer device provided in Example 4 of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0025] Example 1

[0026] Figure 1a This is a flow chart of a vehicle perception information fusion method provided by the first embodiment of the present invention. This embodiment is applicable to the situation when the perception information of multiple vehicles is fused. The method can be executed by a vehicle perception information fusion device, which can be implemented in software and / or hardware. For example, the vehicle perception information fusion device can be configured in a computer device, such as a vehicle. Figure 1a As shown, the method includes:

[0027] S110. Generate local grid map information of the perception information fusion vehicle based on the sensor detection data of each perception information fusion vehicle.

[0028] In this embodiment, the perception information fusion device can construct and fuse a local grid map. However, when the perception information fusion device constructs the local grid map, the perception information fusion vehicle must send all vehicle sensor detection data to the perception information fusion device. The perception information fusion device then obtains the sensor detection data of all perception information fusion vehicles and constructs local grid map information for each perception information fusion vehicle. It is understandable that when there are a large number of perception information fusion vehicles, the amount of data that needs to be transmitted is large, and the amount of data calculation required by the perception information fusion device will also increase. Therefore, each perception information fusion vehicle can also construct its own local grid map information. Each perception information fusion vehicle sends its constructed local grid map information to the perception information fusion device, and the perception information fusion device only needs to fuse the received local grid map information.

[0029] That is to say, generating the local grid map information of the perception information fusion vehicle based on the sensor detection data of the perception information fusion vehicle can be executed by the perception information fusion vehicle itself or by the perception information fusion device. The specific execution entity can be set according to actual needs and is not restricted here.

[0030] In one embodiment of the present invention, local grid map information of the perception information fusion vehicle is generated based on sensor detection data of the perception information fusion vehicle, including: obtaining sensor detection data collected by at least one sensor of the perception information fusion vehicle; determining the probability density distribution of obstacles in a set area based on the sensor detection data; and generating local grid map information based on the probability density distribution of obstacles in the set area.

[0031] Considering the discrete distribution of lidar point clouds in traditional multi-sensor fusion algorithms, there's no guarantee that every grid cell occupied by the vehicle will contain a data point. The bounding box generated by the image detection algorithm may include obstacles, but it also includes a portion of the background area, making it difficult to ensure that obstacles are always fully contained. Therefore, both lidar and visual sensor image data contain a certain degree of uncertainty. In embodiments of the present invention, discrete sensor detection data can be converted into continuously distributed probability density information and mapped onto a grid map to generate local grid map information, providing secure and reliable information for perceptual information fusion. Optionally, each perceptual information fusion vehicle can be equipped with multiple sensors to collect sensor detection data, such as lidar and visual sensors. The perceptual information fusion vehicle fuses the multi-sensor detection data collected by its sensors to generate sensor detection data used to generate the local grid map. The discrete sensor detection data is then processed to obtain continuously distributed probability density information, and the local grid map is generated based on this probability density information. Probability density information can be understood as the probability of an obstacle existing in a given area.

[0032] When the generation of local grid map information is realized in the perception information fusion device, the perception information fusion vehicle can send the original detection information collected by the sensor to the perception information fusion device, and the perception information fusion device realizes the fusion of multi-sensor information and the generation of the local grid map; the perception information fusion vehicle can also fuse the original detection information collected by multiple sensors to generate sensor detection data and send it to the perception information fusion device, and the perception information fusion device only needs to generate the local grid map based on the sensor detection data.

[0033] When the generation of local grid map information is implemented in the perception information fusion vehicle, the perception information fusion vehicle directly fuses the original detection information collected by multiple sensors to generate sensor detection data, generates a local grid map based on the sensor detection data, and then sends the local grid map to the perception information fusion device.

[0034] In one embodiment, determining the probability density distribution of obstacles within a set area based on sensor detection data includes: determining the probability density distribution of obstacles within the set area using a kernel density estimation algorithm based on the sensor detection data. Optionally, the kernel density estimation algorithm can be used to convert discretely distributed sensor detection data into a continuously distributed probability density distribution.

[0035] Kernel density estimation (KDE) is a nonparametric method used to estimate the probability density function of a random variable. Similar to the purpose of occupancy probability distribution algorithms, KDE estimates the probability density of an occupied area based on the position information of a limited number of data points returned by sensors. Therefore, KDE can be applied to the estimation of the probability density of obstacles.

[0036] For example, (x1x2,...,x n ) are independent identically distributed samples drawn from the unknown density p(x), and the fixed bandwidth kernel density estimator is defined as:

[0037]

[0038] Where k is the kernel function, which is a non-negative function that integrates to 1 in a given area. Usually a unimodal symmetric function is chosen as the kernel function. h is a smoothing parameter greater than 0, called the bandwidth. k h is the scaling kernel, defined as:

[0039]

[0040] In the kernel density estimation process, the selection of bandwidth h is a complex issue. If the bandwidth is too large, the estimated result will be too smooth, thereby masking the data structure of the true probability distribution and resulting in a large deviation from the true probability density; if the bandwidth is too small, the deviation between the estimated result and the true density will be reduced, but the variance of the estimated value will be large and the result will be too sharp. Therefore, the process of selecting the optimal h bandwidth is a trade-off between the bias and variance of the estimator, which is a difficulty in estimating kernel density. The mean integrated squared error (MISE) is often used in the selection process of the bandwidth h because it can measure the overall performance of the estimator and is related to the mean squared error. Assume that the detection point comes from the distribution p(x), and the estimated result of p(x) is p KDE (x), the mean integrated square error can be expressed as:

[0041] MISE(h)=E[(p KDE (x)-p(x)) 2 dx]

[0042] h can be obtained by minimizing the equation MISE(h), when n→∞ and h=h(n)→0, and asymptotically expanding the first two terms of AMISE:

[0043] AMISE(h)=(nh) -1 R(K)+h 4 R(f * )(∫(x2 K / 2)) 2

[0044] in, ∫(x 2 K)=∫(x 2 K(x))dx, the minimum value of AMISE(h) can be simply calculated as:

[0045]

[0046] From the above formula, it can be seen that a smaller bandwidth is better for a larger number of detection points n, because the estimation process will be performed with better resolution when more information is provided. However, h AMISE It is not always a good approximation to h MISE In contrast to the choice of bandwidth, the choice of kernel function does not play an important role in the kernel density estimation process, as the kernel function only affects the estimation error by a very small constant offset. Without loss of generality, a Gaussian kernel is chosen.

[0047] In one embodiment, determining the probability density distribution of obstacles within a set area based on sensor detection data includes: determining the probability density distribution of obstacles within the set area using an occupancy probability distribution algorithm based on the sensor detection data. Optionally, the occupancy probability distribution algorithm can be used to convert discretely distributed sensor detection data into a continuously distributed probability density distribution.

[0048] Assuming that there are N vehicles working together, the number of obstacles detected by M sensors in the main vehicle's map can be expressed as: The output of the occupancy probability distribution algorithm is: m(x,y) indicates that the grid at position (x,y) is occupied; Indicates grid positions (x,y) based on measured values Since the perceptual information about the environment has not yet been obtained, the initial value of the occupancy probability of all grids can be set to 0.5.

[0049] Here, the standard two-dimensional normal distribution is used as the kernel function, which is expressed as:

[0050]

[0051] Where z = (x, y) is a two-dimensional row vector. For two-dimensional kernel density estimation, the bandwidth h is represented by H, which is a two-dimensional matrix. The result of kernel estimation is:

[0052]

[0053] Among them, z represents the coordinates of the point to be estimated z = (x, y), is the scaled kernel function, which can be expressed as:

[0054]

[0055] A general rule of thumb can be used to determine the h-bandwidth matrix. Assuming that the probability distribution of the target vehicle's occupied area is Gaussian, the h-bandwidth matrix can be expressed as:

[0056] H=M -1 / 6 ∑^ 1 / 2

[0057] Where ∑^ is the covariance matrix of the sensor data points, which is used as an estimate of the covariance matrix in the Gaussian distribution. n Substitute P KDE , we can get the probability density distribution of the target object’s occupied area, that is, the required occupancy probability distribution is

[0058] In one embodiment, determining the probability density distribution of obstacles within a set area based on sensor detection data includes: determining the probability density distribution of obstacles within the set area using an occupancy probability distribution conversion algorithm based on the sensor detection data. Optionally, the occupancy probability distribution conversion algorithm can be used to convert discretely distributed sensor detection data into a continuously distributed probability density distribution.

[0059] The main purpose of the mapping process is to construct the area occupied by obstacles in the map using the occupancy probability at the previous moment and the probability of the current measurement value obtained based on the occupancy probability distribution algorithm. First, the occupancy probability distribution obtained in the occupancy probability distribution algorithm needs to be normalized. The normalized result is On the map (x i ,y i ) grid, and p t-1 (x i ,y i ) is expressed as the occupancy rate at the previous moment, and p t (x i ,y i ) represents its current occupancy.

[0060] (x i ,y i ) is mapped to the map, and the mapping algorithm can be expressed as:

[0061] enter: p t-1 (x i ,y i );

[0062] Output: p t (x i ,yi )

[0063] 1. If p t-1 (x i ,y i )=1 or Then p t (x i ,y i )=1;

[0064] 2. If Then p t (x i ,y i )=0.5;

[0065] 3. If p t-1 (x i ,y i )>0.5, then

[0066]

[0067]

[0068] 4. In other cases,

[0069] Return p t (x i ,y i ).

[0070] Among them, the first case means that if the probability of any input is 1, the grid is considered to be occupied and its occupancy probability is assigned to 1. The second case means that if the returned probability of 1 is less than 0.5 based on the measurement value, it is not considered to be included in the occupied area of ​​the obstacle. The probability distribution of the free area will be discussed in the free area determination algorithm section, and its status is set to uncertain here. The third case means that if both input probabilities are greater than 0.5, it indicates that there is an obstacle in the map, which may be close to the measured vehicle, resulting in some area overlap. Here, the log-likelihood ratio is used to fuse the probabilities. The fourth case means that if the current input does not belong to the above case, the probability based on the current measurement value will be directly mapped to the grid map.

[0071] In one embodiment, determining the probability density distribution of obstacles within a predetermined area based on sensor detection data includes: determining the probability density distribution of obstacles within the predetermined area using a free region determination algorithm based on the sensor detection data. Optionally, the free region determination algorithm may be used to convert discretely distributed sensor detection data into a continuously distributed probability density distribution.

[0072] The distribution of the idle areas is determined according to the positions of the occupied areas determined in the occupancy probability distribution algorithm. Figure 1b This is a schematic diagram of a sensor beam model provided by the second embodiment of the present invention. Figure 1b As shown in the figure, the driving environment around the ego vehicle is divided according to the sensor's maximum detection range D and the distribution of detected obstacles. For simplicity, it can be assumed that there is only one obstacle within the detection range, which is the elliptical shaded area in the figure. The darker the color, the greater the probability of occupancy. Uncertain states are represented by diagonal lines.

[0073] First, for the grids in the area beyond the maximum range of the sensor in the map (the sensor cannot detect it), its value is considered to be an uncertain state of 0.5. Then, if the return range of the sensor is the maximum range, it means that there are no obstacles within the detection range in this direction, so all grids in this direction are in a free state. For the area blocked by undetectable obstacles, the state is uncertain and the value is 0.5. On the contrary, there is no blockage in the path from the sensor to the occupied part, so the grids in this area are in a free state. For grids with a free state, the occupancy probability value is lower than 0.5 and increases linearly with the distance from the sensor. At the boundary of the detectable area, the maximum value of the free grid is 0.5, because the state of the maximum range of the sensor cannot be determined. Based on this, a continuous probability distribution can be obtained.

[0074] S120 , fusing each perception information with the local grid map information of the vehicle to obtain target grid map information.

[0075] In this embodiment, there are no restrictions on the method for fusing local grid map information. All perception information fusion vehicles in the scene share map data with the perception information fusion device. The perception information fusion device fuses the local grid map information of multiple vehicles and returns the global map as the fusion result to each perception information fusion vehicle. Local grid map information fusion primarily involves two steps: map alignment and probabilistic fusion. Map alignment can employ existing map alignment methods, which will not be elaborated here.

[0076] In one embodiment, the local grid map information of each sensory information fusion vehicle is fused to obtain target grid map information, including: performing coordinate transformation on each local grid map to obtain transformed grid map information in the same coordinate system; and fusing the transformed grid map information to obtain target grid map information. Optionally, given that the sensory information fusion device knows the position of each sensory information fusion vehicle in the global coordinate system, the local grid map information of each sensory information fusion vehicle can be converted into a global map through a simple coordinate transformation, preparing for the next step of merging the probabilities of different maps in overlapping areas. To combine different maps, the log-likelihood ratio can be used for probability fusion in overlapping areas. This is because, compared to probabilities between 0 and 1, log-likelihood ratios within this range can avoid unnecessary truncation errors. Furthermore, the occupancy rate of each grid cell can be interpreted as a hypothesis testing problem, and since different sensory information fusion vehicles are independently detecting the occupancy status of the grid cells, log-likelihood ratio combination is the optimal method for quantitatively combining sensory information.

[0077] Assume that at time t, there are N vehicles participating in the cooperation, numbered V1...V N After coordinate transformation, their corresponding maps in the global coordinate system are represented as follows: Then, for the rth grid in the global map, it is related to the vehicle V i The log-likelihood ratio corresponding to the detected occupancy probability can be expressed as:

[0078]

[0079] Among them, (x r ,y r ) is the coordinate of the rth grid in the global map, Display map The occupancy probability of grid r in the grid, and its corresponding log-likelihood ratio is

[0080] The log-likelihood ratios of the grid occupancy rates detected by N vehicles are added together, and the result is converted into a probability form, which can be mapped as a probability fusion result to the corresponding position of the global occupancy grid map. This process can be expressed as:

[0081]

[0082]

[0083] in, is the occupancy probability of the fused grid r.

[0084] S130. Send the target grid map information to each perception information fusion vehicle, so that each perception information fusion vehicle determines the environmental information according to the target grid map information.

[0085] After the perception information fusion device fuses the local grid map information to obtain the target grid map information, it sends the target grid map information to each perception information fusion vehicle. The perception information fusion vehicle determines the environmental information based on the received target grid map information and performs operations such as path planning.

[0086] The vehicle perception information fusion method provided by an embodiment of the present invention generates local grid map information for the perception information fusion vehicle based on sensor detection data from the perception information fusion vehicle; fuses the local grid map information of each perception information fusion vehicle to obtain target grid map information; and transmits the target grid map information to the perception information fusion vehicle, allowing the perception information fusion vehicle to determine environmental information based on the target grid map information. By generating grid map information based on sensor detection data and then fusing it, the information generated based on the sensor detection data is continuous, ensuring data accuracy and, therefore, improving the accuracy of the perception information fusion results.

[0087] Example 2

[0088] Figure 2 This is a schematic diagram of a multi-vehicle collaborative perception process provided by the second embodiment of the present invention. This embodiment provides a preferred embodiment based on the above solution.

[0089] like Figure 2 As shown in the figure, each perception information fusion vehicle performs multi-sensor data fusion and local map generation to obtain a local grid map of each perception information fusion vehicle. The local grid map is transformed in coordinates and then fused to obtain a global grid map (i.e., target grid map information), completing the fusion of perception information.

[0090] The multi-sensor data fusion can be performed by the sensor data fusion method in the prior art, which will not be described in detail here.

[0091] The generation of the local grid map can be specifically as follows: discrete sensor data is converted into a continuous probability density distribution, and mapped to a grid map to obtain a local grid map.

[0092] Converting discrete sensor data into a continuous probability density distribution can be achieved by using at least one of a kernel density estimation algorithm, an occupancy probability distribution algorithm, an occupancy probability distribution conversion algorithm, and a free area determination algorithm. The specific implementation method can be found in the above embodiments and will not be further described here.

[0093] When fusing local grid maps, the position of the vehicle in the global coordinate system can be fused according to the perception information of the local grid maps, the local grid maps can be converted to the global coordinate system, and all local grid maps in the global coordinate system can be fused to obtain a global grid map.

[0094] It should be noted that the embodiments of this invention validate multi-vehicle collaborative perception based on image data collected in real-world scenarios. A selection of real-world scenes were included in the original dataset. In these selected scenes, the perception information fusion method provided by this embodiment achieved superior detection results compared to existing perception information fusion methods.

[0095] The embodiment of the present invention builds a multi-vehicle, multi-sensor collaborative perception framework. In order to map the data after multi-vehicle, multi-sensor fusion to a grid map in space and probabilistic terms, a kernel density estimation algorithm is introduced, and an occupancy probability distribution algorithm, an occupancy probability distribution conversion algorithm, a free area determination algorithm, and a multi-vehicle map fusion algorithm are designed. This provides a broader perspective for collaborative perception of autonomous driving vehicles, enriches the details of dynamic targets, and improves perception accuracy.

[0096] Example 3

[0097] Figure 3 This is a schematic diagram of the structure of a vehicle perception information fusion device provided by the third embodiment of the present invention. The vehicle perception information fusion device can be implemented in software and / or hardware. For example, the vehicle perception information fusion device can be configured in a computer device. Figure 3 As shown, the device includes a raster image information acquisition module 310, a raster image information fusion module 320 and a raster image information sending module 330, wherein:

[0098] A grid map information acquisition module 310 is configured to generate local grid map information of the perception information fusion vehicle based on sensor detection data of each perception information fusion vehicle;

[0099] The grid map information fusion module 320 is used to fuse the various sensing information with the local grid map information of the vehicle to obtain target grid map information;

[0100] The grid map information sending module 330 is used to send the target grid map information to each perception information fusion vehicle, so that each perception information fusion vehicle determines the environmental information according to the target grid map information.

[0101] The vehicle perception information fusion method provided by an embodiment of the present invention generates local grid map information for the perception information fusion vehicle based on sensor detection data from each perception information fusion vehicle; fuses the local grid map information of each perception information fusion vehicle to obtain target grid map information; and transmits the target grid map information to each perception information fusion vehicle so that each perception information fusion vehicle determines environmental information based on the target grid map information. By generating grid map information based on sensor detection data and then fusing it, the information generated based on the sensor detection data is continuous, ensuring data accuracy and thereby improving the accuracy of the perception information fusion results.

[0102] Optionally, based on the above solution, the grid image information acquisition module 310 is specifically configured to:

[0103] Acquiring sensor detection data collected by at least one sensor of the perception information fusion vehicle;

[0104] Determine the probability density distribution of obstacles in a set area based on sensor detection data;

[0105] Generate local grid map information based on the probability density distribution of obstacles in the set area.

[0106] Optionally, based on the above solution, the grid image information acquisition module 310 is specifically configured to:

[0107] Based on the sensor detection data, the probability density distribution of obstacles in the set area is determined by the kernel density estimation algorithm.

[0108] Optionally, based on the above solution, the grid image information acquisition module 310 is specifically configured to:

[0109] Based on the sensor detection data, the probability density distribution of obstacles in the set area is determined by the occupancy probability distribution algorithm.

[0110] Optionally, based on the above solution, the grid image information acquisition module 310 is specifically configured to:

[0111] Based on the sensor detection data, the probability density distribution of obstacles in the set area is determined through the occupancy probability distribution conversion algorithm.

[0112] Optionally, based on the above solution, the grid image information acquisition module 310 is specifically configured to:

[0113] Based on the sensor detection data, the probability density distribution of obstacles in the set area is determined by the free area determination algorithm.

[0114] Optionally, based on the above solution, the grid image information fusion module 320 is specifically configured to:

[0115] Perform coordinate conversion on each local grid map information to obtain each converted grid map information in the same coordinate system;

[0116] The information of each converted raster map is fused to obtain the target raster map information.

[0117] The vehicle perception information fusion device provided in the embodiment of the present invention can execute the vehicle perception information fusion method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0118] Example 4

[0119] Figure 4 It is a structural diagram of a computer device provided in Example 4 of the present invention. Figure 4 A block diagram of an exemplary computer device 412 suitable for use in implementing embodiments of the present invention is shown. Figure 4 The computer device 412 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present invention.

[0120] like Figure 4 As shown, computer device 412 is implemented as a general-purpose computing device. Components of computer device 412 may include, but are not limited to, one or more processors 416, a system memory 428, and a bus 418 that connects various system components (including system memory 428 and processor 416).

[0121] Bus 418 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a local bus to processor 416, or a bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0122] The computer device 412 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 412, including volatile and non-volatile media, removable and non-removable media.

[0123] System memory 428 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 430 and / or cache memory 432. Computer device 412 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage device 434 may be configured to read and write non-removable, non-volatile magnetic media ( Figure 4 Not shown, often called a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 418 via one or more data media interfaces. Memory 428 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0124] A program / utility 440 having a set (at least one) of program modules 442 may be stored, for example, in memory 428. Such program modules 442 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 442 generally implement the functions and / or methodologies of the embodiments described herein.

[0125] The computer device 412 can also communicate with one or more external devices 414 (e.g., a keyboard, pointing device, display 424, etc.), one or more devices that enable a user to interact with the computer device 412, and / or any device that enables the computer device 412 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication can occur via an input / output (I / O) interface 422. Furthermore, the computer device 412 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 420. As shown, the network adapter 420 communicates with the other modules of the computer device 412 via a bus 418. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with the computer device 412, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0126] The processor 416 executes various functional applications and data processing by running programs stored in the system memory 428, such as implementing the vehicle perception information fusion method provided in an embodiment of the present invention, which includes:

[0127] Generate local grid map information of the perception information fusion vehicle based on sensor detection data of each perception information fusion vehicle;

[0128] The perception information is integrated with the local grid map information of the vehicle to obtain the target grid map information;

[0129] The target grid map information is sent to each perception information fusion vehicle, so that each perception information fusion vehicle determines the environmental information according to the target grid map information.

[0130] Of course, those skilled in the art will understand that the processor can also implement the technical solution of the vehicle perception information fusion method provided by any embodiment of the present invention.

[0131] Example 5

[0132] The fifth embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the vehicle perception information fusion method provided in the embodiment of the present invention is implemented. The method includes:

[0133] Generate local grid map information of the perception information fusion vehicle based on sensor detection data of the perception information fusion vehicle;

[0134] The perception information is integrated with the local grid map information of the vehicle to obtain the target grid map information;

[0135] The target grid map information is sent to the perception information fusion vehicle so that the perception information fusion vehicle determines the environmental information according to the target grid map information.

[0136] Of course, the computer-readable storage medium provided by an embodiment of the present invention stores a computer program thereon that is not limited to the above method operations, but can also execute related operations of the vehicle perception information fusion method provided by any embodiment of the present invention.

[0137] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0138] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0139] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0140] Computer program code for carrying out the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0141] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will appreciate that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments, and substitutions are possible for those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A vehicle perception information fusion method, characterized in that: include: Generating local grid map information of the perception information fusion vehicle based on sensor detection data of each perception information fusion vehicle; Fusing the perception information with the local grid map information of the vehicle to obtain target grid map information; Sending the target grid map information to each of the perception information fusion vehicles, so that each of the perception information fusion vehicles determines environmental information according to the target grid map information; The generating of the local grid map information of the perception information fusion vehicle according to the sensor detection data of each perception information fusion vehicle includes: Acquiring sensor detection data collected by at least one sensor of the perception information fusion vehicle; After processing the discrete sensor detection data, the continuously distributed probability density information is obtained as the probability density distribution of the obstacle in the set area. The probability density information is the probability of the obstacle existing in the set area. Generating the local grid map information according to the probability density distribution of the obstacles in the set area; The method of processing the discrete sensor detection data to obtain continuously distributed probability density information as the probability density distribution of obstacles in the set area includes: Based on the sensor detection data, determining the occupancy probability distribution of each grid in the set area by the obstacle through an occupancy probability distribution conversion algorithm, and normalizing the occupancy probability distribution of each grid to obtain a normalized result for each grid; Mapping the normalized result of the grid to a map to obtain the probability density distribution of obstacles in a set area; The input of the mapping algorithm is the normalized result of the grid and the obstacle occupancy rate of the grid at the previous moment. The output of the mapping algorithm is the current obstacle occupancy rate of the grid. For any grid, the normalized result of the grid is mapped to the map, including: When the normalized result of the grid or the obstacle occupancy rate of the grid at the previous moment is 1, the obstacle occupancy rate of the grid at the current moment is 1; When the normalized result of the grid is less than 0.5, the grid is a free area, and the obstacle occupancy rate of the grid at the current moment is determined by a free area determination algorithm; When the obstacle occupancy rate of the grid at the previous moment is greater than 0.5, the obstacle occupancy rate of the grid at the previous moment and the normalized result of the grid are fused by log-likelihood ratio to obtain the obstacle occupancy rate of the grid at the current moment; In cases other than the above case, the normalized result of the grid is used as the obstacle occupancy rate of the grid at the current moment.

2. The method according to claim 1, characterized in that The step of fusing the perception information with the local grid map information of the vehicle to obtain target grid map information includes: Perform coordinate conversion on each local grid map information to obtain each converted grid map information in the same coordinate system; The converted raster map information is fused to obtain the target raster map information.

3. A vehicle perception information fusion device, characterized in that: include: A grid map information acquisition module, configured to generate local grid map information of the perception information fusion vehicle based on sensor detection data of each perception information fusion vehicle; A grid map information fusion module is used to fuse the perception information with the local grid map information of the vehicle to obtain target grid map information; a grid map information sending module, configured to send each target grid map information to the perception information fusion vehicle, so that the perception information fusion vehicle determines environmental information according to the target grid map information; The grid image information acquisition module is specifically used to: Acquiring sensor detection data collected by at least one sensor of the perception information fusion vehicle; processing the discrete sensor detection data to obtain continuously distributed probability density information as a probability density distribution of an obstacle in a set area, the probability density information being a probability of an obstacle existing in the set area; and generating the local grid map information based on the probability density distribution of the obstacle in the set area. The grid image information acquisition module is specifically used to: Based on the sensor detection data, determining the occupancy probability distribution of each grid in the set area by the obstacle through an occupancy probability distribution conversion algorithm, and normalizing the occupancy probability distribution of each grid to obtain a normalized result for each grid; Mapping the normalized result of the grid to a map to obtain the probability density distribution of obstacles in a set area; The input of the mapping algorithm is the normalized result of the grid and the obstacle occupancy rate of the grid at the previous moment. The output of the mapping algorithm is the current obstacle occupancy rate of the grid. For any grid, the grid map information acquisition module is specifically used to: When the normalized result of the grid or the obstacle occupancy rate of the grid at the previous moment is 1, the obstacle occupancy rate of the grid at the current moment is 1; When the normalized result of the grid is less than 0.5, the grid is a free area, and the obstacle occupancy rate of the grid at the current moment is determined by a free area determination algorithm; When the obstacle occupancy rate of the grid at the previous moment is greater than 0.5, the obstacle occupancy rate of the grid at the previous moment and the normalized result of the grid are fused by log-likelihood ratio to obtain the obstacle occupancy rate of the grid at the current moment; In cases other than the above case, the normalized result of the grid is used as the obstacle occupancy rate of the grid at the current moment.

4. A computer device, characterized in that: The device comprises: one or more processors; a storage device for storing one or more programs; When one or more programs are executed by one or more processors, the one or more processors implement the vehicle perception information fusion method as described in any one of claims 1-2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the vehicle perception information fusion method as described in any one of claims 1-2 is implemented.

Images