Vehicle aerial view construction method and device, storage medium and electronic equipment

Abstract

The invention provides a vehicle aerial view construction method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining a plurality of pieces of sensing data, the plurality of pieces of sensing data comprise self-vehicle sensing data, other-vehicle sensing data and road end sensing data, the self-vehicle sensing data is data detected by a sensor of a self-vehicle, and the road end sensing data is data detected by the sensor of the self-vehicle; the other vehicle sensing data is data detected by sensors of other vehicles except the own vehicle in a preset range, and the road end sensing data is data detected by a sensor of road end equipment arranged on a traffic lane; data fusion is conducted on the multiple pieces of sensing data, an environment sensing result of the vehicle is obtained, and the environment sensing result comprises a target object in the driving environment of the vehicle; and constructing a bird's-eye view of the own vehicle according to the environment perception result. According to the invention, the recognition area of the vehicle is expanded, and the sensing blind area of the vehicle is reduced, so that the accuracy of sensing the vehicle environment and the accuracy of constructing the aerial view are improved, and the safety of vehicle driving is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle safety technology, and in particular to a method, device, storage medium and electronic device for constructing a bird's-eye view of a vehicle. Background Art

[0002] In the current intelligent driving field, a bird's eye view (BEV) can be constructed by multi-camera fusion and multi-mode fusion of multiple on-board sensors on the vehicle to predict driving risks through the bird's eye view, but this method cannot perceive and predict the detection blind spots of on-board sensors. The construction of the bird's eye view mainly relies on the perception function of the vehicle's sensors. When the vehicle's sensors perceive environmental data, due to their own performance limitations and the complexity of the real environment, the perception results may be inaccurate. For example, due to the limited perception distance of the sensor, the vehicle's ability to identify potential obstacles ahead will be affected. Or because the sensor's recognition accuracy for dynamic targets is low, the sensor may have difficulty distinguishing small objects or fast-moving objects, resulting in misjudgment or missed judgment, which has a negative impact on driving decisions. Or because the design of the vehicle and the arrangement of sensors still result in some areas not being effectively covered, there are blind spots in the sensor's detection area, and potential dangers may be hidden in the blind spots, which have an adverse effect on driving safety.

[0003] Therefore, it is necessary to further improve the accuracy of the vehicle's perception of environmental information and build an accurate bird's-eye view to reduce the vehicle's detection blind spots. Summary of the invention

[0004] In view of this, the present disclosure provides a method, device, storage medium and electronic device for constructing a bird's-eye view of a vehicle, which can reduce the perception blind spot of the vehicle and improve the safety of vehicle driving.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for constructing a vehicle bird's-eye view is provided, the method comprising:

[0006] Acquire multiple sensor data, wherein the multiple sensor data include own vehicle sensor data, other vehicle sensor data and roadside sensor data, the own vehicle sensor data is data detected by the sensors of the own vehicle, the other vehicle sensor data is data detected by sensors of other vehicles other than the own vehicle within a preset range, and the roadside sensor data is data detected by sensors of roadside equipment arranged on the driving road;

[0007] Performing data fusion on the multiple sensor data to obtain an environmental perception result of the vehicle, wherein the environmental perception result includes a target object in the driving environment of the vehicle;

[0008] A bird's-eye view of the vehicle is constructed based on the environmental perception result.

[0009] In one embodiment, the performing data fusion on the plurality of sensor data to obtain the environmental perception result of the vehicle includes:

[0010] In the case where the plurality of sensor data are raw data collected by a sensor, feature extraction is performed on the plurality of sensor data to obtain first feature data of the self-vehicle sensor data, second feature data of the other-vehicle sensor data, and third feature data of the roadside sensor data;

[0011] Performing feature fusion on the first feature data, the second feature data and the second feature data to obtain a first feature fusion result;

[0012] According to the first feature fusion result, the target object in the driving environment of the vehicle is identified to obtain the environmental perception result.

[0013] In one embodiment, the performing data fusion on the plurality of sensor data to obtain the environmental perception result of the vehicle includes:

[0014] In the case where the plurality of sensor data are feature data, feature fusion is performed on the plurality of sensor data to obtain a second feature fusion result; the feature data is data obtained after the sensor performs feature extraction on the collected raw data;

[0015] According to the second feature fusion result, the target object in the driving environment of the vehicle is identified to obtain the environmental perception result.

[0016] In one embodiment, the performing data fusion on the plurality of sensor data to obtain the environmental perception result of the vehicle includes:

[0017] If the plurality of sensor data include data information of the target object, the environmental perception result is determined based on the data information of the target object in each sensor data.

[0018] In one embodiment, after fusing the plurality of sensor data to obtain the environment perception result of the vehicle, the method further includes:

[0019] If the category of the target object in the environmental perception result belongs to a preset category and / or the target object is a stationary object, determining that the target object is a long-term target object;

[0020] In the case where the target object is a long-term target object, the environment perception result is stored in a storage device, so that the storage device sends the environment perception result to other vehicles that enter the driving area of ​​the own vehicle.

[0021] In one embodiment, after fusing the plurality of sensor data to obtain the environmental perception result of the vehicle, the method further includes:

[0022] The environmental perception result is sent to the other vehicles, so that the other vehicles determine a driving path according to the environmental perception result.

[0023] According to a second aspect of an embodiment of the present disclosure, a device for constructing a bird's-eye view of a vehicle is provided, the device comprising:

[0024] an acquisition unit, configured to acquire a plurality of sensor data, wherein the plurality of sensor data include self-vehicle sensor data, other-vehicle sensor data and road-side sensor data, the self-vehicle sensor data is data detected by sensors of the self-vehicle, the other-vehicle sensor data is data detected by sensors of other vehicles other than the self-vehicle within a preset range, and the road-side sensor data is data detected by sensors of road-side equipment arranged on the driving road;

[0025] A fusion unit, configured to fuse the plurality of sensor data to obtain an environmental perception result of the vehicle, wherein the environmental perception result includes a target object in the driving environment of the vehicle;

[0026] A construction unit is used to construct a bird's-eye view of the vehicle according to the environmental perception result.

[0027] In one embodiment, the fusion unit is used to:

[0028] In the case where the plurality of sensor data are raw data collected by a sensor, feature extraction is performed on the plurality of sensor data to obtain first feature data of the self-vehicle sensor data, second feature data of the other-vehicle sensor data, and third feature data of the roadside sensor data;

[0029] Performing feature fusion on the first feature data, the second feature data and the second feature data to obtain a first feature fusion result;

[0030] According to the first feature fusion result, the target object in the driving environment of the vehicle is identified to obtain the environmental perception result.

[0031] In one embodiment, the fusion unit is used to:

[0032] In the case where the plurality of sensor data are feature data, feature fusion is performed on the plurality of sensor data to obtain a second feature fusion result; the feature data is data obtained after the sensor performs feature extraction on the collected raw data;

[0033] According to the second feature fusion result, the target object in the driving environment of the vehicle is identified to obtain the environmental perception result.

[0034] In one embodiment, the fusion unit is used to:

[0035] If the plurality of sensor data include data information of the target object, the environmental perception result is determined based on the data information of the target object in each sensor data.

[0036] In one embodiment, the apparatus further comprises:

[0037] a determination unit, configured to determine that the target object is a long-term target object if the category of the target object in the environmental perception result belongs to a preset category and / or the target object is a stationary object;

[0038] The first sending unit is used to store the environmental perception result in a storage device when the target object is a long-term target object, so that the storage device sends the environmental perception result to other vehicles that enter the driving area of ​​the vehicle.

[0039] In one embodiment, the apparatus further comprises:

[0040] The second sending unit is used to send the environmental perception result to the other vehicles, so that the other vehicles determine the driving path according to the environmental perception result.

[0041] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, and when the program is executed by a processor, the steps of any method described in the first aspect are implemented.

[0042] According to a fourth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any method described in the first aspect are implemented.

[0043] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, which, when executed by a processor, implements the steps of any method described in the first aspect.

[0044] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0045] By integrating the sensor data of the vehicle, other vehicles and roadside sensors, the recognition area of ​​the vehicle is expanded, the blind spot of the vehicle perception is reduced, and the accuracy of the vehicle environment perception is improved. The vehicle bird's-eye view is constructed based on the perception results obtained by fusion of multiple features, which improves the accuracy of the bird's-eye view and thus improves the safety of vehicle driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flow chart of a method for constructing a bird's-eye view of a vehicle shown in an exemplary embodiment of the present disclosure;

[0047] Figure 2 is a flow chart of a method for data fusion shown in an exemplary embodiment of the present disclosure;

[0048] Figure 3 is a flow chart of another data fusion method shown in an exemplary embodiment of the present disclosure;

[0049] Figure 4 is a flow chart of a data sharing method shown in an exemplary embodiment of the present disclosure;

[0050] Figure 5 is a flow chart of another method for constructing a bird's-eye view of a vehicle according to an exemplary embodiment of the present disclosure;

[0051] Figure 6 is a block diagram of a device for constructing a bird's-eye view of a vehicle shown in an exemplary embodiment of the present disclosure;

[0052] Figure 7 It is a hardware structure diagram of a data fusion terminal shown in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0053] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0054] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0055] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0056] In the current intelligent driving field, BEV can be built by multi-camera fusion and multi-mode fusion of multiple on-board sensors on the vehicle, so that the vehicle can predict driving risks based on a bird's-eye view. However, the recognition distance and recognition range of the on-board sensors of a single vehicle are limited. Combined with the complex driving environment in reality, it is easy to cause the sensor's recognition blind spot. The observation area of ​​the human eye is also greatly limited. When blocked by other vehicles, the human eye's observation blind spot will also be relatively large. As a result, there are certain blind spots in the field of vision during vehicle driving, which has an adverse effect on the vehicle's driving safety.

[0057] Therefore, the present disclosure provides a method for constructing a vehicle bird's-eye view, which can be applied to vehicles, roadside devices or cloud devices. This method can improve the accuracy of constructing a bird's-eye view, thereby improving the safety of vehicle driving. For the implementation steps of the method, please refer to Figure 1 The flowchart shown, Figure 1 FIG. 1 is a flow chart of a method for constructing a vehicle bird's-eye view shown in an exemplary embodiment of the present disclosure, which specifically includes the following steps:

[0058] S101, acquiring multiple sensor data.

[0059] In some embodiments, the sensor data may include image data collected by visual sensors, point cloud data collected by laser radars, speed data and distance data collected by millimeter wave radars, and other data information. In the vehicle sensing system, multiple sensors may be set, and the data information collected by each sensor is within the scope of the sensor data described in the present disclosure. Furthermore, the data sources of the multiple sensor data may include the vehicle sensing data collected by the vehicle's sensors, the other vehicle sensing data detected by the sensors of other vehicles other than the vehicle within a preset range, and the road-side sensing data detected by the sensors of the road-side equipment set on the road. For example, the multiple sensor data may include image data and point cloud data collected by the vehicle's sensors, and may also include image data collected by sensors of other vehicles within 50 meters of the vehicle, and point cloud data collected by vehicle detection devices set beside the road.

[0060] It should be noted that in order to successfully obtain multiple sensor data, it is necessary to ensure that data can be transmitted between the vehicle and other vehicles, the cloud device and the road-side device, so that after the data fusion end is determined, the data fusion end can accurately obtain the multiple sensor data. For example, if the vehicle bird's-eye view construction method is applied to the cloud device, the cloud device can obtain the vehicle sensor data, other vehicle sensor data and road-side sensor data according to the communication protocol, obtain multiple sensor data, and perform subsequent processing steps based on the multiple sensor data.

[0061] S102, performing data fusion on the multiple sensor data to obtain an environmental perception result of the vehicle.

[0062] In some embodiments, the environmental perception result includes a target object in the driving environment of the vehicle. For example, there is a pedestrian in the right front of the driving environment of the vehicle. Due to the different information levels of the sensor data, when fusing multiple sensor data, different levels of fusion need to be performed according to the specific content of the sensor data.

[0063] For example, in one embodiment, the multiple sensor data are raw data collected by the sensor, then it is necessary to Figure 2 The flowchart shown fuses multiple sensor data, and the flowchart may include the following steps:

[0064] S201, when the multiple sensor data are raw data collected by sensors, feature extraction is performed on the multiple sensor data to obtain first feature data of the own vehicle sensor data, second feature data of the other vehicle sensor data and third feature data of the road end sensor data.

[0065] S202, performing feature fusion on the first feature data, the second feature data and the second feature data to obtain a first feature fusion result.

[0066] S203: Identify the target object in the driving environment of the vehicle according to the first feature fusion result to obtain the environment perception result.

[0067] The first feature data is feature data in the vehicle sensor data, and the first feature data may contain multiple types of data information, such as feature data of image data and feature data of point cloud data. Similarly, the second feature data and the third feature data also contain multiple types of data information, which are specifically determined according to the data types collected from the other vehicle sensor data and the roadside sensor data.

[0068] For the unprocessed raw data collected by the sensor, such as the original image data, the data fusion end needs to extract the vital signs of the raw data. The specific feature extraction process can be performed separately according to the type of raw data. For example, for the image data of the vehicle itself, a pre-trained deep learning model can be used to extract features to obtain the first image feature data of the vehicle's sensor data. For the point cloud data of the vehicle itself, PointNet (point cloud network, a deep learning model that specializes in processing point cloud data) can be used to extract features from the point cloud data to obtain the first point cloud feature data of the vehicle's sensor data. The first feature data can be constructed based on the first image feature data and the first point cloud feature data. For the sensor data of other vehicles and the road-side sensor data, the same method can be used to extract features to obtain the second feature data and the third feature data.

[0069] After obtaining the feature data, the first feature data, the second feature data and the third feature data can be feature fused. Specifically, the first feature data, the second feature data and the third feature data can be directly spliced ​​to form a unified input as the first feature fusion result, and then the first feature fusion result is input into the recognition model to identify the target object and determine the environmental perception result. When identifying the target object, the recognition model will integrate multiple feature data to determine the characteristics of the target object and finally determine the category of the target object. For example, the target object may be a pedestrian, a road sign, a stone, etc.

[0070] In another embodiment, the plurality of sensor data are feature data processed by the sensor. For example, the visual sensor can extract features from the collected image data to obtain image feature data. Figure 3 The flowchart shown fuses multiple sensor data, and the flowchart may include the following steps:

[0071] S301, when the plurality of sensor data are feature data, perform feature fusion on the plurality of sensor data to obtain a second feature fusion result.

[0072] S302: Identify the target object in the driving environment of the vehicle according to the second feature fusion result to obtain the environment perception result.

[0073] The feature data is the data obtained by the sensor after feature extraction of the collected raw data, for example, the image feature data obtained after the visual sensor of the vehicle extracts the features of the collected image data, or the point cloud feature data obtained after the laser radar of the road-side equipment extracts the features of the collected point cloud data. Because the sensor data obtained by the data fusion end is feature data, when performing feature fusion on multiple sensor data, feature fusion can be performed by weighting or splicing. In an optional embodiment, when performing feature fusion in a weighted manner, different weights can be assigned to the vehicle sensor data, other vehicle sensor data and road-side sensor data, for example, a weight of 0.4 can be assigned to the vehicle sensor data, and a weight of 0.3 can be assigned to the other vehicle sensor data and the road-side sensor data, respectively. This can make the feature fusion result integrate the feature data of multiple terminals, and there is a certain proportion bias, reduce the interference of a single feature data on the overall fusion result, and improve the accuracy of feature fusion. After the feature fusion is completed, the second feature fusion result can be obtained, and the second feature fusion result is input into the recognition model. The target object in the second feature fusion result is identified by the recognition model, and the type of the target object is determined, thereby obtaining the environmental perception result.

[0074] In an optional embodiment, the multiple sensor data may also be data information after target recognition by the vehicle, other vehicles or road-side equipment. Therefore, the data fusion end only needs to perform the final target recognition based on the data information of the target object in the multiple sensor data. The specific recognition process can be to vote, weight and average the data information of the target object in the multiple sensor data through the recognition model to obtain the final target object type and determine the environmental perception result.

[0075] S103: construct a bird's-eye view of the vehicle based on the environmental perception result.

[0076] In some embodiments, based on the image data collected by at least one of the vehicle sensors, other vehicle sensors and roadside sensors, a panoramic image of the surroundings of the vehicle is generated, and the panoramic image is converted into a bird's-eye view through perspective transformation and distortion correction algorithms to obtain an initial bird's-eye view. Furthermore, combined with the data information of the target object in the real-time detected environmental perception results, the type, position, and moving speed of the target object can be determined. According to the environmental perception results, the initial bird's-eye view is improved, and the annotation information of the target object is generated in the corresponding position of the bird's-eye view, so that the driver or vehicle can drive the vehicle safely according to the final bird's-eye view, thereby improving the safety of vehicle driving.

[0077] By integrating the sensor data of the vehicle, other vehicles and roadside sensors, the recognition area of ​​the vehicle is expanded, the blind spot of the vehicle perception is reduced, and the accuracy of the vehicle environment perception is improved. The vehicle bird's-eye view is constructed based on the perception results obtained by fusion of multiple features, which improves the accuracy of the bird's-eye view and thus improves the safety of vehicle driving.

[0078] In some embodiments, after the plurality of sensor data are fused to obtain the environment perception result of the vehicle, the perception result can also be sent to other vehicles to achieve data sharing. The specific sending process is as follows: Figure 4 The flowchart shown may include the following steps:

[0079] S401: If the category of the target object in the environmental perception result belongs to a preset category and / or the target object is a stationary object, determine that the target object is a long-term target object.

[0080] S402: When the target object is a long-term target object, store the environment perception result in a storage device.

[0081] In one embodiment, the target object in the environmental perception result can be further classified by the data fusion end to determine whether the target object will affect the driving of other vehicles other than the vehicle. Specifically, it can be determined that the category of the target object belongs to a preset category and / or that the target object is a stationary object. For example, according to the environmental perception result, the target object is determined to be a road construction sign, and the road construction sign is in the preset category, so it can be determined that the target object will affect the driving of other vehicles other than the vehicle. For another example, according to the environmental perception result, the moving speed of the target object can be determined. If the target object is an absolutely stationary object, it means that the target object will exist for a period of time in the future, and will affect the driving of other vehicles other than the vehicle. Therefore, after determining that the target object is a long-term target object that will affect the driving of other vehicles other than the vehicle, the environmental perception result can be stored in a storage device. The storage device can be a cloud device or a road-side device. After the environmental perception result is stored in the storage device, when other vehicles enter the current position or current road, the storage device can distribute the environmental perception result to other vehicles to improve the accuracy of the bird's-eye view constructed by other vehicles and improve the safety of vehicle driving.

[0082] In addition, after identifying the target object, the data fusion end can also send the environmental perception results including the target object to other vehicles within the preset range in real time through the communication protocol. For example, if the target object is a running child, after the feature fusion segment identifies the child, the environmental perception results are sent to the vehicle and other vehicles, so that the vehicles on the road can make accurate judgments and take emergency measures to reduce the probability of traffic accidents and improve the safety of vehicle driving.

[0083] In one embodiment, the method for constructing the vehicle bird's-eye view can refer to Figure 5 The flowchart shown is implemented, and the specific steps are as follows:

[0084] S501, multiple sensors sense the driving environment and collect multiple sensing data. The multiple sensors include own vehicle sensors, other vehicle sensors and roadside sensors.

[0085] S502, transmitting the plurality of sensor data to a data fusion terminal, which may be a vehicle terminal, another vehicle terminal, a cloud terminal, or a road terminal device.

[0086] S503, performing feature fusion on multiple sensor data through the data fusion terminal, identifying the target object, and obtaining the final environment perception result.

[0087] S504: Distribute the environmental perception results to the vehicle end or other vehicles through the data fusion end.

[0088] S505, displaying environmental information based on the environmental perception result at the own vehicle end or the other vehicle end, and specifically displaying the vehicle environment by constructing a bird's-eye view.

[0089] S506: Plan a vehicle driving strategy based on the environmental perception result.

[0090] S507, determining whether the target object in the environmental perception result is a long-term target that remains stationary for a period of time.

[0091] S508, if the target object is a long-term target, the target object is stored in the cloud or the road-side device; if the target object is not a long-term target, there is no need to store the target object.

[0092] S509, when the target object is a long-term target, if other vehicles enter the same area, the long-term target is sent to other vehicles entering through the cloud or road-side equipment, so that other vehicles can build a bird's-eye view based on the long-term target.

[0093] By integrating the sensor data of the vehicle, other vehicles and roadside sensors, the recognition area of ​​the vehicle is expanded, the blind spot of the vehicle perception is reduced, and the accuracy of the vehicle environment perception is improved. The vehicle bird's-eye view is constructed based on the perception results obtained by fusion of multiple features, which improves the accuracy of the bird's-eye view and thus improves the safety of vehicle driving.

[0094] Corresponding to the embodiment of the method for constructing a bird's-eye view of a vehicle described above, the present disclosure also provides an embodiment of an apparatus for constructing a bird's-eye view of a vehicle.

[0095] Please refer to Figure 6 , Figure 6 : is a block diagram of a device for constructing a bird's-eye view of a vehicle shown in an exemplary embodiment of the present disclosure, the device comprising:

[0096] An acquisition unit 601 is used to acquire a plurality of sensor data, wherein the plurality of sensor data includes self-vehicle sensor data, other-vehicle sensor data and road-end sensor data, wherein the self-vehicle sensor data is data detected by sensors of the self-vehicle, the other-vehicle sensor data is data detected by sensors of other vehicles other than the self-vehicle within a preset range, and the road-end sensor data is data detected by sensors of road-end equipment arranged on the driving road;

[0097] A fusion unit 602 is used to perform data fusion on the multiple sensor data to obtain an environmental perception result of the vehicle, wherein the environmental perception result includes a target object in the driving environment of the vehicle;

[0098] The construction unit 603 is used to construct a bird's-eye view of the vehicle according to the environmental perception result.

[0099] In the embodiment of the present disclosure, the fusion unit 602 is used to:

[0100] In the case where the plurality of sensor data are raw data collected by a sensor, feature extraction is performed on the plurality of sensor data to obtain first feature data of the self-vehicle sensor data, second feature data of the other-vehicle sensor data, and third feature data of the roadside sensor data;

[0101] Performing feature fusion on the first feature data, the second feature data and the second feature data to obtain a first feature fusion result;

[0102] According to the first feature fusion result, the target object in the driving environment of the vehicle is identified to obtain the environmental perception result.

[0103] In the embodiment of the present disclosure, the fusion unit 602 is used to:

[0104] In the case where the plurality of sensor data are feature data, feature fusion is performed on the plurality of sensor data to obtain a second feature fusion result; the feature data is data obtained after the sensor performs feature extraction on the collected raw data;

[0105] According to the second feature fusion result, the target object in the driving environment of the vehicle is identified to obtain the environmental perception result.

[0106] In the embodiment of the present disclosure, the fusion unit 602 is used to:

[0107] If the plurality of sensor data include data information of the target object, the environmental perception result is determined based on the data information of the target object in each sensor data.

[0108] In the embodiment of the present disclosure, the device further includes:

[0109] a determination unit, configured to determine that the target object is a long-term target object if the category of the target object in the environmental perception result belongs to a preset category and / or the target object is a stationary object;

[0110] The first sending unit is used to store the environmental perception result in a storage device when the target object is a long-term target object, so that the storage device sends the environmental perception result to other vehicles that enter the driving area of ​​the vehicle.

[0111] In the embodiment of the present disclosure, the device further includes:

[0112] The second sending unit is used to send the environmental perception result to the other vehicles, so that the other vehicles determine the driving path according to the environmental perception result.

[0113] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0114] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0115] The embodiment of the vehicle bird's-eye view construction device disclosed in the present invention can be applied to the data fusion terminal. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of the data fusion terminal where it is located reading the corresponding computer program instructions in the non-volatile memory into the memory and running them. From the hardware level, if Figure 7 As shown, it is a hardware structure diagram of the data fusion end where the vehicle bird's-eye view construction device of the present disclosure is located, except Figure 7 In addition to the processor, memory, network interface, and non-volatile memory shown, the data fusion end where the device is located in the embodiment may also include other hardware according to the actual function of the data fusion end, which will not be described in detail.

[0116] Based on the same inventive concept, the present disclosure also provides a computer-readable storage medium, and when the program is executed by a processor, the steps of the method for constructing a vehicle bird's-eye view are implemented.

[0117] Based on the same inventive concept, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for constructing a bird's-eye view of the vehicle when executing the program.

[0118] Based on the same inventive concept, the present disclosure also provides a computer program product, which implements the steps of the method for constructing a bird's-eye view of the vehicle when the program product is executed by a processor.

[0119] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0120] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A method for constructing a vehicle bird's-eye view, characterized in that: The method comprises: Acquire multiple sensor data, wherein the multiple sensor data include own vehicle sensor data, other vehicle sensor data and roadside sensor data, the own vehicle sensor data is data detected by the sensors of the own vehicle, the other vehicle sensor data is data detected by sensors of other vehicles other than the own vehicle within a preset range, and the roadside sensor data is data detected by sensors of roadside equipment arranged on the driving road; Performing data fusion on the multiple sensor data to obtain an environmental perception result of the vehicle, wherein the environmental perception result includes a target object in the driving environment of the vehicle; A bird's-eye view of the vehicle is constructed based on the environmental perception result.

2. The method according to claim 1, characterized in that The step of fusing the plurality of sensor data to obtain the environmental perception result of the vehicle includes: In the case where the plurality of sensor data are raw data collected by a sensor, feature extraction is performed on the plurality of sensor data to obtain first feature data of the self-vehicle sensor data, second feature data of the other-vehicle sensor data, and third feature data of the roadside sensor data; Performing feature fusion on the first feature data, the second feature data and the second feature data to obtain a first feature fusion result; According to the first feature fusion result, the target object in the driving environment of the vehicle is identified to obtain the environmental perception result.

3. The method according to claim 1, characterized in that The step of fusing the plurality of sensor data to obtain the environmental perception result of the vehicle includes: In the case where the plurality of sensor data are feature data, feature fusion is performed on the plurality of sensor data to obtain a second feature fusion result; the feature data is data obtained after the sensor performs feature extraction on the collected raw data; According to the second feature fusion result, the target object in the driving environment of the vehicle is identified to obtain the environmental perception result.

4. The method according to claim 1, characterized in that: The step of fusing the plurality of sensor data to obtain the environmental perception result of the vehicle includes: If the plurality of sensor data include data information of the target object, the environment perception result is determined based on the data information of the target object in each sensor data.

5. The method according to claim 1, characterized in that After fusing the plurality of sensor data to obtain the environment perception result of the vehicle, the method further includes: If the category of the target object in the environmental perception result belongs to a preset category and / or the target object is a stationary object, determining that the target object is a long-term target object; In the case where the target object is a long-term target object, the environment perception result is stored in a storage device, so that the storage device sends the environment perception result to other vehicles that enter the driving area of ​​the own vehicle.

6. The method according to claim 1, characterized in that After fusing the plurality of sensor data to obtain the environment perception result of the vehicle, the method further includes: The environmental perception result is sent to the other vehicles, so that the other vehicles determine a driving path according to the environmental perception result.

7. A device for constructing a bird's-eye view of a vehicle, characterized in that: The device comprises: an acquisition unit, configured to acquire a plurality of sensor data, wherein the plurality of sensor data include self-vehicle sensor data, other-vehicle sensor data and road-side sensor data, the self-vehicle sensor data is data detected by sensors of the self-vehicle, the other-vehicle sensor data is data detected by sensors of other vehicles other than the self-vehicle within a preset range, and the road-side sensor data is data detected by sensors of road-side equipment arranged on the driving road; A fusion unit, configured to fuse the plurality of sensor data to obtain an environmental perception result of the vehicle, wherein the environmental perception result includes a target object in the driving environment of the vehicle; A construction unit is used to construct a bird's-eye view of the vehicle according to the environmental perception result.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that When the program product is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images