Target fusion method and device, electronic equipment and computer program product

Abstract

The invention discloses a target fusion method and device, electronic equipment and a computer program product. The target fusion method comprises the steps that an initial target fusion result of an automatic driving vehicle is acquired; the initial target fusion result is matched with a preset semantic map, a semantic map matching result is obtained, and the semantic map matching result comprises a background target matching result; and updating the initial target fusion result according to the semantic map matching result to obtain an updated target fusion result. According to the target fusion method provided by the embodiment of the invention, on the basis of an original multi-sensor fusion framework, a layer of post-processing link of semantic map matching is added, and the initial target fusion result of the multiple sensors is matched with the background target information in the semantic map; interference of background target information in the initial target fusion result can be further filtered out, the accuracy of the target fusion result is improved, and then the safety of automatic driving and user experience are improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a target fusion method, device, electronic equipment, and computer program product. Background Art

[0002] In the rapid development of contemporary automotive technology, the integration of assisted driving and even autonomous driving functions has become a trend that cannot be ignored, greatly improving the safety and convenience of driving. Among them, object detection technology, as a core component of the autonomous driving system, plays a vital role. This technology can be divided into two major areas: foreground object detection and background object detection. The latter focuses on the recognition of static objects, which are often the cause of unnecessary braking operations of autonomous vehicles.

[0003] At present, mainstream target detection methods are deeply rooted in deep learning algorithms and are usually supplemented by complex post-processing processes in order to achieve high-precision target identification. However, this solution that is highly dependent on computing resources is not only costly, but also faces severe challenges in the flexibility and scalability of the model when facing complex and changing road environments. Specifically, deep learning models have limited adaptability to dynamically changing road scenes, and it is difficult to ensure that foreground and background targets can be accurately distinguished in all situations.

[0004] What is even more difficult is that due to map accuracy limitations and update lags, the effective filtering of background information has become a major problem in the traditional map data matching method. This not only affects the decision-making accuracy of the autonomous driving system, but may also cause the system to frequently request manual takeover or perform unnecessary braking operations (such as braking), seriously weakening the user experience and safety of autonomous driving. Therefore, developing a target fusion solution that can efficiently and accurately identify and filter background targets while having good environmental adaptability and model scalability has become a key technical problem that needs to be solved in the field of autonomous driving. Summary of the invention

[0005] The embodiments of the present application provide a target fusion method, device, electronic device, and computer program product to improve the accuracy of target fusion, thereby improving the safety and user experience of autonomous driving.

[0006] The present application embodiment adopts the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a target fusion method, wherein the target fusion method includes:

[0008] Obtain the initial target fusion results of the autonomous driving vehicle;

[0009] Matching the initial target fusion result with a preset semantic map to obtain a semantic map matching result, wherein the semantic map matching result includes a background target matching result;

[0010] The initial target fusion result is updated according to the semantic map matching result to obtain an updated target fusion result.

[0011] Optionally, the obtaining of the initial target fusion result of the autonomous driving vehicle includes:

[0012] Obtain object detection results from multiple sensors on an autonomous vehicle;

[0013] The target detection results of multiple sensors are fused using a preset target fusion strategy to obtain the initial target fusion result.

[0014] Optionally, the target detection result includes multi-dimensional target information, the multi-dimensional target information includes at least two of a target detection frame, a target speed, a target heading angle, and a relative distance between the target and the autonomous driving vehicle, and the target detection results of multiple sensors are fused using a preset target fusion strategy to obtain the initial target fusion result, including:

[0015] According to the multi-dimensional target information of multiple sensors, weighted matching is performed on the target detection results of each pair of sensors in the multiple sensors to obtain the target matching weights of each pair of sensors;

[0016] According to the target matching weights of two sensors among the multiple sensors, the matching is optimized by using an optimization algorithm to obtain target matching results of two sensors among the multiple sensors;

[0017] Target fusion is performed according to the target matching results of two sensors among the multiple sensors to obtain the initial target fusion result.

[0018] Optionally, performing weighted matching on target detection results of pairs of sensors in the multiple sensors according to multi-dimensional target information of the multiple sensors to obtain target matching weights of pairs of sensors includes:

[0019] Calculate multi-dimensional target difference information of each of the multiple sensors according to the multi-dimensional target information of the multiple sensors, wherein the target difference information includes at least two of an intersection-and-joint ratio of target detection frames, a target speed difference, a target type difference, a target heading angle difference, and a relative distance difference between the target and the autonomous driving vehicle;

[0020] Weighted matching is performed on multi-dimensional target difference information of each pair of sensors among multiple sensors to obtain target matching weights of each pair of sensors.

[0021] Optionally, the target fusion method further includes:

[0022] Get the target fusion result of the previous frame;

[0023] The target fusion result of the previous frame and the target fusion result of the current frame are fused using the preset target fusion strategy to obtain the target tracking result.

[0024] Optionally, matching the initial target fusion result with a preset semantic map to obtain a semantic map matching result includes:

[0025] Get the current location information of the autonomous vehicle;

[0026] Acquire a corresponding local semantic map according to the current position information;

[0027] The initial target fusion result is matched with the corresponding local semantic map to obtain a semantic map matching result.

[0028] Optionally, the preset semantic map is a background target semantic map, and the background target semantic map is constructed in the following manner:

[0029] Acquire road perception information collected by sensors and perform background target detection to obtain background target detection results;

[0030] Constructing the background target semantic map according to the background target detection result;

[0031] When a map update condition is triggered, the background object semantic map is updated.

[0032] In a second aspect, an embodiment of the present application further provides a target fusion device, wherein the target fusion device comprises:

[0033] A first acquisition unit, used to acquire an initial target fusion result of the autonomous driving vehicle;

[0034] A matching unit, used for matching the initial target fusion result with a preset semantic map to obtain a semantic map matching result, wherein the semantic map matching result includes a background target matching result;

[0035] An updating unit is used to update the initial target fusion result according to the semantic map matching result to obtain an updated target fusion result.

[0036] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0037] A processor; and a memory arranged to store computer executable instructions, which, when executed, cause the processor to perform any of the target fusion methods described above.

[0038] In a fourth aspect, an embodiment of the present application further provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements any of the aforementioned target fusion methods.

[0039] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: the target fusion method in the embodiments of the present application first obtains the initial target fusion result of the autonomous driving vehicle; then matches the initial target fusion result with the preset semantic map to obtain a semantic map matching result, and the semantic map matching result includes a background target matching result; finally, the initial target fusion result is updated according to the semantic map matching result to obtain an updated target fusion result. The target fusion method in the embodiments of the present application adds a layer of post-processing of semantic map matching on the basis of the original multi-sensor fusion framework. By matching the initial target fusion result of the multi-sensor with the background target information in the semantic map, the interference of the background target information in the initial target fusion result can be further filtered out, thereby improving the accuracy of the target fusion result, and thus improving the safety and user experience of autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 A schematic diagram of a target fusion method in an embodiment of the present application;

[0042] Figure 2 This is a schematic diagram of the structure of a target fusion device in an embodiment of the present application;

[0043] Figure 3 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0045] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0046] The present application embodiment provides a target fusion method, such as Figure 1 As shown, a flow chart of a target fusion method in an embodiment of the present application is provided, and the target fusion method at least includes the following steps S110 to S130:

[0047] Step S110, obtaining the initial target fusion result of the autonomous driving vehicle.

[0048] When performing target fusion in the embodiments of the present application, it is necessary to first obtain the initial target fusion result of the autonomous driving vehicle. The initial target fusion result refers to the initial result obtained after fusing the target detection results of multiple sensors on the autonomous driving vehicle. The multiple sensors here may include, for example, cameras, lidars, millimeter-wave radars, etc. The specific way of fusing the target detection results of multiple sensors can be flexibly set by those skilled in the art in combination with the existing technology and is not specifically limited here.

[0049] Step S120, matching the initial target fusion result with a preset semantic map to obtain a semantic map matching result, wherein the semantic map matching result includes a background target matching result.

[0050] In the embodiment of the present application, a semantic map is constructed in advance for background targets existing in the road. The semantic map includes semantic information of background targets in the road. The initial target fusion result obtained in the above steps is matched with the constructed semantic map to obtain a semantic map matching result. The semantic map matching result may specifically include a background target matching result. The background target matching result reflects whether the initial target fusion result includes a fusion result belonging to the background target.

[0051] The background target of the embodiment of the present application is a concept corresponding to the foreground target. The foreground target is mainly objects that can move on the road surface, such as vehicles, pedestrians and other targets, while the background target mainly refers to immovable objects on the road surface, such as vegetation, guardrails and the like.

[0052] Step S130, updating the initial target fusion result according to the semantic map matching result to obtain an updated target fusion result.

[0053] Based on the above background target matching results, the initial target fusion results can be optimized and updated, and the background target information contained in the initial target fusion results can be filtered out, thereby avoiding the impact of background target information on the safety and user experience of autonomous driving.

[0054] The target fusion method of the embodiment of the present application adds a layer of post-processing of semantic map matching on the basis of the original multi-sensor fusion framework. By matching the initial target fusion results of the multiple sensors with the background target information in the semantic map, the interference of the background target information in the initial target fusion results can be further filtered out, thereby improving the accuracy of the target fusion results, and thereby improving the safety and user experience of autonomous driving.

[0055] In some embodiments of the present application, obtaining the initial target fusion result of the autonomous driving vehicle includes: obtaining target detection results of multiple sensors on the autonomous driving vehicle; fusing the target detection results of multiple sensors using a preset target fusion strategy to obtain the initial target fusion result.

[0056] The embodiment of the present application defines that the autonomous driving vehicle has been equipped with sensors such as lidar, camera, millimeter-wave radar, etc., and the parameters are adjusted so that relevant perception information of the surrounding environment can be obtained in real time. When obtaining the initial target fusion result of the autonomous driving vehicle, the road perception information of multiple sensors such as camera, lidar, millimeter-wave radar can be obtained separately and target detection can be performed to obtain the target detection results of multiple sensors. Then, a certain target fusion strategy can be adopted to perform target-level matching and fusion on the target detection results of multiple sensors to obtain the initial target-level fusion result.

[0057] The above-mentioned target fusion strategy can adopt the traditional matching fusion method based on Euclidean distance, or the multi-dimensional matching fusion method of the following embodiment. Of course, those skilled in the art can also flexibly adopt other fusion strategies according to actual needs, which are not specifically limited here.

[0058] In some embodiments of the present application, the target detection result includes multi-dimensional target information, and the multi-dimensional target information includes at least two of a target detection box, a target speed, a target heading angle, and a relative distance between the target and the autonomous driving vehicle. The target detection results of multiple sensors are fused using a preset target fusion strategy to obtain the initial target fusion result, including: performing weighted matching on the target detection results of pairs of sensors in the multiple sensors according to the multi-dimensional target information of the multiple sensors to obtain pairwise target matching weights; performing matching optimization using an optimization algorithm according to the pairwise target matching weights of the multiple sensors to obtain pairwise target matching results of the multiple sensors; and performing target fusion according to the pairwise target matching results of the multiple sensors to obtain the initial target fusion result.

[0059] The target fusion of the embodiment of the present application is to fuse the target information of sensors from different sources. The current fusion method mainly adopts the traditional Euclidean distance calculation method. This method is relatively simple, but in complex scenarios such as mixed traffic of people and vehicles, it is easy to have mis-association matching, which leads to deviation in target calculation, thus affecting the experience of autonomous driving and bringing certain safety hazards to autonomous driving. Even some mis-association matching causes the autonomous driving vehicle to brake suddenly, causing a certain degree of harm to pedestrians on the vehicle.

[0060] Based on this, the embodiment of the present application adopts a multi-dimensional target information matching method to replace or optimize the existing target association matching scheme, thereby improving the accuracy of multi-sensor fusion target matching and reducing the problems of somatosensory and takeover caused by mismatching. Specifically, the target detection results of the embodiment of the present application may include multi-dimensional target information such as target detection frame, target speed, target heading angle, and the relative distance between the target and the autonomous driving vehicle. When performing target fusion, target-level weighted matching can be performed based on the multi-dimensional target information of any two sensors. For example, it may include matching of target detection frame, matching of speed, matching of heading angle, matching of relative distance between target and autonomous driving vehicle, etc., so as to obtain the target-level matching weight. The size of the matching weight reflects the association strength between the two targets to a certain extent, that is, the larger the matching weight, the greater the target association strength.

[0061] Finally, the target-level matching results of the above multiple dimensions are integrated, and optimization algorithms such as the Hungarian algorithm are used to solve the optimal target-level association matching result and perform target fusion, so as to obtain the initial target-level fusion result between multiple sensors.

[0062] In some embodiments of the present application, weighted matching is performed on target detection results of pairs of sensors among the multiple sensors based on multi-dimensional target information of the multiple sensors to obtain pairwise target matching weights, including: calculating multi-dimensional target difference information of pairs of sensors among the multiple sensors based on multi-dimensional target information of the multiple sensors, the target difference information including at least two of the intersection-and-joint ratio of target detection frames, target speed difference, target type difference, target heading angle difference, and relative distance difference between the target and the autonomous driving vehicle; weighted matching is performed on multi-dimensional target difference information of pairs of sensors among the multiple sensors to obtain pairwise target matching weights.

[0063] When performing weighted matching based on the multi-dimensional target information of two sensors, the multi-dimensional target difference information of the two sensors can be calculated respectively. For example, for cameras and lidars, the intersection ratio of any two target detection frames, the difference in target speeds, the difference in target categories, the difference in target heading angles, and the difference in relative distances between the target and the autonomous driving vehicle can be calculated.

[0064] Different weight coefficients can be set for the target difference information of different dimensions, and then weighted sum is performed according to the target difference information of different dimensions and the corresponding weight coefficients to obtain the target matching weight of the target level. For example, it can be expressed as follows:

[0065] Final_weight=weight1*IOU+weight2*velocity+weight3*type+weight4*heading+weight5*distance

[0066] Among them, Final_weight is the matching weight of the two targets, weight1 is the weight coefficient of the intersection ratio, weight2 is the weight coefficient of the speed difference, weight3 is the weight coefficient of the type difference, weight4 is the weight coefficient of the heading angle difference, and weight5 is the weight coefficient of the relative distance difference. The sizes of weight1, weight2, weight3, weight4, and weight5 can be customized according to actual needs.

[0067] It should be noted that since the target information dimensions obtained by different types of sensors are not exactly the same, for example, cameras and lidars can both detect the information of the target detection frame, but millimeter-wave radars cannot obtain the target detection frame information. Therefore, when weighted matching is performed based on the above method, for a target information dimension that a sensor does not have, the target difference information of that dimension can be directly set to 0, which does not affect the weighted calculation of other dimensional information.

[0068] In some embodiments of the present application, the target fusion method further includes: obtaining a target fusion result of a previous frame; and fusing the target fusion result of the previous frame and the target fusion result of the current frame using a preset target fusion strategy to obtain a target tracking result.

[0069] The above-mentioned target fusion strategy of the embodiment of the present application can not only be applied to target fusion between multiple sensors, but also be applied to target fusion between adjacent time frames after multi-sensor target fusion, thereby realizing target fusion and tracking in the time dimension.

[0070] Specifically, when performing target fusion in the time dimension, the target fusion result of the previous frame and the target fusion result of the current frame can be obtained first. The target fusion result of the previous frame here can refer to the target fusion result of the previous frame after being updated using the semantic map in the aforementioned embodiment, and the target fusion result of the current frame can refer to the target fusion result of the current frame after being updated using the semantic map. Then, the target fusion result of the previous frame is matched and fused with the result of multi-sensor target fusion of the current frame using the weighted matching method of the above-mentioned multi-dimensional target information, so as to obtain the target tracking results of adjacent frames.

[0071] In some embodiments of the present application, matching the initial target fusion result with a preset semantic map to obtain a semantic map matching result includes: obtaining current position information of the autonomous driving vehicle; obtaining a corresponding local semantic map based on the current position information; matching the initial target fusion result with the corresponding local semantic map to obtain a semantic map matching result.

[0072] Since the semantic map is a global map constructed for the entire operating road scene of the autonomous vehicle, if the global semantic map is loaded for each match, the matching calculation amount will be large and the matching time will be too long, which will affect the real-time performance of the match. Therefore, when matching the initial target fusion result with the preset semantic map, the embodiment of the present application can first obtain the local semantic map data within the area corresponding to the current position according to the current position information of the autonomous vehicle, match the initial target fusion result with the local semantic map, and obtain the semantic map matching result, thereby reducing the matching calculation amount and improving the matching efficiency.

[0073] In some embodiments of the present application, the preset semantic map is a background target semantic map, and the background target semantic map is constructed in the following manner: acquiring road perception information collected by sensors and performing background target detection to obtain background target detection results; constructing the background target semantic map based on the background target detection results; and updating the background target semantic map when a map update condition is triggered.

[0074] The embodiments of the present application can construct a background target semantic map based on road perception information collected by sensors such as cameras or lidars. Taking lidar as an example, in the road area where the semantic map is to be constructed, road point cloud data collected by the lidar is obtained, and then background target detection is performed on the road point cloud data to obtain background target detection results. The detection of background targets can be combined with existing target detection model training to obtain a background target detection model, which will not be elaborated here.

[0075] According to the above background target detection results, the semantic information of the background target can be obtained, and mapped to the spatial map to obtain the background target semantic map. According to the changes of the background target in the actual road scene, the background target semantic map can be dynamically updated to ensure the accuracy of the background target semantic map.

[0076] The present application embodiment also provides a target fusion device 200, such as Figure 2 As shown, a schematic diagram of the structure of a target fusion device in an embodiment of the present application is provided, wherein the target fusion device 200 at least includes: a first acquisition unit 210, a matching unit 220 and an updating unit 230, wherein:

[0077] A first acquisition unit 210, used to acquire an initial target fusion result of the autonomous driving vehicle;

[0078] A matching unit 220, configured to match the initial target fusion result with a preset semantic map to obtain a semantic map matching result, wherein the semantic map matching result includes a background target matching result;

[0079] The updating unit 230 is used to update the initial target fusion result according to the semantic map matching result to obtain an updated target fusion result.

[0080] In some embodiments of the present application, the first acquisition unit 210 is specifically used to: obtain target detection results of multiple sensors on the autonomous driving vehicle; fuse the target detection results of multiple sensors using a preset target fusion strategy to obtain the initial target fusion result.

[0081] In some embodiments of the present application, the target detection result includes multi-dimensional target information, and the multi-dimensional target information includes at least two of a target detection frame, a target speed, a target heading angle, and a relative distance between the target and the autonomous driving vehicle. The first acquisition unit 210 is specifically used to: perform weighted matching on the target detection results of each pair of sensors in the multiple sensors according to the multi-dimensional target information of the multiple sensors to obtain the target matching weights of each pair of sensors; perform matching optimization using an optimization algorithm according to the target matching weights of each pair of sensors in the multiple sensors to obtain the target matching results of each pair of sensors in the multiple sensors; perform target fusion according to the target matching results of each pair of sensors in the multiple sensors to obtain the initial target fusion result.

[0082] In some embodiments of the present application, the first acquisition unit 210 is specifically used to: calculate multi-dimensional target difference information of each pair of sensors in the multiple sensors based on multi-dimensional target information of the multiple sensors, the target difference information including at least two of the intersection-and-union ratio of the target detection frame, the target speed difference, the target type difference, the target heading angle difference, and the relative distance difference between the target and the autonomous driving vehicle; perform weighted matching on the multi-dimensional target difference information of each pair of sensors in the multiple sensors, respectively, to obtain the target matching weights of the each pair of sensors.

[0083] In some embodiments of the present application, the target fusion device 200 also includes: a second acquisition unit, used to obtain the target fusion result of the previous frame; a fusion unit, used to fuse the target fusion result of the previous frame and the target fusion result of the current frame using a preset target fusion strategy to obtain a target tracking result.

[0084] In some embodiments of the present application, the matching unit 220 is specifically used to: obtain the current position information of the autonomous driving vehicle; obtain the corresponding local semantic map based on the current position information; match the initial target fusion result with the corresponding local semantic map to obtain a semantic map matching result.

[0085] In some embodiments of the present application, the preset semantic map is a background target semantic map, and the background target semantic map is constructed in the following manner: acquiring road perception information collected by sensors and performing background target detection to obtain background target detection results; constructing the background target semantic map based on the background target detection results; and updating the background target semantic map when a map update condition is triggered.

[0086] It can be understood that the above-mentioned target fusion device can implement each step of the target fusion method provided in the above-mentioned embodiment, and the relevant explanations about the target fusion method are applicable to the target fusion device and will not be repeated here.

[0087] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 3 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0088] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0089] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0090] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming the target fusion device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0091] Obtain the initial target fusion results of the autonomous driving vehicle;

[0092] Matching the initial target fusion result with a preset semantic map to obtain a semantic map matching result, wherein the semantic map matching result includes a background target matching result;

[0093] The initial target fusion result is updated according to the semantic map matching result to obtain an updated target fusion result.

[0094] The above application Figure 1The method performed by the target fusion device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in software form. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in a decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0095] The present application also provides a computer program product, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The method performed by the target fusion device in the illustrated embodiment is specifically used to perform:

[0096] Obtain the initial target fusion results of the autonomous driving vehicle;

[0097] Matching the initial target fusion result with a preset semantic map to obtain a semantic map matching result, wherein the semantic map matching result includes a background target matching result;

[0098] The initial target fusion result is updated according to the semantic map matching result to obtain an updated target fusion result.

[0099] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0100] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0101] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0103] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0104] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0105] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0106] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0107] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0108] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A target fusion method, wherein: The target fusion method comprises: Obtain the initial target fusion results of the autonomous driving vehicle; Matching the initial target fusion result with a preset semantic map to obtain a semantic map matching result, wherein the semantic map matching result includes a background target matching result; The initial target fusion result is updated according to the semantic map matching result to obtain an updated target fusion result.

2. The target fusion method according to claim 1, wherein: The obtaining of the initial target fusion result of the autonomous driving vehicle includes: Obtain object detection results from multiple sensors on an autonomous vehicle; The target detection results of multiple sensors are fused using a preset target fusion strategy to obtain the initial target fusion result.

3. The target fusion method according to claim 2, wherein: The target detection result includes multi-dimensional target information, and the multi-dimensional target information includes at least two of a target detection frame, a target speed, a target heading angle, and a relative distance between the target and the autonomous driving vehicle. The target detection results of multiple sensors are fused using a preset target fusion strategy to obtain the initial target fusion result, which includes: According to the multi-dimensional target information of multiple sensors, weighted matching is performed on the target detection results of each pair of sensors in the multiple sensors to obtain the target matching weights of each pair of sensors; According to the target matching weights of two sensors among the multiple sensors, the matching is optimized by using an optimization algorithm to obtain target matching results of two sensors among the multiple sensors; Target fusion is performed according to the target matching results of two sensors among the multiple sensors to obtain the initial target fusion result.

4. The target fusion method according to claim 3, wherein: According to the multi-dimensional target information of the multiple sensors, weighted matching is performed on the target detection results of the multiple sensors in pairs to obtain the target matching weights of the two sensors in pairs, which includes: Calculate multi-dimensional target difference information of each of the multiple sensors according to the multi-dimensional target information of the multiple sensors, wherein the target difference information includes at least two of an intersection-and-joint ratio of target detection frames, a target speed difference, a target type difference, a target heading angle difference, and a relative distance difference between the target and the autonomous driving vehicle; Weighted matching is performed on multi-dimensional target difference information of each pair of sensors among multiple sensors to obtain target matching weights of each pair of sensors.

5. The target fusion method according to claim 1, wherein: The target fusion method also includes: Get the target fusion result of the previous frame; The target fusion result of the previous frame and the target fusion result of the current frame are fused using the preset target fusion strategy to obtain the target tracking result.

6. The target fusion method according to claim 1, wherein: The matching of the initial target fusion result with the preset semantic map to obtain the semantic map matching result includes: Get the current location information of the autonomous vehicle; Acquire a corresponding local semantic map according to the current position information; The initial target fusion result is matched with the corresponding local semantic map to obtain a semantic map matching result.

7. The target fusion method according to any one of claims 1 to 6, wherein: The preset semantic map is a background target semantic map, and the background target semantic map is constructed in the following way: Acquire road perception information collected by sensors and perform background target detection to obtain background target detection results; Constructing the background target semantic map according to the background target detection result; When a map update condition is triggered, the background object semantic map is updated.

8. A target fusion device, wherein: The target fusion device comprises: A first acquisition unit, used to acquire an initial target fusion result of the autonomous driving vehicle; A matching unit, used for matching the initial target fusion result with a preset semantic map to obtain a semantic map matching result, wherein the semantic map matching result includes a background target matching result; An updating unit is used to update the initial target fusion result according to the semantic map matching result to obtain an updated target fusion result.

9. An electronic device, comprising: processor; and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the target fusion method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program or an instruction, wherein when the computer program or the instruction is executed by a processor, the target fusion method according to any one of claims 1 to 7 is implemented.

Images