Method and device for creating a target vehicle based on multi-sensor fusion

Abstract

This invention provides a method and apparatus for creating a target vehicle based on multi-sensor fusion. The method includes: using target vehicle perception information from millimeter-wave radar and a camera to determine if a target vehicle at the same distance and speed exists ahead; if so, determining whether the millimeter-wave radar target vehicle is lost or misdetected; and if so, fitting a new millimeter-wave radar target vehicle. This invention solves the problem of inability to fuse radar and camera target vehicles due to missed or incorrect detections of long-range, same-speed, same-distance target vehicles by fitting a new radar target vehicle when a target vehicle at the same distance and speed exists ahead but millimeter-wave radar target vehicle information is missing.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, and in particular to a method and apparatus for creating a detection target vehicle based on multi-sensor fusion. Background Technology

[0002] In the field of intelligent driving, target vehicle fusion using cameras and millimeter-wave radar currently relies primarily on the simultaneous presence of both millimeter-wave radar and camera-based target vehicles. However, when a target vehicle at the same speed and distance exists at a long distance, millimeter-wave radar may miss or misdetect the target vehicle, preventing the camera and radar target vehicles from being fused. Therefore, developing a method and device for creating detectable target vehicles based on multi-sensor fusion, which can effectively overcome the shortcomings of the aforementioned technologies, has become a pressing technical problem for the industry. Summary of the Invention

[0003] To address the aforementioned problems in the existing technology, embodiments of the present invention provide a method and apparatus for creating a detection target vehicle based on multi-sensor fusion.

[0004] In a first aspect, embodiments of the present invention provide a method for creating a target vehicle based on multi-sensor fusion, comprising: a perception module outputting target vehicle perception information from millimeter-wave radar and a camera respectively; a tracking module associating the target vehicle perception information input by the perception module with the target vehicle perception information of the previous frame, and performing target vehicle tracking prediction, monitoring the movement trend of the target vehicle, and determining whether there is a target vehicle at the same distance and speed ahead; if there is a similar target vehicle, the tracking module further determines whether the longitudinal distance difference and lateral distance difference of the similar target vehicles meet the requirements; if the requirements are met, it determines whether the difference in the target vehicle's motion vector and the difference in the target vehicle's motion speed meet the requirements; if the requirements are met, it is determined that there is a target vehicle at the same distance and speed ahead, and the target vehicle creation module is activated; the target vehicle creation module determines whether the millimeter-wave radar target vehicle is lost or discontinued, and if so, the target vehicle creation module fits a new millimeter-wave radar target vehicle; and the new millimeter-wave radar target vehicle is fused with the target vehicle acquired by the camera using the Dempster-Shafer algorithm.

[0005] Based on the above method embodiments, the target vehicle creation method based on multi-sensor fusion provided in this embodiment of the invention includes the target vehicle perception information of the millimeter-wave radar and camera, including: target vehicle ID, lateral / longitudinal distance, lateral / longitudinal velocity, and target vehicle acceleration.

[0006] Based on the above method embodiments, the multi-sensor fusion-based target vehicle creation method provided in this embodiment of the invention, wherein the condition of determining whether there is a target vehicle at the same distance and speed ahead includes: whether there is a similar target vehicle whose straight-line distance difference satisfies the condition a≤b for the target vehicle being stably tracked by the camera; wherein a is the distance difference between the two target vehicles, and b is the determination threshold.

[0007] Based on the above method embodiments, the multi-sensor fusion-based target vehicle creation method provided in this embodiment of the invention further determines whether the longitudinal distance difference and lateral distance difference of similar target vehicles meet the requirements, including: whether x≤m and y≤n are satisfied, where the values ​​of m and n are adjusted according to the longitudinal distance attribute of the target vehicle; where x is the longitudinal distance difference between the two targets, y is the lateral distance difference between the two targets, m is the lateral distance difference threshold for determining that the two targets are in a scenario with the same distance and speed, and n is the longitudinal distance difference threshold for determining that the two targets are in a scenario with the same distance and speed.

[0008] Based on the above method embodiments, the target vehicle creation method based on multi-sensor fusion provided in this embodiment of the invention determines whether the target vehicle motion vector difference and the target vehicle motion speed difference meet the requirements, including whether g≤j and V≤k, where j and k are calibrated and adjusted according to the actual situation; where g is the motion vector difference between the two target vehicles; v is the longitudinal speed difference between the two target vehicles; j is the vector difference threshold for determining that the two targets are at the same distance and speed; and k is the speed difference threshold for determining that the two target vehicles are at the same distance and speed.

[0009] Based on the above method embodiments, the present invention provides a method for creating a target vehicle based on multi-sensor fusion. The target vehicle creation module determines whether a millimeter-wave radar target vehicle is lost or discontinued. This includes: if at least one of the two similar target vehicles output by the camera does not have a corresponding millimeter-wave radar matching target vehicle, and the number of frames of the missing millimeter-wave radar matching target vehicle exceeds a predetermined frame threshold, then the millimeter-wave radar target vehicle is determined to be lost or discontinued.

[0010] Based on the above method embodiments, the multi-sensor fusion-based target vehicle creation method provided in this embodiment of the invention includes the new millimeter-wave radar target vehicle attribute information, which includes: the lateral attribute information of the target without associated radar selected from the camera target, and the longitudinal attribute information of the millimeter-wave radar target; wherein, the speed information, longitudinal distance information, and motion direction information of the new millimeter-wave radar target vehicle come from the millimeter-wave radar, and the lateral distance information and lateral speed information come from the camera.

[0011] Secondly, embodiments of the present invention provide a target vehicle creation device based on multi-sensor fusion, comprising: a first main module for implementing step 1, whereby the perception module outputs target vehicle perception information from millimeter-wave radar and camera respectively; a second main module for implementing step 2, whereby the tracking module associates the target vehicle perception information input from the perception module with the target vehicle perception information from the previous frame, performs target vehicle tracking prediction, monitors the movement trend of the target vehicle, and determines whether there is a target vehicle at the same distance and speed ahead; and a third main module for implementing step 3, whereby if similar target vehicles exist, the tracking module further determines whether the longitudinal distance difference and lateral distance difference of the similar target vehicles meet the requirements. The fourth main module is used to implement step 4. If the requirements of step 3 are met, it determines whether the difference in the target vehicle's motion vector and the difference in the target vehicle's motion speed meet the requirements. The fifth main module is used to implement step 5. If the requirements of step 4 are met, it is determined that there is a target vehicle at the same distance and speed ahead, and the target vehicle creation module is activated. The sixth main module is used to implement step 6. The target vehicle creation module determines whether the millimeter-wave radar target vehicle is lost or discontinued. If it is lost or discontinued, the target vehicle creation module fits a new millimeter-wave radar target vehicle. The seventh main module is used to implement step 7. The new millimeter-wave radar target vehicle is fused with the target vehicle acquired by the camera using the Dempster-Shafer algorithm.

[0012] Thirdly, embodiments of the present invention provide an electronic device, comprising:

[0013] At least one processor; and

[0014] At least one memory communicatively connected to the processor, wherein:

[0015] The memory stores program instructions that can be executed by the processor. The processor can call the program instructions to execute the multi-sensor fusion-based target vehicle creation method provided by any of the various implementations of the first aspect.

[0016] Fourthly, embodiments of the present invention provide a non-transitory computer-readable storage medium storing computer instructions that cause a computer to execute the multi-sensor fusion-based target vehicle creation method provided in any of the various implementations of the first aspect.

[0017] The method and apparatus for creating a target vehicle based on multi-sensor fusion provided in this invention can fit a new radar target vehicle when there is a target vehicle at the same distance and speed in front of the vehicle but the millimeter-wave radar target vehicle information is missing. This solves the problem of radar and camera target vehicles being unable to be fused due to radar omissions and false detections of target vehicles at the same distance and speed from a distance. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a method for creating a detection target vehicle based on multi-sensor fusion, provided in an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the structure of the target vehicle creation device based on multi-sensor fusion provided in an embodiment of the present invention;

[0021] Figure 3 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention;

[0022] Figure 4 A schematic diagram of the module structure corresponding to the multi-sensor fusion-based target vehicle creation method provided in the embodiments of the present invention;

[0023] Figure 5 A schematic diagram illustrating the effect of camera and radar target vehicle detection matching in an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined with each other to form feasible technical solutions. Such combinations are not constrained by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0025] For details, please refer to [link / reference]. Figure 5The left side shows the actual working conditions, and the right side shows the detection results from the camera and radar. Circles represent camera detection results, and triangles represent radar detection results. It can be seen that radar detection and camera detection cannot match in terms of quantity. Through target tracking and combining target information data from multiple sensors, when a target vehicle at the same distance and speed is present ahead, but millimeter-wave radar target information is missing, a new radar target is fitted. This solves the problem of radar and camera target fusion being unable to be fused due to missed or false detections of target vehicles at the same distance and speed from a distance. Based on this idea, this invention provides a method for creating a detected target vehicle based on multi-sensor fusion. See [link to relevant documentation]. Figure 1 The method includes: Step 1, the perception module outputs target vehicle perception information from millimeter-wave radar and camera respectively; Step 2, the tracking module associates the target vehicle perception information input from the perception module with the target vehicle perception information from the previous frame, performs target vehicle tracking prediction, monitors the movement trend of the target vehicle, and determines whether there is a target vehicle at the same distance and speed ahead; Step 3, if similar target vehicles exist, the tracking module further determines whether the longitudinal distance difference and lateral distance difference of similar target vehicles meet the requirements; Step 4, if the requirements of Step 3 are met, it determines whether the target vehicle motion vector difference and target vehicle motion speed difference meet the requirements; Step 5, if Step 4 is met, it is determined that there is a target vehicle at the same distance and speed ahead, and the target vehicle creation module is activated; Step 6, the target vehicle creation module determines whether the millimeter-wave radar target vehicle is lost or discontinued. If it is lost or discontinued, the target vehicle creation module fits a new millimeter-wave radar target vehicle; Step 7, the new millimeter-wave radar target vehicle is fused with the target vehicle acquired by the camera using the Dempster-Shafer algorithm. This scheme includes a perception module, a target tracking module, and a target vehicle creation module (e.g., Figure 4 A method for target tracking and re-creation was designed. The perception module consists of a forward-looking camera and a forward-facing millimeter-wave radar. It should be noted that "target vehicles at the same distance and speed" refers to two (or more) vehicles in front of this vehicle traveling at the same speed and at the same distance from this vehicle.

[0026] Based on the above method embodiments, as an optional embodiment, the target vehicle creation method based on multi-sensor fusion provided in this embodiment of the invention includes the target vehicle perception information of the millimeter-wave radar and camera, including: target vehicle ID, lateral / longitudinal distance, lateral / longitudinal velocity, and target vehicle acceleration.

[0027] Based on the above method embodiments, as an optional embodiment, the multi-sensor fusion-based target vehicle creation method provided in this embodiment of the invention, in determining whether there is a target vehicle at the same distance and speed ahead, includes: whether there is a similar target vehicle whose straight-line distance difference satisfies the condition a ≤ b for the target vehicle being stably tracked by the camera; where a is the distance difference between the two target vehicles, and b is the determination threshold. It should be noted that the distance difference a between the two target vehicles and the determination threshold b are adjusted and determined according to the actual situation.

[0028] Based on the above method embodiments, as an optional embodiment, the multi-sensor fusion-based target vehicle creation method provided in this embodiment of the invention further determines whether the longitudinal distance difference and lateral distance difference of similar target vehicles meet the requirements, including: whether x≤m and y≤n are satisfied, where the values ​​of m and n are adjusted according to the longitudinal distance attribute of the target vehicle; where x is the longitudinal distance difference between the two targets, y is the lateral distance difference between the two targets, m is the lateral distance difference threshold for determining that the two targets are in a scenario with the same distance and speed; and n is the longitudinal distance difference threshold for determining that the two targets are in a scenario with the same distance and speed.

[0029] Based on the above method embodiments, as an optional embodiment, the method for creating a target vehicle based on multi-sensor fusion provided in this embodiment of the invention determines whether the target vehicle motion vector difference and the target vehicle motion speed difference meet the requirements, including whether g≤j and V≤k, where j and k are calibrated and adjusted according to the actual situation; where g is the motion vector difference between the two target vehicles; v is the longitudinal speed difference between the two target vehicles; j is the vector difference threshold for determining that the two targets are at the same distance and speed; and k is the speed difference threshold for determining that the two target vehicles are at the same distance and speed.

[0030] Based on the above method embodiments, as an optional embodiment, the multi-sensor fusion-based target vehicle creation method provided in this embodiment of the invention determines whether a millimeter-wave radar target vehicle is lost or undetected. This includes: if at least one of the two similar target vehicles output by the camera lacks a corresponding millimeter-wave radar matching target vehicle, and the number of frames for the missing millimeter-wave radar matching target vehicle exceeds a predetermined frame threshold, then the millimeter-wave radar target vehicle is determined to be lost or undetected. It should be noted that while a camera can output two target vehicles, a radar may not necessarily output two target vehicles. If the radar fails to output a target vehicle, then the target vehicle obtained by one of the cameras will lack a matching radar target vehicle.

[0031] Based on the above method embodiments, as an optional embodiment, the multi-sensor fusion-based target vehicle creation method provided in this embodiment of the invention includes the new millimeter-wave radar target vehicle attribute information, which includes: the lateral attribute information of the target without associated radar selected from the camera target, and the longitudinal attribute information of the millimeter-wave radar target; wherein, the speed information, longitudinal distance information, and motion direction information of the new millimeter-wave radar target vehicle come from the millimeter-wave radar, and the lateral distance information and lateral speed information come from the camera.

[0032] The multi-sensor fusion-based target vehicle creation method provided in this invention can fit a new radar target vehicle when there is a target vehicle at the same distance and speed in front of the vehicle but the millimeter-wave radar target vehicle information is missing. This solves the problem of radar and camera target vehicles being unable to be fused due to radar omissions and false detections of target vehicles at the same distance and speed from a distance.

[0033] The various embodiments of this invention are implemented through programmed processing using a device with processor functionality. Therefore, in practical engineering, the technical solutions and functions of the various embodiments of this invention can be encapsulated into various modules. Based on this reality, and building upon the above embodiments, this invention provides a multi-sensor fusion-based target vehicle creation device, which executes the multi-sensor fusion-based target vehicle creation method in the above method embodiments. See also... Figure 2 The device includes: a first main module for implementing step 1, where the perception module outputs target vehicle perception information from millimeter-wave radar and camera respectively; a second main module for implementing step 2, where the tracking module associates the target vehicle perception information input from the perception module with the target vehicle perception information from the previous frame, performs target vehicle tracking prediction, monitors the movement trend of the target vehicle, and determines whether there is a target vehicle at the same distance and speed ahead; a third main module for implementing step 3, where if similar target vehicles exist, the tracking module further determines whether the longitudinal distance difference and lateral distance difference of the similar target vehicles meet the requirements; and a fourth main module for implementing step 4, where if... If the requirements of step 3 are met, it is determined whether the difference in the target vehicle's motion vector and the difference in the target vehicle's motion speed meet the requirements; the fifth main module is used to implement step 5. If step 4 is met, it is determined that there is a target vehicle at the same distance and speed ahead, and the target vehicle creation module is activated; the sixth main module is used to implement step 6. The target vehicle creation module determines whether the millimeter-wave radar target vehicle is lost or discontinued. If it is lost or discontinued, the target vehicle creation module fits a new millimeter-wave radar target vehicle; the seventh main module is used to implement step 7. The new millimeter-wave radar target vehicle is fused with the target vehicle acquired by the camera using the Dempster-Shafer algorithm.

[0034] The multi-sensor fusion-based target vehicle creation device provided in this embodiment of the invention employs... Figure 2 Several modules in the system can fit a new radar target vehicle when there is a target vehicle at the same distance and speed in front of the vehicle but the millimeter-wave radar target vehicle information is missing. This solves the problem of radar and camera target vehicles being unable to be fused due to radar omissions and false detections of target vehicles at the same distance and speed from a distance.

[0035] It should be noted that the apparatus in the device embodiments provided by the present invention can be used not only to implement the methods in the above method embodiments, but also to implement the methods in other method embodiments provided by the present invention. The difference lies only in the setting of corresponding functional modules. Its principle is basically the same as that of the above device embodiments provided by the present invention. As long as those skilled in the art, based on the above device embodiments and referring to the specific technical solutions in other method embodiments, obtain corresponding technical means and technical solutions composed of these technical means by combining technical features, and improve the apparatus in the above device embodiments while ensuring the practicality of the technical solutions, they can obtain corresponding device-type embodiments for implementing the methods in other method-type embodiments. For example:

[0036] Based on the above device embodiments, as an optional embodiment, the multi-sensor fusion-based target vehicle creation device provided in this embodiment of the invention further includes: a first sub-module, used to realize the target vehicle perception information of the millimeter-wave radar and camera, including: target vehicle ID, lateral / longitudinal distance, lateral / longitudinal velocity and target vehicle acceleration.

[0037] Based on the above-described device embodiments, as an optional embodiment, the multi-sensor fusion-based target vehicle creation device provided in this embodiment of the invention further includes: a second sub-module, used to implement the condition of determining whether there is a target vehicle at the same distance and speed ahead, including: whether there is a similar target vehicle whose straight-line distance difference satisfies the condition a≤b for the target vehicle being stably tracked by the camera; where a is the distance difference between the two target vehicles and b is the determination threshold.

[0038] Based on the above-described device embodiments, as an optional embodiment, the multi-sensor fusion-based target vehicle creation device provided in this embodiment of the invention further includes: a third sub-module, used to enable the tracking module to further determine whether the longitudinal distance difference and lateral distance difference of similar target vehicles meet the requirements, including: whether x≤m and y≤n are satisfied, wherein the values ​​of m and n are adjusted according to the longitudinal distance attribute of the target vehicle; wherein x is the longitudinal distance difference between the two targets, y is the lateral distance difference between the two targets, m is the lateral distance difference threshold for determining that the two targets are in a scenario with the same distance and speed; and n is the longitudinal distance difference threshold for determining that the two targets are in a scenario with the same distance and speed.

[0039] Based on the above-described device embodiments, as an optional embodiment, the multi-sensor fusion-based target vehicle creation device provided in this embodiment of the invention further includes: a fourth sub-module, used to determine whether the target vehicle motion vector difference and the target vehicle motion speed difference meet the requirements, including whether g≤j and V≤k are satisfied, where j and k are calibrated and adjusted according to the actual situation; wherein g is the motion vector difference between the two target vehicles; v is the longitudinal speed difference between the two target vehicles; j is the vector difference threshold for determining that the two targets are at the same distance and speed; and k is the speed difference threshold for determining that the two target vehicles are at the same distance and speed.

[0040] Based on the above device embodiments, as an optional embodiment, the multi-sensor fusion-based target vehicle creation device provided in this embodiment of the invention further includes: a fifth sub-module, used to enable the target vehicle creation module to determine whether the millimeter-wave radar target vehicle is lost or discontinued, including: if at least one of the two similar target vehicles output by the camera has no corresponding millimeter-wave radar matching target vehicle, and the number of frames of the missing millimeter-wave radar matching target vehicle exceeds a predetermined frame threshold, then the millimeter-wave radar target vehicle is determined to be lost or discontinued.

[0041] Based on the above-described device embodiments, as an optional embodiment, the multi-sensor fusion-based target vehicle creation device provided in this embodiment of the invention further includes: a sixth sub-module, used to implement the new millimeter-wave radar target vehicle attribute information, including: selecting the lateral attribute information of the target without associated radar from the camera target, and the longitudinal attribute information of the millimeter-wave radar target; wherein, the speed information, longitudinal distance information, and motion direction information of the new millimeter-wave radar target vehicle come from the millimeter-wave radar, and the lateral distance information and lateral speed information come from the camera.

[0042] The method in this embodiment of the invention is implemented using an electronic device; therefore, it is necessary to introduce the relevant electronic device. For this purpose, this embodiment of the invention provides an electronic device, such as... Figure 3 As shown, the electronic device includes at least one processor, a communications interface, at least one memory, and a communications bus, wherein the at least one processor, the communications interface, and the at least one memory communicate with each other via the communications bus. The at least one processor can invoke logical instructions stored in the at least one memory to execute all or part of the steps of the methods provided in the foregoing method embodiments.

[0043] Furthermore, when the logical instructions in at least one of the aforementioned memories can be implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various method embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0044] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0045] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0046] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Based on this understanding, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0047] It should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for creating a broken detection target vehicle based on multi-sensor fusion, characterized by, include: The perception module outputs target vehicle perception information from millimeter-wave radar and camera, respectively. The tracking module associates the target vehicle perception information input by the perception module with the target vehicle perception information of the previous frame, performs target vehicle tracking prediction, monitors the movement trend of the target vehicle, and determines whether there is a target vehicle at the same distance and speed ahead. If similar target vehicles exist, the tracking module further determines whether the longitudinal distance difference and lateral distance difference of the similar target vehicles meet the requirements; If the requirements are met, then determine whether the difference in the target vehicle's motion vector and the difference in the target vehicle's motion speed meet the requirements. If the requirements are met, it is determined that there is a target vehicle at the same distance and speed ahead, and the target vehicle creation module is activated. The target vehicle creation module determines whether the millimeter-wave radar target vehicle is lost or disconnected. If it is lost or disconnected, the target vehicle creation module fits a new millimeter-wave radar target vehicle. The new millimeter-wave radar target vehicle is fused with the target vehicle acquired by the camera using the Dempster-shafer algorithm; The target vehicle creation module determines whether the millimeter-wave radar target vehicle is lost or disconnected, including: If at least one of the two similar target vehicles output by the camera does not have a corresponding millimeter-wave radar matching target vehicle, and the number of frames of the missing millimeter-wave radar matching target vehicle exceeds a predetermined frame threshold, then the millimeter-wave radar target vehicle is determined to be lost and discontinued.

2. The multi-sensor fusion-based missing target vehicle creation method according to claim 1, characterized by, The target vehicle perception information of the millimeter-wave radar and camera includes: target vehicle ID, lateral / longitudinal distance, lateral / longitudinal velocity, and target vehicle acceleration.

3. The method for creating a detection target vehicle based on multi-sensor fusion according to claim 2, characterized in that, The condition for determining whether there is a target vehicle at the same distance and speed ahead includes: Does the target vehicle being stably tracked by the camera have a similar target vehicle whose straight-line distance difference satisfies the condition a≤b? Where a is the distance difference between the two target vehicles, and b is the judgment threshold.

4. The method for creating a detection target vehicle based on multi-sensor fusion according to claim 3, characterized in that, The tracking module further determines whether the longitudinal distance difference and lateral distance difference of similar target vehicles meet the requirements, including: Whether x≤m and y≤n are satisfied, the values ​​of m and n are adjusted according to the longitudinal distance attribute of the target vehicle; where x is the longitudinal distance difference between the two targets, y is the lateral distance difference between the two targets, m is the lateral distance difference threshold for determining that the two targets are in the same distance and speed scenario, and n is the longitudinal distance difference threshold for determining that the two targets are in the same distance and speed scenario.

5. The method for creating a detection target vehicle based on multi-sensor fusion according to claim 4, characterized in that, The determination of whether the difference in the target vehicle's motion vector and the difference in the target vehicle's motion speed meet the requirements includes: Whether g≤j and V≤k are satisfied, j and k are calibrated and adjusted according to the actual situation; where g is the difference in motion vector between the two target vehicles; v is the difference in longitudinal velocity between the two target vehicles; j is the threshold for determining that the two targets are at the same distance and speed; k is the threshold for determining that the two target vehicles are at the same distance and speed.

6. The method for creating a detection target vehicle based on multi-sensor fusion according to claim 1, characterized in that, The new millimeter-wave radar target vehicle attribute information includes: The system selects lateral attribute information of targets without associated radar from camera targets, as well as longitudinal attribute information of targets from millimeter-wave radar; among them, the speed, longitudinal distance and direction of motion information of the new millimeter-wave radar target vehicle comes from the millimeter-wave radar, while the lateral distance and lateral speed information come from the camera.

7. A device for creating a detection target vehicle based on multi-sensor fusion, characterized in that, include: The first main module is used to realize step 1, where the perception module outputs the target vehicle perception information from the millimeter-wave radar and the camera, respectively. The second main module is used to implement step 2: the tracking module associates the target vehicle perception information input by the perception module with the target vehicle perception information of the previous frame, performs target vehicle tracking prediction, monitors the movement trend of the target vehicle, and determines whether there is a target vehicle at the same distance and speed ahead. The third main module is used to implement step 3: if there are similar target vehicles, the tracking module further determines whether the longitudinal distance difference and lateral distance difference of the similar target vehicles meet the requirements. The fourth main module is used to implement step 4: if the requirements of step 3 are met, then determine whether the difference in motion vector of the target vehicle and the difference in motion speed of the target vehicle meet the requirements. The fifth main module is used to implement step 5. If step 4 is met, it is determined that there is a target vehicle at the same distance and speed ahead, and the target vehicle creation module is activated. The sixth main module is used to implement step 6, where the target vehicle creation module determines whether the millimeter-wave radar target vehicle is lost or disconnected. If it is lost or disconnected, the target vehicle creation module will fit a new millimeter-wave radar target vehicle. The seventh main module is used to implement step 7, which involves fusing the new millimeter-wave radar target vehicle with the target vehicle acquired by the camera using the Dempster-shafer algorithm. The target vehicle creation module determines whether the millimeter-wave radar target vehicle is lost or disconnected, including: If at least one of the two similar target vehicles output by the camera does not have a corresponding millimeter-wave radar matching target vehicle, and the number of frames of the missing millimeter-wave radar matching target vehicle exceeds a predetermined frame threshold, then the millimeter-wave radar target vehicle is determined to be lost and discontinued.

8. An electronic device, characterized in that, include: At least one processor, at least one memory, and a communication interface; wherein, The processor, memory, and communication interface communicate with each other; The memory stores program instructions that can be executed by the processor, which invokes the program instructions to perform the method described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that cause the computer to perform the method described in any one of claims 1 to 6.

Images