Data fusion method based on multi-frame information and device thereof

Abstract

The application discloses a kind of data fusion methods based on multi-frame information, comprising: matching detection data and vehicle network connection data, obtain matching result;Processing the matching result, obtain the fusion result of the detection data and the vehicle network connection data.This application also discloses a kind of data fusion devices based on multi-frame information.The data fusion based on multi-frame information of the application can be summarized as the local data resources provided by multiple sensors of the same kind or different kinds distributed in different positions are integrated, analyzed by computer technology, the redundancy and contradiction possibly existing between multi-sensor information are eliminated, complementary, reduce its uncertainty, obtain the consistent interpretation and description of measured object, so as to improve the rapidity and correctness of system decision, planning, reaction, so that system obtains more sufficient information.

Description

Technical Field

[0001] This application relates to the field of information fusion technology, specifically to a data fusion method and apparatus based on multi-frame information. Background Technology

[0002] Currently, with the rapid increase in car ownership in China, the situation regarding road traffic safety is becoming increasingly severe. Simultaneously, with the rapid development of mobile internet and electronic information technologies, data fusion technology is used to fuse different types of data to obtain more complete and comprehensive information, thereby improving transportation efficiency, reducing traffic accidents, and enhancing the driving experience. Typical data fusion mainly includes two parts: matching and filtering. Matching primarily addresses the pairing relationships between obstacles perceived by different types of sensors at the same time. Filtering uses obstacle information perceived by different types of sensors as observations to obtain accurate and robust obstacle information (position, speed, heading, etc.). Most existing data fusion methods are based on matching single-frame sensor data, such as the Hungarian algorithm based on optimal binary matching, and Bayesian algorithms like PDA (Probabilistic Data Association) and MHT (Multihypothesis Tracking). However, matching based on single-frame sensor data may result in matching errors, such as radar ghosting or camera miss detection. Summary of the Invention

[0003] This application provides a data fusion method and apparatus based on multi-frame information to solve the problem of matching errors that occur when matching sensor data based on a single frame.

[0004] On the one hand, embodiments of this application provide a data fusion method based on multi-frame information, including:

[0005] The detection data and vehicle network data are matched to obtain the matching result;

[0006] The matching results are processed to obtain the fusion result of the detection data and the vehicle network data.

[0007] Furthermore, the matching detection data and vehicle network data yield matching results, including:

[0008] The detection data and the vehicle network data are matched using a similar trajectory method. This similar trajectory method combines the points on the first curve that are most similar to the second curve to form the matching result.

[0009] The first curve is formed by selecting one point from each set in the detection data set, and the second curve is a set curve of vehicle network data sequences.

[0010] Furthermore, the method for achieving similar trajectories includes:

[0011] Based on the vehicle network data, the detection data is filtered within a threshold range, where the threshold is Mahalanobis distance or Euclidean distance;

[0012] Calculate the similarity cost between the line segments of the first curve and the line segments of the second curve;

[0013] Expand according to the principle of minimum similarity cost;

[0014] Perform convergence judgment to obtain the most similar trajectory information;

[0015] The matching relationship between detection information and vehicle network information is combined to form a matching result.

[0016] Furthermore, the expansion according to the principle of minimum similarity cost includes:

[0017] The expansion is performed according to the principle of minimum similarity cost, and the expansion methods are Dijkstra's algorithm, A* algorithm, and greedy algorithm.

[0018] Furthermore, convergence is determined based on the threshold of the number of exploration branches or the threshold of line segment similarity cost to obtain the most similar trajectory information;

[0019] The matching results are formed by combining the sensor detection information and vehicle network information on the discovered minimum empirical path.

[0020] Furthermore, the set of vehicle network data sequences is {{O 11}, {O 21}, {O 31 The data set detected by the millimeter-wave radar sensor is {{R}. 11 R 12 R 13}, {R 21 R 22 R 23}, {R 31 R 32 R 33 R 34}}, where O 11 For vehicle network data, R 11 R 12 R 13 For millimeter-wave radar sensor detection data, using Mahalanobis distance as the threshold, O 11 R 12If the Mahalanobis distance between them is less than a set threshold, only R is considered when calculating similar trajectories. 12 Without considering R 12 R 13 ;

[0021] Calculating the similarity cost between line segments of the first curve and line segments of the second curve includes:

[0022] Calculate LIP seg , The area of ​​the convex polygon formed by the four line segments: two line segments with similar costs, a line segment whose first ends are connected, and a line segment whose last ends are connected. i Multiplied by weight w i When the polygon formed by the four line segments is a concave polygon, the concave polygon is split into multiple convex polygons to calculate the LIP. seg Then, they are accumulated; among them, Wherein, Length(O 11 O) represents line segment O 11 The length of O, Length(O, R) 12 ) represents line segment R 12 The length of O, Length(O) 11 O 21 ) represents line segment O 11 O 21 Length, Length(R) 12 R 21 ) represents line segment R 21 R 12 Length; Length(O, O 21 ) represents line segment O 21 The length of O, Length(O, R) 21 ) represents line segment R 21 The length of O.

[0023] Calculate TLIP seg , Where T(R) 12 R 21 ) represents R 12 With R 21 The time interval between them, Max(T(R) 12 R 21 ), T(O 11 O 21 )) represents R 12 With R 21 The time interval and O 11 With O 21 The maximum value within the time interval;

[0024] Calculate Splip seg , Wherein, the numerator is the length difference between the two line segments used to calculate the similarity cost, and the denominator is the length of the baseline line segment;

[0025] Calculate SPSTLIP seg SPSTLIP seg =LIP seg ·(1+k·TLIP seg )·(1+l·SPLIP seg ), where k and l are adjustable parameters for calculating the similarity cost of line segments.

[0026] Furthermore, the matching results are subjected to numerical optimization processing, including: using the matching results as inputs for Kalman filtering, extended Kalman filtering, unscented Kalman filtering, and ion filtering to obtain the fusion result of the detection data and the vehicle network data.

[0027] On the other hand, embodiments of this application provide a data fusion apparatus based on multi-frame information, including:

[0028] The acquisition module is used to acquire detection data and vehicle network data;

[0029] The matching module is used to match the detection data and the vehicle network data to obtain a matching result;

[0030] The fusion module is used to perform numerical optimization processing on the matching results to obtain the fusion result of the detection data and the vehicle network data.

[0031] Furthermore, embodiments of this application also provide a computer device, including a memory and a processor.

[0032] The memory is used to store computer programs;

[0033] The processor is used to run the computer program so that the computer device performs the data fusion method based on multi-frame information.

[0034] Finally, this application embodiment also provides a vehicle that includes the aforementioned data fusion device based on multi-frame information.

[0035] The technical solution of this application has at least the following advantages: Data fusion based on multi-frame information can be summarized as integrating the local data resources provided by multiple sensors of the same or different types distributed in different locations, analyzing them with computer technology, eliminating the redundancy and contradictions that may exist between the information of multiple sensors, complementing each other, reducing their uncertainty, obtaining a consistent interpretation and description of the measured object, thereby improving the speed and accuracy of system decision-making, planning and response, and enabling the system to obtain more complete information. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1 This is a schematic diagram illustrating the relationship between V2X vehicle network data and millimeter-wave radar sensing data in frames 1, 2, and 3, as provided in Embodiment 1 of the present invention.

[0038] Figure 2 This is a schematic flowchart of the method steps for similar trajectories provided in Embodiment 1 of the present invention;

[0039] Figure 3 This is a schematic diagram of Area1 and Area2 provided in Embodiment 1 of the present invention;

[0040] Figure 4 This is a schematic diagram of the data fusion device module based on multi-frame information provided in Embodiment 2 of the present invention. Detailed Implementation

[0041] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] In the description of this application, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0043] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal connection of two components; and they can refer to a wireless connection or a wired connection. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0044] Furthermore, the technical features involved in the different embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0045] Example 1

[0046] Embodiment 1 of the present invention provides a data fusion method based on multi-frame information, comprising:

[0047] The detection data and vehicle network data are matched to obtain the matching result;

[0048] The matching results are processed to obtain the fusion result of the detection data and the vehicle network data.

[0049] First, detection data is acquired using the vehicle's own equipment. The vehicle's sensing devices detect the road environment within their sensing range to obtain detection data. In Embodiment 1 of this invention, the vehicle sensing devices include at least one or a combination of millimeter-wave radar, microwave radar, lidar, and a camera. The detection data includes the position, speed, size, and heading of the target object.

[0050] Secondly, vehicle network data includes V2I (Vehicle to Internet) data, V2V (Vehicle to Vehicle) data, and V2X (Vehicle to Everything) data, etc., which are not limited in this embodiment of the application.

[0051] During actual operation, other vehicle equipment obtains relevant information about the vehicle itself, including the vehicle's ID, location information, speed, size, and heading.

[0052] Matching detection data and vehicle network data to obtain a matching result includes: matching the detection data and the vehicle network data using a similar trajectory method, wherein the similar trajectory method is to combine the points in the first curve that are most similar to the second curve to form a matching result;

[0053] The first curve is formed by selecting one point from each set in the detection data set, and the second curve is a set curve of vehicle network data sequences.

[0054] In Embodiment 1 of the present invention, the vehicle-related information sent by other vehicle devices includes the ID of the vehicle device. The set of trajectory information data in the physical world composed of the IDs sent by other vehicle devices at multiple times is as follows:

[0055] A collection of V2X vehicle-to-everything (V2X) connected data sequences: {{O frame11 O frame21 ,…},{O frame12 O frame22 ,…},…},where {O frame11 O frame21 , ...} represents a sequence of different frames containing the same vehicle equipment, O frameii An element (Pos, Vel, Head) in the sequence represents the vehicle's position, speed, and heading. Meanwhile, the vehicle's detection data can also be organized into multiple datasets based on sensor type and time frame.

[0056] Millimeter-wave radar sensing data set: {{R frame11 R frame12 ,…},{R frame21 R frame22 , ...}, ...}

[0057] Camera perception data set: {{C frame11 C frame12 ,…},{C frame21 C frame22 , ...}, ...}

[0058] LiDAR sensing data set: {{L frame11 L frame12 ,…},{L frame21 L frame22 , ...}, ...}

[0059] Where {R frame11 R frame12 , ...} represents the set of data sensed by millimeter-wave radar at the same time, R frameii An element (Pos, Vel, Head) in the set represents the vehicle's position, speed, and heading.

[0060] In Embodiment 1 of the present invention, detection data is obtained using millimeter-wave radar. Those skilled in the art should understand that detection data obtained using cameras and lidar have the same mathematical meaning as detection data obtained using millimeter-wave radar. That is, in the following applications involving millimeter-wave radar sensing data sets, data from camera sensing data sets or lidar sensing data sets can be substituted.

[0061] Figure 1 This is a schematic diagram illustrating the relationship between V2X vehicle network data and millimeter-wave radar perception data in frames 1, 2, and 3, as provided in Embodiment 1 of the present invention. (Refer to...) Figure 1 , {{O 11}, {O 21}, {O 31}} represents the set of V2X sensing data sequences, {{R 11 R 12 R 13}, {R 21 R 22 R 23}, {R 31 R 32 R 33 R 34} represents the set of millimeter-wave radar sensing data. The problem of trajectory similarity is transformed into... 11 R 12 R 13}, {R 21 R 22 R 23}, {R 31 R 32 R 33 R 34 In the equation}, the curve formed by selecting one point from each set intersects with curve O. 11 O 21 O 31 The combination of the most similar points.

[0062] Figure 2 This is a schematic flowchart illustrating the steps of a method for creating similar trajectories according to Embodiment 1 of the present invention. (Refer to...) Figure 2 Methods for identifying similar trajectories include:

[0063] S21. Based on the vehicle network data, filter the detection data within a threshold range, where the threshold is Mahalanobis distance or Euclidean distance;

[0064] S22. Calculate the similarity cost between the line segments of the first curve and the line segments of the second curve;

[0065] S23. Expand according to the principle of minimum similarity cost;

[0066] S24. Perform convergence judgment to obtain the most similar trajectory information;

[0067] S25. Combine the matching relationship between the detection information and the vehicle network information to form a matching result.

[0068] In S21, O in frame1 11 For the data sensed by the V2X sensor, R 11 R 12 R 13 For data sensed by radar sensors, using Mahalanobis distance as the threshold, O 11 R 12 If the Mahalanobis distance between them is less than a set threshold, only R is considered in frame1 when calculating similar trajectories. 12 Without considering R 12 R 13 .

[0069] To improve the computational efficiency of similar curves, in Embodiment 1 of this invention, the problem of calculating similar trajectories is decomposed into the problem of calculating multiple similar line segments. The curve similarity cost is equal to the sum of the similarity costs of the line segments that make up the curve. (Refer to...) Figure 1 Curve O 11 O 21 O 31 With curve R 11 R 21 R 31 The similarity cost is equal to that of line segment O. 11 O 21 With R 11 R 21 Similar cost plus line segment O 21 O 31 With R 21 R 31 Similarity cost. Calculating the similarity cost between the line segments of the first curve and the line segments of the second curve in S22 includes:

[0070] Calculate LIP seg , Figure 3 This is a schematic diagram of Area 1 and Area 2 provided in Embodiment 1 of the present invention. (Refer to...) Figure 3 Calculate the area of ​​the convex polygon formed by the four line segments: two line segments with similar costs, the line segment whose first ends are connected, and the line segment whose last ends are connected. i Multiplied by weight w i In this embodiment of the invention, w i Generally referring to w1 and w2, when the polygon formed by the four line segments is a concave polygon, the concave polygon is decomposed into multiple convex polygons to calculate LIP. segThen, they are accumulated; among them, Where Length represents the Euclidean distance between two points, i.e., the length of the line segment. In the w1 calculation formula, Length(O 11 O) represents line segment O 11 The length of O, Length(O, R) 12 ) represents line segment R 12 The length of O, the numerator represents the line segment O 11 O and line segment R 12 The length of O is summed, and similarly, the denominator represents the length of line segment O. 11 O 21 With line segment R 21 R 12 The lengths of O and O; in the formula for w2, Length(O, O 21 ) represents line segment O 21 The length of O, Length(O, R) 21 ) represents line segment R 21 The length of O, the numerator represents the line segment O 21 O and line segment R 21 The length of O is summed, and similarly, the denominator represents O. 11 O 21 With line segment R 11 R 12 The length and sum of;

[0071] Calculate TLIP seg , Where T(R) 12 R 21 ) represents R 12 With R 21 The time interval between them, Max(T(R) 12 R 21 ), T(O 11 O 21 )) represents R 12 With R 21 The time interval and O 11 With O 21 The maximum value within the time interval;

[0072] Calculate Splip seg , Wherein, the numerator is the length difference between the two line segments used to calculate the similarity cost, and the denominator is the length of the baseline line segment;

[0073] Calculate SPSTLIP seg SPSTLIP seg =LIP seg ·(1+k·TLIP seg )·(1+l·SPLIPseg ), where k and l are adjustable parameters for calculating the similarity cost of line segments.

[0074] In S23, the expansion is performed according to the principle of minimum similarity cost, and the expansion methods are Dijkstra's algorithm, A* algorithm, and greedy algorithm.

[0075] Based on S24 and S25, convergence judgment is performed according to the threshold of the number of exploration branches or the threshold of line segment similarity cost to obtain the most similar trajectory information; the matching relationship between the sensor detection information and vehicle network information on the explored minimum empirical path is combined to form the matching result.

[0076] Finally, the matching results are numerically optimized, including using the matching results as inputs for Kalman filtering, extended Kalman filtering, unscented Kalman filtering, and ion filtering to obtain the fusion result of the detection data and the vehicle network data.

[0077] Example 2

[0078] Figure 4 This is a schematic diagram of the data fusion device module based on multi-frame information provided in Embodiment 2 of the present invention. (Refer to...) Figure 4 The data fusion apparatus based on multi-frame information provided in Embodiment 2 of this application includes:

[0079] Module 41 is used to acquire detection data and vehicle network data;

[0080] Matching module 42 is used to match the detection data and the vehicle network data to obtain a matching result;

[0081] The fusion module 43 is used to perform numerical optimization processing on the matching results to obtain the fusion result of the detection data and the vehicle network data.

[0082] The acquisition module 41 includes a camera unit 411 and a radar unit 412, which are used to acquire detection data. In the second embodiment of the present invention, millimeter-wave radar is used to acquire detection data. The acquisition module 41 also includes a V2X vehicle terminal unit 413. In the second embodiment of the present invention, the V2X vehicle terminal unit 413 senses vehicle network data through V2V, that is, the relevant information of the vehicle is shared by other vehicles.

[0083] The matching module 42 and the fusion module 43 match the detection data acquired by the millimeter-wave radar and the vehicle network data sensed by the V2X vehicle terminal unit 413, and perform numerical optimization processing on the matching results to obtain the fusion results.

[0084] In the second embodiment of the present invention, the assisted driving module receives data processed by the data fusion device based on multi-frame information after being processed by the matching module 42 and the fusion module 43. After processing through five steps of perception, localization, prediction, planning and control, it outputs the perception results of the physical world, the position of the vehicle body, the future trajectory information and behavior information of obstacles, the planned path and behavior, and the control data of the vehicle body.

[0085] The aforementioned perception specifically refers to using fused data as input to output environmental information around the vehicle, such as lane lines, obstacles, traffic signs, and drivable areas.

[0086] The aforementioned positioning specifically refers to taking fused data and map data as input and outputting the vehicle's motion state, such as speed, angular velocity, and acceleration, as well as information such as the vehicle's position on the map.

[0087] The aforementioned prediction specifically refers to using fused data as input to predict the behavior and trajectory information of dynamic obstacles over a future period of time;

[0088] The aforementioned planning specifically refers to using perception and positioning results as input to plan the trajectory and speed of the vehicle body over a period of time in the future, and making decisions such as acceleration, deceleration, braking, and takeover.

[0089] The aforementioned control specifically refers to the control of the vehicle itself based on the planning results.

[0090] Embodiment 2 of the present invention also provides a vehicle, which includes the above-mentioned data fusion device based on multi-frame information.

[0091] Furthermore, the present invention also provides a computer device. This computer device includes a memory and a processor. The memory can be used to store computer programs, and the processor, by running the computer programs, enables the computer device to perform the functions of the methods described above or the various modules in the data fusion apparatus based on multi-frame information.

[0092] The memory may include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc. The data storage area may store data created based on the use of the mobile terminal (such as audio data, phonebook, etc.). Furthermore, the memory may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0093] This embodiment also provides a computer storage medium for storing computer programs used in the aforementioned computer device.

[0094] It should be understood, in the several embodiments provided in this application, that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0095] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0096] If a function is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0097] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this application.

Claims

1. A data fusion method based on multi-frame information, characterized in that, include: The detection data and vehicle network data are matched to obtain the matching result; The detection data is acquired using the vehicle's own equipment and is obtained by detecting the road environment within the perception range using the vehicle's sensing equipment; the detection data includes the position, speed, size, and heading of the target object; multiple detection data sets are formed according to the type of sensor and the time. The matching results are processed to obtain the fusion result of the detection data and the vehicle network data; The matching results obtained from the matching detection data and vehicle network data include: The detection data and the vehicle network data are matched using a similar trajectory method, wherein the similar trajectory method is as follows: The first curve is formed by selecting one point from each set in the detection data set; the second curve is formed by the set of vehicle network data sequences. The points in the first curve that are most similar to the second curve are combined to form a matching result; The method for similar trajectories also includes: Based on the vehicle network data, the detection data is filtered within a threshold range, where the threshold is Mahalanobis distance or Euclidean distance; Calculate the similarity cost between the line segments of the first curve and the line segments of the second curve; Expand according to the principle of minimum similarity cost; Perform convergence judgment to obtain the most similar trajectory information; The matching relationship between detection information and vehicle network information is combined to form a matching result.

2. The data fusion method based on multi-frame information according to claim 1, characterized in that, The expansion based on the principle of minimum similarity cost includes: The expansion is performed according to the principle of minimum similarity cost, and the expansion methods are Dijkstra's algorithm, A* algorithm, and greedy algorithm.

3. The data fusion method based on multi-frame information according to claim 1, characterized in that, Convergence is determined by the threshold of the number of exploration branches or the threshold of line segment similarity cost, and the most similar trajectory information is obtained. The matching results are formed by combining the sensor detection information and vehicle network information on the discovered minimum empirical path.

4. The data fusion method based on multi-frame information according to claim 1, characterized in that, The set of vehicle network data sequences is {{O 11 }, {O 21 },{O 31 The data set detected by the millimeter-wave radar sensor is {{R}. 11 ,R 12 ,R 13 },{R 21 ,R 22 ,R 23 },{R 31 ,R 32 ,R 33 ,R 34 }}, where O 11 For vehicle network data, R 11 ,R 12 ,R 13 For millimeter-wave radar sensor detection data, using Mahalanobis distance as the threshold, O 11 R 12 If the Mahalanobis distance between them is less than a set threshold, only R is considered when calculating similar trajectories. 12 Without considering R 12 ,R 13 ; Calculating the similarity cost between line segments of the first curve and line segments of the second curve includes: Calculate LIP seg , The area of ​​the convex polygon formed by the four line segments: two line segments with similar costs, a line segment whose first ends are connected, and a line segment whose last ends are connected. i Multiplied by weight w i When the polygon formed by the four line segments is a concave polygon, the concave polygon is split into multiple convex polygons to calculate the LIP. seg Then, they are accumulated; among them, Wherein, Length(O 11 ,O) represents line segment O 11 The length of O, Length(O,R) 12 ) represents line segment R 12 The length of O, Length(O) 11 O 21 ) represents line segment O 11 O 21 Length, Length(R) 12 ,R 21 ) represents line segment R 21 R 12 Length; Length(O,O 21 ) represents line segment O 21 The length of O, Length(O,R) 21 ) represents line segment R 21 The length of O; Calculate TLIP seg , Where T(R) 12 ,R 21 ) represents R 12 With R 21 The time interval between them, Max(T(R) 12 ,R 21 ),T(O 11 O 21 )) represents R 12 With R 21 The time interval and O 11 With O 21 The maximum value within the time interval; Calculate Splip seg , Wherein, the numerator is the length difference between the two line segments used to calculate the similarity cost, and the denominator is the length of the baseline line segment; Calculate SPSTLIP seg SPSTLIP seg =LIP seg ·(1+k·TLIP seg )·(1+l·SPLIP seg ), where k,l are adjustable parameters for calculating the similarity cost of line segments.

5. The data fusion method based on multi-frame information according to claim 1, characterized in that, The matching results are numerically optimized, including using the matching results as inputs for Kalman filtering, extended Kalman filtering, unscented Kalman filtering, and ion filtering to obtain the fusion result of the detection data and the vehicle network data.

6. A data fusion device based on multi-frame information, characterized in that, The apparatus for implementing the data fusion method based on multi-frame information as described in any one of claims 1-5, the apparatus comprising: The acquisition module is used to acquire detection data and vehicle network data; The matching module is used to match the detection data and the vehicle network data to obtain a matching result; The fusion module is used to perform numerical optimization processing on the matching results to obtain the fusion result of the detection data and the vehicle network data.

7. A computer device, characterized in that, Including memory and processor, The memory is used to store computer programs; The processor is used to run the computer program to enable the computer device to perform the data fusion method based on multi-frame information as described in any one of claims 1 to 5.

8. A vehicle, characterized in that, Includes the data fusion device based on multi-frame information as described in claim 6.

Images