Radar and vision fusion method and device based on binary bayesian filter

Abstract

The application provides a radar and vision fusion method and device based on binary Bayesian filtering, and relates to the technical field of multi-sensor target fusion; the method first pre-processes vision collection data and radar collection data to obtain vision detection results and radar point track detection results; then, invalid radar point tracks are filtered out in combination with vehicle motion information to determine radar effective point track targets; subsequently, the vision detection results are combined with the radar effective point track targets, a first matching probability of each combination in a current period is determined based on a radar and vision target attribute difference weight proportion; then, binary Bayesian filtering is performed with the first matching probability as a prior probability to obtain a second matching probability of multiple periods; finally, matching tracking is performed on the target combination determined by the second matching probability, and vision and radar detection results are fused; and thus, the problem of poor vision and radar correlation matching effect is effectively solved, and the accuracy and stability of multi-sensor target fusion are improved.

Description

Technical Field

[0001] This invention relates to the technical field of multi-sensor target fusion, and in particular to a radar and vision fusion method and apparatus based on binary Bayesian filtering. Background Technology

[0002] With the rapid development of autonomous driving technology, multi-sensor fusion has become an important research direction in the field of environmental perception. Radar and vision, as two main types of perception sensors, each have their unique advantages and limitations. Millimeter-wave radar performs excellently in ranging and velocity measurement, maintaining stable performance even in adverse weather conditions, but its accuracy in detecting the lateral position of targets is relatively low. Visual sensors, on the other hand, can provide rich texture and color information, achieving high-precision lateral detection, but their longitudinal distance measurement is easily affected by environmental factors such as lighting and weather.

[0003] In existing multi-sensor fusion systems, target association is a crucial step in achieving effective fusion. The Hungarian algorithm, a classic bipartite graph matching method, is widely used for association matching between radar and visual targets. This algorithm constructs a cost matrix to achieve optimal matching by calculating the distance and velocity differences between visually detected targets and radar-detected targets. However, when the target is stationary, due to the lack of velocity information as a matching basis, association judgment can only rely on the distance difference. In this case, if the distance measurement of the visual sensor has a large error due to environmental interference, or if the radar detects a large amount of stationary clutter, it is easy for the visual target to be matched with the incorrect radar detection result, resulting in a jump in the position of the fused target, which seriously affects the subsequent path planning and vehicle control performance. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a radar and vision fusion method and apparatus based on binary Bayesian filtering to alleviate the technical problem of poor correlation matching effect between vision and radar.

[0005] In a first aspect, the present invention provides a radar and vision fusion method based on binary Bayesian filtering, comprising: The visual and radar data acquired by the vehicle are preprocessed to obtain the visual detection results for the current period and the radar detection results for each radar point. Based on the current vehicle motion information, the visual detection results and the radar detection results filter out invalid radar traces and determine valid radar trace targets. Each visual detection result and each radar detection result of effective radar trace targets are combined, and the first matching probability of each combination in the current period is determined based on the weight ratio of the attribute difference between the visual detection result and the radar detection result in each combination. The first matching probability of each current combination in the current period is used as the prior probability for binary Bayesian filtering to determine the second matching probability of each current combination in multiple periods; The target combination determined by the second matching probability is matched and tracked, and the visual detection result and the radar detection result corresponding to the target combination are fused.

[0006] In an optional implementation, the step of preprocessing the visual and radar data acquired by the current vehicle to obtain the visual detection result for the current period and the radar detection result for each radar point includes: Transform the visual data acquisition data from the camera coordinate system to the radar coordinate system where the radar data acquisition data is located; Based on the time difference between the visual acquisition time and the radar acquisition time and the current period, time compensation is performed on the visual acquisition data and the radar acquisition data to obtain the visual detection result for the current period and the radar detection result for each radar point.

[0007] In an optional implementation, the visual detection result includes at least the lateral velocity, lateral distance, longitudinal velocity, and longitudinal distance of each preset physical target relative to the current vehicle; the radar detection result includes at least the lateral velocity, lateral distance, longitudinal velocity, and longitudinal distance of each radar dot relative to the current vehicle; the preset physical targets include lane lines detected visually; based on the motion information of the current vehicle, the step of filtering out invalid radar dots and determining valid radar dot targets based on the visual detection result and the radar detection result includes: The absolute velocity of each radar dot is determined based on the motion information of the current vehicle and the longitudinal velocity of each radar dot relative to the current vehicle. Radar spots that are stationary and whose absolute velocity is less than the velocity threshold are identified as the first pre-screened spot targets. According to the longitudinal distance of the first pre-selected point targets relative to the current vehicle, the first pre-selected point targets are sorted, and second pre-selected point targets that meet the preset sequence requirements are selected. Based on the comparison between the first average value of the first lateral distance of each of the second pre-screened target points relative to the current vehicle and the second lateral distance of each lane line relative to the current vehicle, the target lane line closest to the current vehicle is determined; Calculate the third lateral distance from each of the second pre-screened target points to the target lane line and the second average value of the third lateral distance, filter out the second pre-screened target points whose third lateral distance is less than the second average value, and obtain the effective radar target points.

[0008] In an optional implementation, the step of combining each visual detection result and each radar detection result of a valid radar trace target, and determining the first matching probability of each combination in the current period based on the weight ratio of the attribute difference between the visual detection result and the radar detection result in each combination, includes: Based on the unique ID carried by each visual detection result and each radar detection result, the radar detection results of each visual detection result and each radar effective point target are combined to determine each combination consisting of the IDs corresponding to the visual detection results and the radar detection results, and the weight value corresponding to each combination. Based on the weight value of each current combination and the weight values ​​of all other combinations that have the same ID as each current combination, the first matching probability of each current combination in the current period is determined.

[0009] In an optional implementation, the step of using the first matching probability of each current combination in the current period as a prior probability for binary Bayesian filtering to determine the second matching probability of each current combination in multiple periods includes: Based on the binary Bayesian filtering formula, with preset observation conditions, the first matching probability of each current combination in the current period and the first posterior advantage ratio of each current combination in the historical period are used to update the second posterior advantage ratio of each current combination in multiple periods. The second posterior advantage ratio is converted into a second matching probability for each current combination over multiple periods using an inverse function.

[0010] In an optional implementation, the step of matching and tracking the target combination determined by the second matching probability, and fusing the visual detection result and the radar detection result corresponding to the target combination, includes: Each current combination is sorted according to the second matching probability, and matching and tracking are performed based on the target combination that meets the pre-arranged requirements to obtain matching results including classification categories; Based on the classification category, the visual detection results and radar detection results corresponding to the target combination are fused.

[0011] In an optional implementation, the step of fusing the visual detection result and the radar detection result corresponding to the target combination based on the classification category includes: The visual detection results and radar detection results corresponding to the target combination are fused to obtain a first fusion result; Based on the classification category, determine the target corresponding to the radar effective point target; Based on the classification category and size information of the target object, the radar detection result corresponding to the radar effective point target is fused with the first fusion result to obtain the second fusion result.

[0012] Secondly, the present invention provides a radar and vision fusion device based on binary Bayesian filtering, comprising: The preprocessing module preprocesses the visual and radar data acquired by the current vehicle to obtain the visual detection results for the current period and the radar detection results for each radar point. The filtering module, based on the current vehicle's motion information, the visual detection results, and the radar detection results, filters out invalid radar markers and determines valid radar marker targets. The first matching module combines each visual detection result and each radar detection result of a radar effective point target, and determines the first matching probability of each combination in the current period based on the weight ratio of the attribute difference between the visual detection result and the radar detection result in each combination. The second matching module uses the first matching probability of each current combination in the current period as a prior probability and performs binary Bayes filtering to determine the second matching probability of each current combination in multiple periods. The fusion module performs matching and tracking on the target combination determined by the second matching probability, and fuses the visual detection results and the radar detection results corresponding to the target combination.

[0013] Thirdly, the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the steps of the method described in any of the foregoing embodiments.

[0014] Fourthly, the present invention provides a machine-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to perform the steps of the method described in any of the foregoing embodiments.

[0015] This invention provides a radar and vision fusion method and apparatus based on binary Bayesian filtering. The method filters out inaccurate radar detection points to obtain valid radar target traces, which are then further correlated and matched with visual detection results. Based on the binary Bayesian filtering and the current matching probability of each correlation combination, the matching probability of each combination in multiple periods is determined to further ensure the accuracy of the matching results. Finally, target combinations with matching probabilities are fused to obtain a more accurate visual-radar fusion result.

[0016] Other features and advantages of this disclosure will be set forth in the following description, or some features and advantages may be inferred from the description or determined without doubt, or may be learned by practicing the techniques described above.

[0017] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 A flowchart of a radar and vision fusion method based on binary Bayesian filtering is provided for an embodiment of the present invention; Figure 2 This is a schematic diagram of radar effective point target screening provided in an embodiment of the present invention; Figure 3 A functional block diagram of a radar and vision fusion device based on binary Bayesian filtering provided for an embodiment of the present invention; Figure 4 This is a schematic diagram of the hardware architecture of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Currently, when the target is stationary, the fusion effect of visual sensors and radar has a large error, which is not conducive to the safe control of vehicle driving.

[0022] Based on this, the present invention provides a radar and vision fusion method and apparatus based on binary Bayesian filtering, which can obtain the matching probabilities of vision and radar under multiple periods according to binary Bayesian filtering, thereby achieving accurate fusion.

[0023] To facilitate understanding of this embodiment, a radar and vision fusion method based on binary Bayesian filtering disclosed in this embodiment of the invention will be described in detail first. This method is applied to intelligent control devices such as vehicle controllers and vehicle infotainment systems that are installed in the whole vehicle.

[0024] Figure 1 A flowchart of a radar and vision fusion method based on binary Bayesian filtering is provided for an embodiment of the present invention.

[0025] like Figure 1 As shown, the method includes the following steps: Step S102: Preprocess the visual acquisition data and radar acquisition data of the current vehicle to obtain the visual detection result for the current period and the radar detection result for each radar point.

[0026] For example, the visual and radar data collected by the vehicle's sensors can be preprocessed by performing coordinate system transformation and time compensation to ensure the accuracy of the visual and radar detection results in subsequent steps, including: Step 1.1: Transform the visual acquisition data from the camera coordinate system to the radar coordinate system where the radar acquisition data is located.

[0027] The visual acquisition data corresponding to the camera installation position coordinate system is converted to the radar installation position coordinate system, that is, the coordinate system of the center of the front bumper of the vehicle.

[0028] Step 1.2: Based on the time difference between the visual acquisition time and the radar acquisition time and the current period, time compensation is performed on the visual acquisition data and the radar acquisition data to obtain the visual detection result for the current period and the radar detection result for each radar point.

[0029] Since vision and radar data are collected according to their respective cycles, to achieve more accurate detection results, it is necessary to perform a first time difference compensation on the visual data based on the difference between the current fusion time and the visual data acquisition time, and a second time difference compensation on the radar data based on the difference between the current fusion time and the radar data acquisition time. This ensures the accuracy of the visual and radar detection results obtained in the current fusion cycle. Here, "current cycle" can also be understood as "current moment."

[0030] Step S104: Based on the current vehicle motion information, visual detection results and radar detection results, invalid radar dots are filtered out to determine valid radar dot targets.

[0031] Here, the visual detection results include at least the lateral velocity, lateral distance, longitudinal velocity, and longitudinal distance of each preset physical target relative to the current vehicle; the radar detection results include at least the lateral velocity, lateral distance, longitudinal velocity, and longitudinal distance of each radar dot relative to the current vehicle; the preset physical targets include lane lines.

[0032] In some embodiments, step S104 may be implemented by the following steps to filter effective radar point targets, including: Step 2.1: Determine the absolute velocity of each radar dot based on the current vehicle's motion information and the longitudinal velocity of each radar dot relative to the current vehicle.

[0033] The current vehicle motion information includes the vehicle speed and angular velocity. Here, by acquiring the vehicle motion information, the vehicle speed can be known. The longitudinal velocity of the radar dot relative to the vehicle can be known from the radar dot, and thus the absolute velocity of each radar dot can be obtained. Based on this absolute velocity, the motion state of the radar dot can be determined.

[0034] Step 2.2: Identify radar spots that are stationary and whose absolute velocity is less than the velocity threshold as the first pre-screened spot targets.

[0035] It is understood that radar tracks with an absolute speed less than a speed threshold are considered stationary, while radar tracks with a speed greater than a threshold are considered moving. In this embodiment of the invention, all radar tracks in a stationary state are used as the first pre-screened track targets.

[0036] Step 2.3: Sort the first pre-selected point targets according to their longitudinal distance relative to the current vehicle, and select the second pre-selected point targets that meet the preset sequence requirements. The pre-screened radar targets (meeting the requirements) are sorted from near to far according to their longitudinal distance relative to the vehicle. The radar targets that meet the preset sequence requirements are selected as the second pre-screened radar targets. The preset sequence requirements may include a specific number of radar targets that are ranked first.

[0037] Step 2.4: Based on the comparison between the first average value of the first lateral distance of each second pre-screened target point relative to the current vehicle and the second lateral distance of each lane line relative to the current vehicle, determine the target lane line closest to the current vehicle.

[0038] Here, each second pre-screened target point has a corresponding first lateral distance relative to the current vehicle. The first average value is obtained by averaging all the first lateral distances. In addition, each lane line also has its own different second lateral distance relative to the current vehicle. The difference between the first average value and the second lateral distance is compared, and the lane line with the smallest difference is the target lane line closest to the current vehicle.

[0039] Step 2.5: Calculate the third lateral distance from each second pre-screened target to the target lane line and the second average value of the third lateral distance. Filter out second pre-screened targets whose third lateral distance is less than the second average value to obtain the effective radar targets.

[0040] The formula for the distance from a point to a line describes the shortest distance from a point on a plane to a straight line. Let the equation of the line be (Ax + By + C = 0), and the coordinates of the point be... Then the distance (d) from the point to the line can be expressed as: Based on this formula, the corresponding third lateral distance from each second pre-screened target to the target lane line can be calculated. Then, the average of all the third lateral distances is calculated to obtain the second average value. At this time, each third lateral distance is compared with the second average value, and radar targets with third lateral distances less than the second average value are filtered out. The remaining targets are the effective radar targets.

[0041] like Figure 2 As shown, the red vehicle is the current vehicle (self-vehicle), and the black dots are radar dots. The left side of the self-vehicle is the nearest lane line LB to the current vehicle. Through the steps of the aforementioned embodiment, radar dots that are relatively close to the nearest lane line can be filtered out because the third lateral distance from the nearest lane line is less than the second average value. It should be noted that the radar dot in the lower left corner of the green vehicle that belongs to the same lane as the current vehicle is a valid radar dot target.

[0042] Step S106: Combine each visual detection result with the radar detection result of each effective radar target, and determine the first matching probability of each combination in the current period based on the weight ratio of the attribute difference between the visual detection result and the radar detection result in each combination.

[0043] Based on the foregoing embodiments, relatively accurate radar detection results and visual detection results can be correlated and combined, and the matching result of each combination can be calculated; for example, step S106 includes: Step 3.1: Based on the unique ID carried by each visual detection result and each radar detection result, combine each visual detection result and the radar detection result of each effective radar point target to determine each combination consisting of the IDs corresponding to the visual detection results and the radar detection results, as well as the weight value corresponding to each combination.

[0044] Here, each visual detection result and radar detection result carries a unique ID, such as IDs C0, C1, etc. for visual detection results, and IDs R0, R1, etc. for radar detection results. All visual and radar detection results are combined to form combinations consisting of their respective IDs, such as (C0, R0), (C0, R1), (C1, R0), (C1, R1), etc. The weight value of each combination is then calculated according to the following formula:

[0045] Where wx is the weighting coefficient for lateral distance, deltaX is the difference in lateral distance between the associated visual and radar targets, wy is the weighting coefficient for longitudinal distance, deltaY is the difference in longitudinal distance between the associated visual and radar targets, wv is the weighting coefficient for longitudinal relative velocity, and deltaV is the difference in relative velocity between the associated targets.

[0046] Step 3.2: Determine the first matching probability of each current combination in the current period based on the weight value of each current combination and the weight values ​​of all other combinations that have the same ID as each current combination.

[0047] Here, based on the visual and radar IDs in each combination, all combinations containing the same visual or radar ID are filtered out. For example, if the current combination is (C0, R0), then all combinations with visual or radar IDs are filtered out as (C0, R1) and (C1, R0). According to the aforementioned embodiment, the weight of the (C0, R0) combination is calculated as W00, the weight of the (C0, R1) combination is W01, and the weight of the (C1, R0) combination is W10, with the total weight being W = W00 + W01 + W10. Based on this, the matching probability of each combination is calculated. Taking the (C0, R0) combination as an example, the first matching probability of the (C0, R0) combination is calculated. .

[0048] Step S108: Use the first matching probability of each current combination in the current period as the prior probability and perform binary Bayesian filtering to determine the second matching probability of each current combination in multiple periods.

[0049] Based on the aforementioned embodiments, step S108 further processes the first matching probability of each combination in the current period using binary Bayesian filtering to obtain the second matching probability for multiple periods, thereby ensuring higher matching accuracy for each combination, including: Step 4.1: Based on the binary Bayesian filtering formula, preset observation conditions, the first matching probability of each current combination in the current period and the first posterior advantage ratio of each current combination in the historical period, and update the second posterior advantage ratio of each current combination in multiple periods.

[0050] Binary Bayesian filtering uses the formula to calculate the probability of each combination across multiple periods. The combined probability values ​​of each period t under observation condition z. Binary Bayesian filtering is used for dynamic systems whose state space contains only two values ​​(e.g., x=0 or x=1), recursively updating the posterior probability of the state through observation data. The core assumptions of binary Bayesian filtering include: State binaryity: the variable to be estimated x∈{0,1} (e.g., "occupied" and "blank" in a raster map). Static system: the state does not change over time, and the estimation accuracy is optimized only through multiple observations. Observation independence: each observation z is independent of other observations, depending only on the current state x=1.

[0051] Substitute the above probability values ​​into the following formula:

[0052] in, This is the posterior probability (the probability of state x under observation condition z). Let t be the prior probability and t be the period. Wherein is the posterior advantage ratio obtained in the previous period. This is the advantage ratio obtained in the current period after the update.

[0053] Step 4.2: Convert the second posterior advantage ratio into the second matching probability of each current combination over multiple periods using an inverse function.

[0054] Here, we can also use an inverse function to... The second matching probability is converted as shown in the following formula:

[0055] Step S110: Match and track the target combination determined by the second matching probability, and fuse the visual detection results and radar detection results corresponding to the target combination.

[0056] Here, based on the aforementioned second matching probability, target combinations with higher matching accuracy are selected for matching and tracking. Then, the visual and radar detection results of the target combinations are fused. For example, this step S110 includes the following steps: Step 5.1: Sort each current combination according to the second matching probability, and perform matching tracking based on the target combination that meets the pre-arranged requirements to obtain matching results including classification categories.

[0057] Each combination is sorted according to the second matching probability value. A preset number of combinations at the top can be selected as the target combination, and then the matching and tracking operation is continued. The matching result includes the lateral speed difference, longitudinal speed difference, lateral distance difference, longitudinal distance difference, relative speed difference value of the visual and radar relative to the vehicle, as well as the classification category of the target.

[0058] Step 5.2: Based on the classification category, fuse the visual detection results and radar detection results corresponding to the target combination.

[0059] For successfully matched target combinations, the visual detection results and radar detection results can be directly superimposed and fused. However, for some large targets, such as large trucks, the radar may detect radar points at additional angles without matching visual detection results. In this case, fusion can be achieved through the following steps: Step 5.2.1: Fuse the visual detection results and radar detection results corresponding to the target combination to obtain the first fusion result.

[0060] Step 5.2.2: Determine the target corresponding to the effective radar point target based on the classification category.

[0061] For example, the classification of the matching results can reveal that the target of the radar's effective point targets is a large truck.

[0062] Step 5.2.3: Based on the classification and size information of the target, fuse the radar detection results corresponding to the effective radar point targets with the first fusion result to obtain the second fusion result.

[0063] Based on the classification category, namely the length and width information of the truck, the first fusion result is obtained by combining the lateral distance difference, longitudinal distance search, speed difference, and each valid point of the correctly matched target in the matching result, and then merging them to obtain the second fusion result for a complete object target (truck).

[0064] In a preferred embodiment of the practical application, the present invention mainly includes preprocessing, target matching and tracking, and target attribute fusion and updating. Specifically, effective target screening primarily combines lane line information detected by vision to filter out guardrail targets (invalid targets) detected by radar, reducing the impact of radar clutter on the association matching module, improving the correctness of association matching and reducing computation time. Then, binary Bayesian filtering is used to calculate the association probability of each radar and vision pair in the current period, combining distance and speed differences. The association probability of each radar and vision pair combination is then calculated by combining historical matching information to obtain the optimal matching result for the current period. This method considers the probability of each sensor associating with other sensors and, by combining historical information, solves the problem of erroneous association matching caused by short-term inaccuracies in sensor attributes, improving the success rate of association matching, especially in stationary target scenarios.

[0065] In some embodiments, such as Figure 3 As shown, this embodiment of the invention also provides a radar and vision fusion device 200 based on binary Bayesian filtering, comprising: The preprocessing module 201 preprocesses the visual acquisition data and radar acquisition data of the current vehicle to obtain the visual detection results for the current period and the radar detection results for each radar point. The filtering module 202, based on the current vehicle motion information, the visual detection results and the radar detection results, filters out invalid radar traces and determines valid radar trace targets; The first matching module 203 combines each visual detection result and each radar detection result of the radar effective point target, and determines the first matching probability of each combination in the current period based on the attribute difference weight ratio of the visual detection result and the radar detection result in each combination. The second matching module 204 uses the first matching probability of each current combination in the current period as a prior probability and performs binary Bayesian filtering to determine the second matching probability of each current combination in multiple periods. The fusion module 205 performs matching and tracking on the target combination determined by the second matching probability, and fuses the visual detection result and the radar detection result corresponding to the target combination.

[0066] This invention filters targets that are effectively detected by radar and optimizes the association matching between radar and vision. Especially when the target is stationary, it reduces the interference of stationary targets on the fusion algorithm. For example, it significantly improves the effect on scenes of stationary cars parked next to guardrails and stationary cars parked near billboards above, effectively improving the accuracy of association matching, reducing false matching, and making more accurate, reliable and trustworthy state estimations for environmental perception.

[0067] Figure 4This is a schematic diagram of the hardware architecture of the electronic device 300 provided in an embodiment of the present invention. See also... Figure 4 As shown, the electronic device 300 includes a machine-readable storage medium 301 and a processor 302, and may also include a non-volatile storage medium 303, a communication interface 304, and a bus 305; wherein the machine-readable storage medium 301, the processor 302, the non-volatile storage medium 303, and the communication interface 304 communicate with each other through the bus 305. The processor 302 can execute the radar and vision fusion method based on binary Bayesian filtering described in the above embodiments by reading and executing the machine-executable instructions for radar and vision fusion based on binary Bayesian filtering in the machine-readable storage medium 301.

[0068] The machine-readable storage medium mentioned in this article can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0069] Non-volatile media can be non-volatile memory, flash memory, storage drives (such as hard disk drives), any type of storage disk (such as optical discs, DVDs, etc.), or similar non-volatile storage media, or combinations thereof.

[0070] It is understood that the specific operation methods of each functional module in this embodiment can be referred to the detailed description of the corresponding steps in the above method embodiment, and will not be repeated here.

[0071] The computer-readable storage medium provided in the embodiments of the present invention stores a computer program. When the computer program code is executed, it can implement the radar and vision fusion method based on binary Bayesian filtering as described in any of the above embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0072] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and apparatus described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0073] Furthermore, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0074] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0075] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the scope of the technology disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention.

Claims

1. A radar and vision fusion method based on binary Bayesian filtering, characterized in that, include: The visual and radar data acquired by the vehicle are preprocessed to obtain the visual detection results for the current period and the radar detection results for each radar point. Based on the current vehicle motion information, the visual detection results and the radar detection results filter out invalid radar traces and determine valid radar trace targets. Each visual detection result and each radar detection result of a valid radar point target are combined, and the first matching probability of each combination in the current period is determined based on the weight ratio of the attribute difference between the visual detection result and the radar detection result in each combination. Based on the binary Bayesian filtering formula, with preset observation conditions, the first matching probability of each current combination in the current period and the first posterior advantage ratio of each current combination in the historical period are used to update the second posterior advantage ratio of each current combination in multiple periods; the second posterior advantage ratio is converted into the second matching probability of each current combination in multiple periods through an inverse function; The target combination determined by the second matching probability is matched and tracked, and the visual detection result and the radar detection result corresponding to the target combination are fused.

2. The method according to claim 1, characterized in that, The steps for preprocessing the current vehicle's visual and radar data to obtain the visual detection results for the current period and the radar detection results for each radar point include: Transform the visual data acquisition data from the camera coordinate system to the radar coordinate system where the radar data acquisition data is located; Based on the time difference between the visual acquisition time and the radar acquisition time and the current period, time compensation is performed on the visual acquisition data and the radar acquisition data to obtain the visual detection result for the current period and the radar detection result for each radar point.

3. The method according to claim 1, characterized in that, The visual detection results include at least the lateral velocity, lateral distance, longitudinal velocity, and longitudinal distance of each preset physical target relative to the current vehicle; the radar detection results include at least the lateral velocity, lateral distance, longitudinal velocity, and longitudinal distance of each radar dot relative to the current vehicle; the preset physical targets include lane lines detected visually. Based on the current vehicle motion information, the steps of filtering out invalid radar markers and determining valid radar marker targets using the visual detection results and radar detection results include: The absolute velocity of each radar dot is determined based on the motion information of the current vehicle and the longitudinal velocity of each radar dot relative to the current vehicle. Radar spots that are stationary and whose absolute velocity is less than the velocity threshold are identified as the first pre-screened spot targets. According to the longitudinal distance of the first pre-selected point targets relative to the current vehicle, the first pre-selected point targets are sorted, and second pre-selected point targets that meet the preset sequence requirements are selected. Based on the comparison between the first average value of the first lateral distance of each of the second pre-screened target points relative to the current vehicle and the second lateral distance of each lane line relative to the current vehicle, the target lane line closest to the current vehicle is determined; Calculate the third lateral distance from each of the second pre-screened target points to the target lane line and the second average value of the third lateral distance, filter out the second pre-screened target points whose third lateral distance is less than the second average value, and obtain the effective radar target points.

4. The method according to claim 1, characterized in that, The step of combining each visual detection result and each radar detection result of a valid radar trace target, and determining the first matching probability of each combination in the current period based on the weight ratio of the attribute difference between the visual detection result and the radar detection result in each combination, includes: Based on the unique ID carried by each visual detection result and each radar detection result, the radar detection results of each visual detection result and each radar effective point target are combined to determine each combination consisting of the IDs corresponding to the visual detection results and the radar detection results, and the weight value corresponding to each combination. Based on the weight value of each current combination and the weight values ​​of all other combinations that have the same ID as each current combination, the first matching probability of each current combination in the current period is determined.

5. The method according to claim 1, characterized in that, The steps of matching and tracking the target combination determined by the second matching probability, and fusing the visual detection result and the radar detection result corresponding to the target combination, include: Each current combination is sorted according to the second matching probability, and matching and tracking are performed based on the target combination that meets the pre-arranged requirements to obtain matching results including classification categories; Based on the classification category, the visual detection results and radar detection results corresponding to the target combination are fused.

6. The method according to claim 5, characterized in that, The step of fusing the visual detection results and radar detection results corresponding to the target combination based on the classification category includes: The visual detection results and radar detection results corresponding to the target combination are fused to obtain a first fusion result; Based on the classification category, determine the target corresponding to the radar effective point target; Based on the classification category and size information of the target object, the radar detection result corresponding to the radar effective point target is fused with the first fusion result to obtain the second fusion result.

7. A radar and vision fusion device based on binary Bayesian filtering, characterized in that, include: The preprocessing module preprocesses the visual and radar data acquired by the current vehicle to obtain the visual detection results for the current period and the radar detection results for each radar point. The filtering module, based on the current vehicle's motion information, the visual detection results, and the radar detection results, filters out invalid radar markers and determines valid radar marker targets. The first matching module combines each visual detection result and each radar detection result of a radar effective point target, and determines the first matching probability of each combination in the current period based on the weight ratio of the attribute difference between the visual detection result and the radar detection result in each combination. The second matching module, based on the binary Bayesian filtering formula, presets the observation conditions, the first matching probability of each current combination in the current period and the first posterior advantage ratio of each current combination in the historical period, and updates the second posterior advantage ratio of each current combination in multiple periods. The second posterior advantage ratio is converted into the second matching probability of each current combination over multiple periods using an inverse function; The fusion module performs matching and tracking on the target combination determined by the second matching probability, and fuses the visual detection results and the radar detection results corresponding to the target combination.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 6.

9. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-executable instructions that, when invoked and executed by a processor, cause the processor to perform the steps of the method according to any one of claims 1 to 6.

Images