Radar and camera detection target matching method and system in highway scene

Abstract

The invention discloses a radar and camera detection target matching method and system in an expressway scene, and relates to the technical field of target detection and tracking. The method comprises the following steps: acquiring radar detection information and visual detection information; matching the radar detection information and the visual detection information by using a fuzzy matching algorithm; determining the coordinate position of each visual reference line in the actual space; matching the radar detection information with the visual detection information to generate a training data set with a corresponding ID; based on the training data set, a deep neural network MLP is adopted to construct a radar visual mapping model; and converting the radar detection information by using the radar visual mapping model, and binding the radar target ID in the radar detection information and the visual target ID in the visual detection information in real time. Through the fuzzy matching algorithm, the dependence on manual calibration data is reduced, the implementation cost is reduced, and real-time matching of the radar and the target detected by the camera is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection and tracking, and in particular to a method and system for matching radar and camera detection targets in a highway scenario. Background Art

[0002] In modern intelligent transportation systems, target detection and tracking in highway scenarios are key factors in improving road safety and traffic efficiency. To achieve all-round, all-weather traffic monitoring, highways are usually equipped with two sets of radar and camera combinations on smart poles to monitor traffic conditions on the roads on both sides of the smart poles. This multi-sensor combination takes advantage of the respective technical advantages of radar and camera, aiming to improve the limitations of a single sensor under different environmental conditions. Radar has good adaptability to bad weather and light changes, and can provide accurate real-time speed and distance measurements, while cameras can capture the detailed texture and shape of targets through advanced image processing algorithms. However, cameras are highly dependent on ambient lighting and may fail in night scenes; similarly, although radar can provide accurate distance information, it lacks a detailed description of the target's appearance. In order to overcome these limitations, target matching and information fusion between radar and camera have become a research hotspot.

[0003] In the prior art, target matching methods based on radar and camera are mainly divided into three categories: data layer fusion method, feature layer fusion method and decision layer fusion method. Data layer and feature layer fusion methods usually rely on comprehensive analysis of raw data (radar point cloud or raw image) from multiple sensors, which requires complex neural network models to extract and process features, resulting in high computational complexity and difficulty in meeting the strict real-time requirements of highway scenarios. Therefore, the decision layer fusion method is more suitable for real-time monitoring of highways because it makes comprehensive judgments directly based on the detection results of radar and camera. However, these existing fusion methods still face many challenges in highway scenarios. Due to the changes in straight and curved roads and the differences in camera deployment angles, it is difficult to uniformly determine the external parameters of the camera. The existing decision layer fusion method requires a large amount of manual calibration data training for each specific monitoring range, resulting in high labor and time costs. These factors make it difficult to find a unified algorithm or model in the prior art to effectively handle the matching of radar and camera target detection. Therefore, a general and efficient solution needs to be developed. Summary of the invention

[0004] In order to overcome the above problems or at least partially solve the above problems, the present invention provides a method and system for matching radar and camera detection targets in a highway scenario. Through a fuzzy matching algorithm, the dependence on manual calibration data is reduced, the implementation cost is reduced, and real-time matching of targets detected by radar and camera is achieved.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] In a first aspect, the present invention provides a method for matching radar and camera detection targets in a highway scenario, comprising the following steps:

[0007] Based on the radars and cameras installed on the highway, corresponding radar detection information and visual detection information are obtained respectively;

[0008] Using fuzzy matching algorithm, radar detection information and visual detection information are matched in time and space domain to obtain fuzzy matching results;

[0009] According to the fuzzy matching results, the visual reference line is associated with each lane detected by the radar, and the coordinate position of each visual reference line in the actual space is determined;

[0010] According to the coordinate position of the visual reference line in the actual space, the radar detection information and the visual detection information are matched to generate a training data set with corresponding IDs;

[0011] Based on the training data set, a radar visual mapping model is constructed using a deep neural network MLP;

[0012] The radar detection information is transformed using the radar-vision mapping model, and the radar target ID in the radar detection information and the visual target ID in the visual detection information are bound in real time to obtain the corresponding binding result.

[0013] The present invention realizes data set pairing without manual calibration by utilizing radars and cameras installed on the road, thereby pairing radar detection targets with visual detection targets at a lower cost. This not only lays the foundation for subsequent event detection and business logic judgment, but also has the following two advantages compared to the prior art: First, in highway scenarios, the solution is applicable to a variety of road types, does not require additional logic design and manual calibration data, and has a high degree of universality. Secondly, at night or in low light conditions, when visual target information is lost, the radar can still independently provide target detection information to assist monitoring personnel in quickly locating the location of the event. The fuzzy matching strategy and training data set generation method designed for highway scenarios in the present invention effectively improve the efficiency and accuracy of data processing, and provide a solid foundation for multi-sensor fusion applications.

[0014] Based on the first aspect, further, the radar detection information includes a timestamp, a radar target ID, a target real-world coordinate, a target size, and information about the lane where the target is located, which are denoted as T r ,ID r ,P r ,S r ,Lr The visual detection information includes timestamp, visual target ID, visual target detection frame information, target type and target lane information, which are denoted as T c ,ID c ,B c ,C c ,L c .

[0015] Based on the first aspect, further, the method for matching radar detection information and visual detection information in the time and space domain using a fuzzy matching algorithm comprises the following steps:

[0016] Calculate and based on the timestamp T r and T c The difference between the values ​​of the radar detection information and the visual detection information is set to a time window, and the radar detection information and the visual detection information within the time window are filtered out;

[0017] According to the target lane information L r and L c , preliminarily classify the screened radar detection information and visual detection information to generate preliminary classification results;

[0018] Based on the preliminary classification results, the appearance and departure time of the target on each lane are counted, and the longest time when a single target appears on each lane and its corresponding time interval are screened out;

[0019] In the corresponding time interval, combined with the target lane information L r The radar detection information is screened again. If there is only one target in the radar data in a certain time interval and on the corresponding lane, the radar target ID and the visual detection target ID are matched and bound.

[0020] Based on the first aspect, further, the method for associating the visual reference line with each lane detected by the radar according to the fuzzy matching result and determining the coordinate position of each visual reference line in the actual space includes the following steps:

[0021] Based on the video information in the visual detection information, a pixel reference line is set, and a pixel coordinate position of the pixel reference line in the corresponding video information is recorded;

[0022] According to the fuzzy matching results, the lane is used as the classification basis, and all the targets that have been successfully fuzzy matched on the corresponding lane are recorded;

[0023] Based on the visual target detection box information, track the center point of the visual detection box of the corresponding target. When the center point passes through the pixel reference line, record the real-world coordinate value in the corresponding radar detection information, and construct a mapping relationship from the pixel reference line coordinates to the real-world coordinates.

[0024] Based on the first aspect, further, the method for matching radar detection information with visual detection information according to the coordinate position of the visual reference line in the actual space to generate a training data set with a corresponding ID includes the following steps:

[0025] Classify radar detection information and visual detection information according to lanes;

[0026] When the center point of the detection frame of the visual target passes through the pixel reference line, the nearest radar target is found according to the real-world coordinate value corresponding to the corresponding lane, and the radar target ID is bound to the visual target ID to obtain the corresponding binding result;

[0027] A training data set including matched target IDs and associated information thereof is generated based on multiple binding results.

[0028] Based on the first aspect, further, the method for transforming the radar detection information using the radar visual mapping model and binding the radar target ID in the radar detection information and the visual target ID in the visual detection information in real time comprises the following steps:

[0029] The radar detection information is converted into corresponding visual target frame information using the radar visual mapping model;

[0030] Use the Kalman filter to perform smoothing combined with the visual target frame information to obtain the corresponding target detection frame information;

[0031] The radar target ID and the visual target ID are bound according to the target detection frame information and the visual target detection frame information.

[0032] Based on the first aspect, further, the radar and camera detection target matching method in the highway scene also includes the following steps:

[0033] Analyze the binding results, generate and map the analysis results to the business platform, and realize multiple functions such as traffic warning, flow statistics, and accident detection.

[0034] In a second aspect, the present invention provides a radar and camera detection target matching system in a highway scenario, including an information acquisition module, a fuzzy matching module, a reference line calibration module, a data set generation module, a model construction module and a target matching binding module, wherein:

[0035] An information acquisition module, used to acquire corresponding radar detection information and visual detection information based on radars and cameras installed on the highway;

[0036] A fuzzy matching module is used to match radar detection information and visual detection information in the time and space domain using a fuzzy matching algorithm to obtain a fuzzy matching result;

[0037] A reference line calibration module is used to associate the visual reference line with each lane detected by the radar according to the fuzzy matching results, and determine the coordinate position of each visual reference line in the actual space;

[0038] A data set generation module is used to match the radar detection information with the visual detection information according to the coordinate position of the visual reference line in the actual space, and generate a training data set with a corresponding ID;

[0039] A model building module is used to build a radar visual mapping model based on the training data set using a deep neural network MLP;

[0040] The target matching binding module is used to transform the radar detection information using the radar vision mapping model, and to bind the radar target ID in the radar detection information and the visual target ID in the visual detection information in real time to obtain the corresponding binding result.

[0041] This system, through the cooperation of multiple modules such as information acquisition module, fuzzy matching module, reference line calibration module, data set generation module, model construction module and target matching binding module, designs fuzzy matching strategies and training data set generation methods for highway scenarios, effectively improves the efficiency and accuracy of data processing, and provides a solid foundation for multi-sensor fusion applications.

[0042] In a third aspect, the present application provides an electronic device, comprising a memory for storing one or more programs; a processor; when the one or more programs are executed by the processor, the method as described in any one of the first aspects above is implemented.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of the first aspects above.

[0044] The present invention has at least the following advantages or beneficial effects:

[0045] 1. The present invention uses radars and cameras installed on the road to achieve data set pairing without manual calibration, thereby pairing radar detection targets with visual detection targets at a lower cost. The fuzzy matching strategy and training data set generation method designed for highway scenarios effectively improve the efficiency and accuracy of data processing, and provide a solid foundation for multi-sensor fusion applications.

[0046] 2. Based on the radar prediction frame, the Kalman filter is combined for smooth tracking, which effectively reduces the impact of noise, thereby providing a more stable and continuous target position output. This method improves the reliability and accuracy of target detection and is suitable for real-time applications in complex dynamic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 This is a flow chart of a method for matching radar and camera detection targets in a highway scenario according to an embodiment of the present invention;

[0049] Figure 2 This is a principle block diagram of a radar and camera target detection matching system in a highway scenario according to an embodiment of the present invention;

[0050] Figure 3 A structural block diagram of an electronic device provided by an embodiment of the present invention.

[0051] Explanation of the accompanying drawings: 100, information acquisition module; 200, fuzzy matching module; 300, reference line calibration module; 400, data set generation module; 500, model building module; 600, target matching binding module; 101, memory; 102, processor; 103, communication interface. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0053] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0055] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0056] In the description of the embodiments of the present invention, "plurality" means at least 2.

[0057] Example:

[0058] like Figure 1 As shown, in a first aspect, an embodiment of the present invention provides a method for matching radar and camera detection targets in a highway scenario, comprising the following steps:

[0059] S1. Based on the radar and camera installed on the highway, the corresponding radar detection information and visual detection information are obtained respectively; the above radar detection information includes timestamp, radar target ID, target real world coordinates, target size and target lane information, which are denoted as T r ,ID r ,P r ,S r ,L r The visual detection information includes timestamp, visual target ID, visual target detection frame information, target type and target lane information, which are denoted as T c ,ID c ,B c ,C c ,L c .

[0060] S2. Using a fuzzy matching algorithm, the radar detection information and the visual detection information are matched in the time and space domain to obtain a fuzzy matching result;

[0061] Further, including: calculating and according to the timestamp T r and T c The time window is set to filter out the radar detection information and visual detection information within the time window; according to the lane information L where the target is located r and L c, perform preliminary classification on the selected radar detection information and visual detection information, and generate preliminary classification results; based on the preliminary classification results, count the appearance time and departure time of the target on each lane, and select the longest time when a single target appears on each lane and its corresponding time interval; within the corresponding time interval, combine the lane information L where the target is located r The radar detection information is screened again. If there is only one target in the radar data in a certain time interval and on the corresponding lane, the radar target ID and the visual detection target ID are matched and bound.

[0062] In some embodiments of the present invention, first, according to the timestamp T r and T c The difference between lane information L is used to set a time window and filter out the radar and camera detection results within the time window. This step ensures that the temporally related data can be aligned. r and L c , preliminarily classifies the detection targets of the radar and camera for subsequent matching analysis within the same lane.

[0063] On the basis of the above steps, the appearance time and departure time of the target on each lane are counted, and the longest time when a single target appears on each lane and its corresponding time interval are screened out. For these time intervals, the data is screened again on the radar side in combination with the lane information. If there is only one target in the radar data in a certain time interval and the corresponding lane, the radar target ID and the visual detection target ID can be bound. This process is repeated to select multiple sets of data, which are sorted from long to short according to the longest duration of a single target appearing in each lane to ensure the accuracy and stability of the matching. This step method effectively matches highly reliable target correspondences by focusing on analyzing the duration of a single target in a specific lane and combining time and space information, providing a stable foundation for downstream data processing.

[0064] S3, according to the fuzzy matching result, associating the visual reference line with each lane detected by the radar, and determining the coordinate position of each visual reference line in the actual space;

[0065] Furthermore, it includes: setting a pixel reference line based on the video information in the visual detection information, and recording the pixel coordinate position of the pixel reference line in the corresponding video information; based on the fuzzy matching result, taking the lane as the classification basis, recording all the targets that have successfully performed fuzzy matching on the corresponding lane; based on the visual target detection frame information, tracking the center point of the visual detection frame of the corresponding target, when the center point passes through the pixel reference line, recording its corresponding real-world coordinate value in the radar detection information, and constructing a mapping relationship from the pixel reference line coordinates to the real-world coordinates.

[0066] In some embodiments of the present invention, a reference line is manually selected in the video information captured by the camera, and the pixel coordinate position of the line in the video is recorded. Based on the lane classification, all targets that are successfully fuzzy matched on the lane in step S2 are recorded. The center points of the visual detection frame of the tracked target are recorded when these center points pass through the previously selected pixel reference line. The real-world coordinate values ​​in the corresponding radar target detection data are recorded. The multiple sets of data collected in this process are averaged to construct a mapping relationship from the pixel reference line coordinates to the real-world coordinates. This mapping relationship provides the necessary reference for subsequent precise target positioning and identification, enabling the system to accurately link visual observations with the actual road environment.

[0067] S4. According to the coordinate position of the visual reference line in the actual space, the radar detection information and the visual detection information are matched to generate a training data set with a corresponding ID;

[0068] Furthermore, it includes: classifying radar detection information and visual detection information according to lanes; when the center point of the detection frame of the visual target passes through the pixel reference line, finding the nearest radar target according to the real-world coordinate value corresponding to the corresponding lane, and binding the radar target ID with the visual target ID to obtain a corresponding binding result; generating a training data set containing matched target IDs and their associated information based on multiple binding results.

[0069] In some embodiments of the present invention, the above-mentioned pixel reference line is used to automatically match the data of the millimeter wave radar and the camera to generate a training data set with a corresponding ID. First, all detected data are classified and managed according to the lane. Then, when the center point of the detection frame of the visual target passes through the preset pixel reference line, the real-world coordinate value corresponding to the lane obtained in the above steps is used to find the nearest radar detection target. If the distance between the coordinate position of the radar target and the real coordinate reference position of the lane line is less than a specified threshold, it can be confirmed that the two targets belong to the same object. Therefore, the radar target ID is bound to the visual target ID. Through this process, a training data set containing the matched target ID and its associated information is generated, providing accurate and reliable input for subsequent algorithm training and model optimization.

[0070] S5. Based on the training data set, a deep neural network MLP is used to build a radar visual mapping model;

[0071] In some embodiments of the present invention, a deep neural network (MLP) is used to train the generated data set to establish an effective mapping model to ensure that the target frame information in the visual domain can be output using the radar detection data. The specific steps are as follows: The data part of the training set contains the target coordinates, target size, and lane information processed by one-hot encoding in the radar target detection data. The label information of the data set (i.e., the output of the model) is the target detection frame information of the visual detection target that matches the radar data. During the training process, the stochastic gradient descent (SGD) optimization algorithm is used to adjust and optimize the model parameters to improve the accuracy and robustness of the model. In this way, the model can effectively convert radar information into usable data to assist visual detection, providing a solid foundation for real-time applications of multi-sensor fusion.

[0072] S6. Use the radar vision mapping model to transform the radar detection information, bind the radar target ID in the radar detection information and the visual target ID in the visual detection information in real time, and obtain a corresponding binding result.

[0073] Further, it includes: using a radar visual mapping model to convert radar detection information into corresponding visual target frame information; using a Kalman filter to perform smoothing in combination with the visual target frame information to obtain corresponding target detection frame information; binding the radar target ID and the visual target ID according to the target detection frame information and the visual target detection frame information. In this process, the target attributes of the radar (such as real-time speed, longitude and latitude, and coordinates in the real world) are associated with the visual detection target.

[0074] In some embodiments of the present invention, in real-time detection of visual targets, radar data is converted into corresponding visual target frame information with the help of a trained mapping model. This conversion is a basic step to ensure that the radar detection results can be synchronously reflected in visual monitoring. The detection frame obtained by the mapping model is smoothed by applying a Kalman filter. The Kalman filter effectively reduces the impact of noise and jitter on the target position, and provides a more stable and continuous target position output. According to the predicted target detection frame information after the radar data is mapped, the pixel distance between the center point of the prediction frame and the center point of the visual target detection frame, as well as the overlap degree between the prediction frame and the visual target frame, is used to preliminarily determine the binding relationship of the target ID. Through multiple consecutive detections, if the ID binding relationship remains unchanged, it is confirmed that the relationship is valid. If a new binding request appears at this time (such as the visual detection target changes its ID due to factors such as occlusion), the original binding is released, and whether a new ID binding relationship is established is re-determined according to the above method.

[0075] The present invention realizes data set pairing without manual calibration by utilizing radars and cameras installed on the road, thereby pairing radar detection targets with visual detection targets at a lower cost. This not only lays the foundation for subsequent event detection and business logic judgment, but also has the following two advantages compared to the prior art: First, in highway scenarios, the solution is applicable to a variety of road types, without the need for additional logic design and manual calibration data, and has a high degree of universality. Secondly, at night or in low light conditions, when visual target information is lost, the radar can still independently provide target detection information to assist monitoring personnel in quickly locating the location of the event. In terms of algorithm improvement, by introducing the Kalman filter, the present invention provides a more stable and continuous target position output. Even when the visual target ID jumps due to factors such as occlusion, the algorithm can quickly rebind a new matching relationship to ensure the reliability and real-time performance of the system.

[0076] The fuzzy matching strategy and training data set generation method designed for highway scenarios in the present invention effectively improve the efficiency and accuracy of data processing and provide a solid foundation for multi-sensor fusion applications.

[0077] Based on the first aspect, further, the radar and camera detection target matching method in the highway scene also includes the following steps:

[0078] Analyze the binding results, generate and map the analysis results to the business platform, and realize multiple functions such as traffic warning, flow statistics, and accident detection.

[0079] In some embodiments of the present invention, the binding results are further analyzed. Event detection is performed based on target type, speed, lane and other information, and the results are mapped to the business platform to achieve early warning, traffic statistics, accident detection and other functions, thereby improving the overall intelligence and response speed in the transportation field.

[0080] like Figure 2 As shown, in the second aspect, an embodiment of the present invention provides a radar and camera detection target matching system in a highway scene, including an information acquisition module 100, a fuzzy matching module 200, a reference line calibration module 300, a data set generation module 400, a model construction module 500 and a target matching binding module 600, wherein:

[0081] The information acquisition module 100 is used to acquire corresponding radar detection information and visual detection information based on radars and cameras installed on the highway;

[0082] A fuzzy matching module 200 is used to match the radar detection information and the visual detection information in the time and space domain using a fuzzy matching algorithm to obtain a fuzzy matching result;

[0083] A reference line calibration module 300 is used to associate a visual reference line with each lane detected by the radar according to the fuzzy matching result, and determine the coordinate position of each visual reference line in the actual space;

[0084] The data set generation module 400 is used to match the radar detection information with the visual detection information according to the coordinate position of the visual reference line in the actual space, and generate a training data set with a corresponding ID;

[0085] A model building module 500 is used to build a radar visual mapping model based on a training data set using a deep neural network MLP;

[0086] The target matching binding module 600 is used to transform the radar detection information using the radar vision mapping model, and to bind the radar target ID in the radar detection information and the visual target ID in the visual detection information in real time to obtain a corresponding binding result.

[0087] Through the cooperation of multiple modules such as the information acquisition module 100, the fuzzy matching module 200, the reference line calibration module 300, the data set generation module 400, the model construction module 500 and the target matching binding module 600, the system designs a fuzzy matching strategy and a training data set generation method for highway scenarios, which effectively improves the efficiency and accuracy of data processing and provides a solid foundation for multi-sensor fusion applications.

[0088] like Figure 3 As shown, in a third aspect, an embodiment of the present application provides an electronic device, which includes a memory 101 for storing one or more programs and a processor 102. When the one or more programs are executed by the processor 102, any method in the first aspect described above is implemented.

[0089] The memory 101, the processor 102 and the communication interface 103 are directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, and the processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used to communicate signaling or data with other node devices.

[0090] Among them, the memory 101 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0091] The processor 102 may be an integrated circuit chip with signal processing capability. The processor 102 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0092] In the embodiments provided in the present application, it should be understood that the disclosed method and system can also be implemented in other ways. The method and system embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the method and system, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0093] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0094] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by the processor 102, a method as in any one of the first aspects described above is implemented. If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk.

[0095] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0096] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present application. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A method for matching radar and camera detection targets in a highway scenario, characterized in that: The following steps are involved: Based on the radars and cameras installed on the highway, corresponding radar detection information and visual detection information are obtained respectively; Using fuzzy matching algorithm, radar detection information and visual detection information are matched in time and space domain to obtain fuzzy matching results; According to the fuzzy matching results, the visual reference line is associated with each lane detected by the radar, and the coordinate position of each visual reference line in the actual space is determined; According to the coordinate position of the visual reference line in the actual space, the radar detection information and the visual detection information are matched to generate a training data set with corresponding IDs; Based on the training data set, a radar visual mapping model is constructed using a deep neural network MLP; The radar detection information is transformed using the radar-vision mapping model, and the radar target ID in the radar detection information and the visual target ID in the visual detection information are bound in real time to obtain the corresponding binding result.

2. The method for matching radar and camera detection targets in a highway scene according to claim 1 is characterized in that: The radar detection information includes timestamp, radar target ID, target real world coordinates, target size and target lane information, which are denoted as T r ,ID r ,P r ,S r ,L r The visual detection information includes timestamp, visual target ID, visual target detection frame information, target type and target lane information, which are denoted as T c ,ID c ,B c ,C c ,L c .

3. The method for matching radar and camera detection targets in a highway scene according to claim 2 is characterized in that: The method for matching radar detection information and visual detection information in the time and space domain by using a fuzzy matching algorithm comprises the following steps: Calculate and based on the timestamp T r and T c The difference between the values ​​of the radar detection information and the visual detection information is set to a time window, and the radar detection information and the visual detection information within the time window are filtered out; According to the target lane information L r and L c , preliminarily classify the screened radar detection information and visual detection information to generate preliminary classification results; Based on the preliminary classification results, the appearance and departure time of the target on each lane are counted, and the longest time when a single target appears on each lane and its corresponding time interval are screened out; In the corresponding time interval, combined with the target lane information L r The radar detection information is screened again. If there is only one target in the radar data in a certain time interval and on the corresponding lane, the radar target ID and the visual detection target ID are matched and bound.

4. The method for matching radar and camera detection targets in a highway scene according to claim 2 is characterized in that: The method of associating the visual reference line with each lane detected by the radar according to the fuzzy matching result and determining the coordinate position of each visual reference line in the actual space comprises the following steps: Based on the video information in the visual detection information, a pixel reference line is set, and a pixel coordinate position of the pixel reference line in the corresponding video information is recorded; According to the fuzzy matching results, the lane is used as the classification basis, and all the targets that have been successfully fuzzy matched on the corresponding lane are recorded; Based on the visual target detection box information, track the center point of the visual detection box of the corresponding target. When the center point passes through the pixel reference line, record the real-world coordinate value in the corresponding radar detection information, and construct a mapping relationship from the pixel reference line coordinates to the real-world coordinates.

5. The method for matching radar and camera detection targets in a highway scene according to claim 4 is characterized in that: The method of matching radar detection information with visual detection information according to the coordinate position of the visual reference line in the actual space to generate a training data set with a corresponding ID comprises the following steps: Classify radar detection information and visual detection information according to lanes; When the center point of the detection frame of the visual target passes through the pixel reference line, the nearest radar target is found according to the real-world coordinate value corresponding to the corresponding lane, and the radar target ID is bound to the visual target ID to obtain the corresponding binding result; A training data set including matched target IDs and associated information thereof is generated based on multiple binding results.

6. The method for matching radar and camera detection targets in a highway scene according to claim 2, characterized in that: The method for transforming radar detection information by using a radar visual mapping model and binding the radar target ID in the radar detection information and the visual target ID in the visual detection information in real time comprises the following steps: The radar detection information is converted into corresponding visual target frame information using the radar visual mapping model; Use the Kalman filter to perform smoothing combined with the visual target frame information to obtain the corresponding target detection frame information; The radar target ID and the visual target ID are bound according to the target detection frame information and the visual target detection frame information.

7. The method for matching radar and camera detection targets in a highway scene according to claim 1, characterized in that: The following steps are also included: Analyze the binding results, generate and map the analysis results to the business platform, and realize multiple functions such as traffic warning, flow statistics, and accident detection.

8. A radar and camera detection target matching system in a highway scenario, characterized in that: It includes information acquisition module, fuzzy matching module, reference line calibration module, data set generation module, model building module and target matching binding module, among which: An information acquisition module is used to acquire corresponding radar detection information and visual detection information based on radars and cameras installed on the highway; A fuzzy matching module is used to match radar detection information and visual detection information in the time and space domain using a fuzzy matching algorithm to obtain a fuzzy matching result; A reference line calibration module is used to associate the visual reference line with each lane detected by the radar according to the fuzzy matching results, and determine the coordinate position of each visual reference line in the actual space; A data set generation module is used to match the radar detection information with the visual detection information according to the coordinate position of the visual reference line in the actual space, and generate a training data set with a corresponding ID; A model building module is used to build a radar visual mapping model based on the training data set using a deep neural network MLP; The target matching binding module is used to transform the radar detection information using the radar vision mapping model, and to bind the radar target ID in the radar detection information and the visual target ID in the visual detection information in real time to obtain the corresponding binding result.

9. An electronic device, characterized in that: include: A memory for storing one or more programs; processor; When the one or more programs are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images