Traffic accident early warning method and device based on radar and vision fusion of object imaging and medium

Abstract

The invention discloses a traffic accident early warning method and device based on radar and vision fusion of object imaging and a medium, and can be applied to the technical field of traffic data processing. The method comprises the following steps: acquiring camera acquisition data and radar acquisition data of a to-be-early-warned area according to a preset sampling period, and then preprocessing the camera acquisition data based on an object imaging algorithm to obtain first target data; and processing the radar acquisition data to obtain second target data and distance data, angle data and speed data corresponding to the second target data, and projecting the second target data to the camera acquisition data to obtain a target region of interest corresponding to the second target data so as to realize space-time synchronization of the radar and camera data. And then, after the target region of interest and the first target data are fused to obtain a target fusion result, traffic early warning is performed according to the target fusion result, the distance data, the angle data and the speed data, so that the accuracy of traffic early warning can be effectively improved.

Description

Technical Field

[0001] This application relates to the technical field of traffic data processing, and particularly to a traffic accident warning method, device and medium based on the fusion of radar and vision for object imaging. Background Art

[0002] In the related art, in modern urban traffic management, the intelligent transportation system improves traffic efficiency, ensures driving safety and optimizes resource allocation through intelligent means. However, the existing intelligent transportation system analyzes and warns traffic data in a single way. Among them, when collecting, analyzing and warning traffic data through a single sensor such as a camera or a radar, since the performance of the camera is easily affected by light, weather and complex backgrounds, the collected traffic data may be unclear, thus resulting in low accuracy of analysis and warning; when collecting data through a single radar, due to the low radar resolution, the collected data is not clear, resulting in low accuracy of analysis and warning.

[0003] In summary, the technical problems existing in the related art need to be improved. Summary of the Invention

[0004] The main purpose of the embodiments of this application is to propose a traffic accident warning method, device and medium based on the fusion of radar and vision for object imaging, which can effectively improve the accuracy of traffic warning.

[0005] To achieve the above object, on the one hand, an embodiment of this application proposes a traffic accident warning method based on the fusion of radar and vision for object imaging, and the method includes the following steps:

[0006] Obtain the camera acquisition data and radar acquisition data of the area to be warned according to a preset sampling period;

[0007] Preprocess the camera acquisition data based on the object imaging algorithm to obtain first target data;

[0008] Process the radar acquisition data to obtain second target data and the distance data, angle data and speed data corresponding to the second target data;

[0009] Project the second target data onto the camera acquisition data to obtain the target region of interest corresponding to the second target data;

[0010] Fuse the target region of interest and the first target data to obtain a target fusion result;

[0011] Perform traffic warning according to the target fusion result, the distance data, the angle data and the speed data.

[0012] In some embodiments, the object-based imaging algorithm preprocesses the data collected by the camera to obtain first target data, including:

[0013] Input the data collected by the camera into a preset feature filter for feature extraction;

[0014] Based on the extracted features, determine the corresponding target image of interest in the data collected by the camera as the first target data.

[0015] In some embodiments, processing the data collected by the radar to obtain second target data and the distance data, angle data, and speed data corresponding to the second target data includes:

[0016] Perform data parsing on the data collected by the radar to obtain first radar point cloud data;

[0017] Perform clustering processing on the first radar point cloud data to obtain second radar point cloud data;

[0018] Perform target screening based on the second radar point cloud data to obtain the second target data and the distance data, angle data, and speed data corresponding to the second target data.

[0019] In some embodiments, performing target screening based on the second radar point cloud data to obtain the second target data and the distance data, angle data, and speed data corresponding to the second target data includes:

[0020] Determine the current type of the current target according to the second radar point cloud data;

[0021] Determine that the current type is a preset type, and delete the second radar point cloud data corresponding to the current target. The preset types include an empty target type, a false target type, and a non-dangerous target type;

[0022] Determine the second target data and the distance data, angle data, and speed data corresponding to the second target data from the remaining second radar point cloud data.

[0023] In some embodiments, projecting the second target data onto the data collected by the camera to obtain the target region of interest corresponding to the second target data includes:

[0024] Construct a radar coordinate system corresponding to the second target data;

[0025] Construct a pixel coordinate system corresponding to the data collected by the camera;

[0026] Determine the conversion relationship between the radar coordinate system and the pixel coordinate system according to geometric and projection principles;

[0027] Project the second target data into the pixel coordinate system according to the conversion relationship to obtain the target region of interest corresponding to the second target data.

[0028] In some embodiments, the fusing the target region of interest and the first target data to obtain a target fusion result includes:

[0029] Using the Hungarian algorithm to determine the matching pairs formed by the target region of interest and the target regions corresponding to the first target data;

[0030] Calculate the intersection over union of the target region of interest and the target regions corresponding to the first target data in the matching pairs;

[0031] When the intersection over union is greater than the intersection over union threshold, generate the target fusion result according to the target region of interest and the target regions corresponding to the first target data corresponding to the intersection over union.

[0032] In some embodiments, the using the Hungarian algorithm to determine the matching pairs formed by the target region of interest and the target regions corresponding to the first target data includes:

[0033] Obtain the first centroid coordinates of the target region of interest and the second centroid coordinates of the target regions corresponding to the first target data;

[0034] According to the first centroid coordinates and the second centroid coordinates, perform an association operation between the target region of interest and the target regions corresponding to the first target data using the Hungarian algorithm to obtain the matching pairs.

[0035] To achieve the above object, another aspect of the embodiments of the present application provides a traffic accident warning device based on radar and vision fusion for object imaging, the device includes:

[0036] A first module, configured to obtain camera acquisition data and radar acquisition data of a region to be warned according to a preset sampling period;

[0037] A second module, configured to preprocess the camera acquisition data based on an object imaging algorithm to obtain first target data;

[0038] A third module, configured to process the radar acquisition data to obtain second target data and distance data, angle data, and speed data corresponding to the second target data;

[0039] A fourth module, configured to project the second target data onto the camera acquisition data to obtain a target region of interest corresponding to the second target data;

[0040] The fifth module is used to fuse the target region of interest and the first target data to obtain a target fusion result;

[0041] The sixth module is used to perform traffic warning according to the target fusion result, the distance data, the angle data, and the speed data.

[0042] To achieve the above object, on the other hand, an embodiment of the present application provides a computer device, including:

[0043] At least one processor;

[0044] At least one memory for storing at least one program;

[0045] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0046] To achieve the above object, on the other hand, an embodiment of the present application provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above method is implemented.

[0047] The embodiments of the present application at least include the following beneficial effects: The present application provides a traffic accident warning method, device, and medium based on the fusion of radar and vision with object imaging. This solution realizes the time synchronization of the camera acquisition data and the radar acquisition data by acquiring the camera acquisition data and the radar acquisition data of the area to be warned according to a preset sampling period. Then, after preprocessing the camera acquisition data based on the object imaging algorithm to obtain the first target data, the radar acquisition data is processed to obtain the second target data and the corresponding distance data, angle data, and speed data of the second target data. Then, the second target data is projected onto the camera acquisition data to obtain the target region of interest corresponding to the second target data to achieve the spatial synchronization of the radar data and the camera data. Then, after fusing the target region of interest and the first target data to obtain the target fusion result, traffic warning is performed according to the target fusion result, the distance data, the angle data, and the speed data, thereby effectively improving the accuracy of traffic warning. Description of the Drawings

[0048] Figure 1 is a flowchart of the traffic accident warning method based on the fusion of radar and vision with object imaging provided by the embodiment of the present application;

[0049] Figure 2 is a schematic diagram of the interaction framework of the traffic accident warning method based on the fusion of radar and vision with object imaging provided by the embodiment of the present application;

[0050] Figure 3It is a schematic diagram of an output strategy based on a target fusion result provided by an embodiment of the present application;

[0051] Figure 4 It is a schematic diagram of processing camera-acquired data using an object imaging algorithm provided by an embodiment of the present application;

[0052] Figure 5 It is a flowchart of target screening for radar-acquired data provided by an embodiment of the present application;

[0053] Figure 6 It is a schematic diagram of spatial synchronization between radar-acquired data and camera-acquired data provided by an embodiment of the present application;

[0054] Figure 7 It is a schematic diagram of target association provided by an embodiment of the present application;

[0055] Figure 8 It is a schematic diagram of traffic warning provided by an embodiment of the present application;

[0056] Figure 9 It is a schematic diagram of the structure of a traffic accident warning device based on radar and vision fusion using object imaging provided by an embodiment of the present application. Detailed implementation manners

[0057] In order to make the purpose, technical solutions and advantages of the present application clearer, the following further details the present application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. When the following description involves the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the embodiments of the present application. They are only examples of devices and methods that are consistent with some aspects of the embodiments of the present application.

[0058] It can be understood that the terms "first", "second", etc. used in the present application can be used in this document to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the words "if", "when" as used herein can be interpreted as "when...", "when...", or "in response to determining".

[0059] The terms "at least one", "a plurality", "each", "any one", etc. used in the present application, at least one includes one, two or more than two, a plurality includes two or more than two, each refers to each of the corresponding plurality, and any one refers to any one of the plurality.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0061] Before elaborating on the embodiments of this application in detail, some nouns and terms involved in the embodiments of this application are first explained. The nouns and terms involved in the embodiments of this application are applicable to the following explanations:

[0062] The Hungarian algorithm is a combinatorial optimization algorithm that solves the task assignment problem in polynomial time and has promoted the subsequent primal-dual method.

[0063] In the related art, in modern urban traffic management, the intelligent transportation system improves traffic efficiency, ensures driving safety, and optimizes resource allocation through intelligent means. However, the existing intelligent transportation systems analyze and warn traffic data in a single way. Among them, when collecting, analyzing, and warning traffic data through a single sensor such as a camera or radar, since the performance of the camera is easily affected by light, weather, and complex backgrounds, the collected traffic data may be unclear, which may lead to low accuracy in analysis and warning; when collecting data through a single radar, due to the low radar resolution, the collected data is not clear, resulting in low accuracy in analysis and warning.

[0064] In view of this, the embodiments of this application provide a traffic accident warning method, device, and medium based on the fusion of radar and vision for object imaging. This solution can effectively improve the accuracy of traffic warning.

[0065] The traffic accident warning method based on the fusion of radar and vision for object imaging provided by the embodiments of this application relates to the technical field of traffic data processing. The traffic accident warning method based on the fusion of radar and vision for object imaging provided by the embodiments of this application can be applied to a terminal, a server, or software running on a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a vehicle terminal, etc., but is not limited thereto; the server side can be configured as an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the traffic accident warning method based on the fusion of radar and vision for object imaging, etc., but is not limited to the above forms.

[0066] This application can be used in numerous general or specific computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and so on. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This application can also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0067] The embodiments of this application will be specifically described below with reference to the accompanying drawings:

[0068] Figure 1 is an alternative flowchart of the traffic accident warning method based on object imaging radar and vision fusion provided by the embodiments of this application, Figure 1 The method in may include but is not limited to steps S110 to S160:

[0069] Step S110, obtain the camera acquisition data and radar acquisition data of the area to be warned according to a preset sampling period;

[0070] Step S120, preprocess the camera acquisition data based on the object imaging algorithm to obtain the first target data;

[0071] Step S130, process the radar acquisition data to obtain the second target data and the distance data, angle data, and speed data corresponding to the second target data;

[0072] Step S140, project the second target data onto the camera acquisition data to obtain the region of interest corresponding to the second target data;

[0073] Step S150, fuse the region of interest and the first target data to obtain the target fusion result;

[0074] Step S160, perform traffic warning according to the target fusion result, distance data, angle data, and speed data.

[0075] It can be understood that the method of the embodiments of this application is applied to Figure 2When the interactive framework shown is in the data acquisition stage, the camera is responsible for capturing image information in the traffic environment as the camera acquisition data, and the radar sensor collects data such as the distance, speed, and angle of the target as the radar acquisition data. Specifically, in this embodiment, the time synchronization of the camera and radar data can be achieved by setting a unified sampling period, and the spatial synchronization of the millimeter-wave radar and the camera is achieved through coordinate transformation. In this embodiment, the radar data is preprocessed to filter out interference data such as static targets and false targets to ensure the accuracy of the target information. At the same time, an object imaging algorithm is used for the visual data to generate specific targets of interest. In the data association stage, not only the Hungarian algorithm is used to associate the synchronized detection results, but also the intersection over union (IoU) is calculated for each matching pair, that is, the IoU between the target area obtained by object imaging and the target of interest generated by the radar. An IoU threshold is set to further determine whether the two targets come from the same target. Finally, the target information is judged and output according to the set IoU threshold as the target fusion result. As Figure 3 shown, if it is determined based on the target fusion result that the millimeter-wave radar data is not associated with the target in the camera data, the relative distance and relative speed between the targets are output through the radar data; if it is determined based on the target fusion result that the millimeter-wave radar data is associated with the corresponding target in the camera data, the target category and the external contour are output through the camera data, and the relative distance and relative speed are output through the radar data at the same time; if it is determined based on the target fusion result that the camera data is not associated with the target in the millimeter-wave radar data, the target category and the external contour are output through the camera data. In the embodiment of the present application, the distance and speed between the real-time monitoring vehicles can be determined based on the final target fusion result, and then the potential collision risk can be predicted. According to the collision risk level, a warning message is sent to the traffic management center in real time, and a warning is issued to the relevant vehicles to prompt the driver to take evasive actions, thereby effectively preventing and reducing the occurrence of traffic accidents.

[0076] In the embodiment of the present application, in order to ensure the synchronization of the camera acquisition data and the radar acquisition data, in this embodiment, the time of the camera is first synchronized with the system time through software. Subsequently, the acquisition software is opened and the CAN channel of the millimeter-wave radar is started to make the radar enter the normal working state. During the data acquisition process, the millimeter-wave radar and the camera simultaneously acquire the data of the current frame, and each frame of data is marked with an accurate timestamp. After the acquisition is completed, a suitable sampling period is determined according to the sampling periods of the radar and the camera. This period can cover the least common multiple of the two. The starting point of data fusion is set as the first same timestamp after the start recorded in the radar data frame and the video data frame. Taking this as the reference point, the data points with synchronized timestamps are matched within each selected sampling period to complete the time synchronization of the radar acquisition data and the camera acquisition data.

[0077] In the embodiment of the present application, for the camera acquisition data, asFigure 4 As shown, the data collected by the camera can be input into a preset feature filter for feature extraction, and then the corresponding target image of interest in the camera-collected data can be determined based on the extracted features as the first target data. Specifically, in this embodiment, a large amount of labeled data is first used to train the neural network model so that the neural network model learns and obtains the features that can represent the target of interest. During the training process, the neural network model automatically learns the convolution kernel parameters, thereby capturing the key features of the target object. Subsequently, these convolution kernel parameters are used to construct a preset feature filter for identifying and extracting the target of interest to output the identified target image of interest as the first target data.

[0078] In this embodiment, by using object imaging technology to perform road target detection on the real-time collected camera data, not only the accuracy of target detection is improved, but also data compression is achieved by significantly reducing the amount of processed image data, thereby further improving the processing speed and efficiency of the system.

[0079] It can be understood that for the radar-collected data, after parsing the radar-collected data to obtain the first radar point cloud data in this embodiment, the first radar point cloud data is clustered to obtain the second radar point cloud data, and then target screening is performed according to the second radar point cloud data to obtain the second target data and the distance data, angle data, and speed data corresponding to the second target data. In the embodiments of the present application, since the millimeter-wave radar point cloud data corresponding to the radar-collected data is usually scattered and the amount of single-frame point cloud data is small, it is not easy to filter out the noise points. However, continuous multi-frame data can show an aggregated phenomenon in space. Therefore, according to the characteristic that the multi-frame point cloud data of the target is relatively concentrated in spatial density, a density-based clustering algorithm is used to process the radar data. Specifically, this embodiment adopts the DBSCAN clustering algorithm, which is a density clustering algorithm that does not require presetting the number of clusters and can better adapt to the existence of different density clusters and effectively process the noise. In the embodiments of the present application, the core of the DBSCAN algorithm is to construct clusters by identifying core points and exploring their density reachability. The algorithm first defines the points that meet specific conditions within the clustering neighborhood as the core points of the clustering, and the other points are classified as non-core points. Subsequently, the algorithm constructs neighborhoods around each core point and identifies the non-core points located within the neighborhoods of these core points as the edges, and the points that neither belong to the core points nor are within the neighborhoods of any core points are marked as noise points. This process effectively divides the data into several clusters and identifies the points that are not included in any cluster as noise, thereby completing the clustering task.

[0080] It can be understood that, as Figure 5As shown in the figure, after the clustering operation of the radar-acquired data is completed in this embodiment, the current type of the current target is determined according to the second radar point cloud data; then, if the current type is a preset type, the second radar point cloud data corresponding to the current target is deleted, and the second target data and the distance data, angle data, and speed data corresponding to the second target data are determined from the remaining second radar point cloud data. Among them, the preset types include an empty target type, a false target type, and a non-dangerous target type. In the embodiments of the present application, since the detection range of the millimeter-wave radar is relatively wide, many interference targets will inevitably be detected while obtaining target information during the detection process. Therefore, in this embodiment, the interference targets other than the effective moving targets are classified into: empty targets, false targets, and non-dangerous targets according to the actual situation. The existence of these targets will increase the computational complexity of the subsequent fusion algorithm, reduce the real-time performance of the algorithm, make it difficult for the radar to accurately detect the target in a short time, and easily lead to false alarms and misoperations.

[0081] Specifically, an empty target refers to a target whose detected distance, relative speed, and azimuth angle information are 0, and it is directly filtered out. A false target refers to a non-target object that is misidentified as a target by the radar system due to various factors such as environmental interference, multipath effect, signal noise, or algorithm error during signal processing. They do not correspond to actual existing objects, and filtering is performed by setting the life cycle of the detected target, that is, setting the number of consecutive appearances and the number of consecutive losses of the target and their corresponding thresholds to determine whether the target is false. A non-dangerous target refers to a target that is detected by the radar system but does not pose a direct threat to traffic safety or does not require immediate action. For example, an object outside the road range, and such objects are filtered out by setting a distance threshold.

[0082] It can be understood that after the second target data corresponding to the radar-acquired data is obtained in this embodiment, after constructing the radar coordinate system corresponding to the second target data and constructing the pixel coordinate system corresponding to the camera-acquired data, the conversion relationship between the radar coordinate system and the pixel coordinate system is determined according to geometric and projection principles, and then the second target data is projected into the pixel coordinate system according to the conversion relationship to obtain the target region of interest corresponding to the second target data, so as to realize the spatial synchronization of the radar-acquired data and the camera-acquired data. Exemplarily, as Figure 6 shown, independent coordinate systems are established for the radar and the camera respectively. Then, the conversion relationship between the radar coordinate system and the pixel coordinate system is determined according to geometric and projection principles. Using the above conversion relationship, the second target data detected by the radar is mapped into the pixel coordinate system to realize the visual projection of the target, and further realize the spatial synchronization of the camera-acquired data and the radar-acquired data.

[0083] Specifically, in this embodiment, the position information (r, θ) of the target P detected by the radar is projected onto the position (u, v) in the pixel coordinate system through spatial synchronization, and a region of interest (ROI) is generated with the projection point as the center by using the prior knowledge of the target shape and size and the target distance detected by the radar. In formulas (1), (2), (3), and (4), appropriate rectangle width W and height H are selected, and combined with the camera focal length f, target distance r, angle θ, and pixel size (dx, dy), the width w and height h of the rectangle in the image are calculated, and then the upper left coordinates (x0, y0) of the ROI region are determined.

[0084] w = f × W × (r cos θ) -1 × dx Formula (1);

[0085] h = f × H × (r cos θ) -1 × dy Formula (2);

[0086] x0 = u - w × 0.5 Formula (3);

[0087] y0 = u - h × 0.5 Formula (4).

[0088] In the embodiment of the present application, after the time synchronization and spatial synchronization of the camera-acquired data and the radar-acquired data are completed, the first target data corresponding to the camera-acquired data is fused with the target region of interest corresponding to the radar-acquired data. Specifically, this embodiment uses the Hungarian algorithm to determine the matching pairs formed by the target region of interest and the target region corresponding to the first target data, and then calculates the intersection over union of the target region of interest and the target region corresponding to the first target data in the matching pairs. When the intersection over union is greater than the intersection over union threshold, the target fusion result is generated according to the target region of interest corresponding to the intersection over union and the target region corresponding to the first target data. Among them, the determination process of the matching pairs can be to obtain the first centroid coordinates of the target region of interest and the second centroid coordinates of the target region corresponding to the first target data; according to the first centroid coordinates and the second centroid coordinates, the Hungarian algorithm is used to perform the association operation between the target region of interest and the target region corresponding to the first target data to obtain the matching pairs.

[0089] Exemplarily, such as Figure 7As shown in the figure, first, a matrix reflecting the association cost between targets is constructed based on the target data (target region of interest and first target data) obtained from the radar and the camera; then, the Hungarian algorithm is applied to optimize the matching process to ensure the minimization of the association cost between each radar target and the camera target, thereby finding the optimal matching solution. In addition, to further improve the fineness and accuracy of the matching, the intersection over union (IOU) can be used as an evaluation metric in this embodiment. Specifically, for each matching pair found by the Hungarian algorithm, its IOU value is calculated, which is based on the ratio of the overlapping region to the union region between the ROI generated by the radar and the ROI generated by vision. The quality of the matching pair is evaluated through the IOU threshold, and when the IOU value is higher than the IOU threshold, the matching is considered successful. By combining the Hungarian algorithm and IOU, not only the spatial position relationship of the targets is considered, but also the matching degree of the target shape and size is combined, thus achieving a more comprehensive target evaluation and further obtaining a more accurate target fusion result.

[0090] In the embodiment of the present application, as Figure 8 shown, when the distance between the detected targets is determined to be lower than the safety threshold based on the target fusion result, the potential collision risk and the risk level will be evaluated. If there is a collision risk, a warning message will be immediately generated and sent to the traffic management center and the vehicle-mounted system to warn the relevant personnel and vehicles and prompt the relevant personnel and drivers to take evasive measures. In addition, this embodiment can also provide emergency command support in case of emergency, be able to command rescue vehicles and personnel to reach the accident scene in real time, provide real-time traffic condition information for the traffic management department, and assist in emergency dispatching and accident handling. Through the accident prevention method integrating radar and visual information, the accuracy of road target recognition is effectively improved, and thus the safety of road users can be improved and the probability of traffic accidents can be reduced.

[0091] Referring to Figure 9 , the embodiment of the present application provides a traffic accident warning device based on the fusion of radar and vision for object imaging. The device includes:

[0092] The first module 910 is configured to obtain the camera acquisition data and the radar acquisition data of the area to be warned according to a preset sampling period;

[0093] The second module 920 is configured to preprocess the camera acquisition data based on the object imaging algorithm to obtain the first target data;

[0094] The third module 930 is configured to process the radar acquisition data to obtain the second target data and the distance data, angle data, and speed data corresponding to the second target data;

[0095] The fourth module 940 is configured to project the second target data onto the camera acquisition data to obtain a target region of interest corresponding to the second target data;

[0096] The fifth module 950 is configured to fuse the target region of interest and the first target data to obtain a target fusion result;

[0097] The sixth module 960 is configured to perform traffic warning according to the target fusion result, distance data, angle data, and speed data.

[0098] It can be understood that the content in the above method embodiments is applicable to the device embodiments of the present application. The functions specifically implemented in the device embodiments of the present application are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those in the above method embodiments.

[0099] The embodiments of the present application further provide a computer device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the above method is implemented. The computer device can be any intelligent terminal including a tablet computer, an in-vehicle computer, etc.

[0100] It can be understood that the content in the above method embodiments is applicable to the device embodiments of the present application. The functions specifically implemented in the device embodiments of the present application are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those in the above method embodiments.

[0101] The embodiments of the present application further provide a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above method is implemented.

[0102] It can be understood that the content in the above method embodiments is applicable to the storage medium embodiments of the present application. The functions specifically implemented in the storage medium embodiments of the present application are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those in the above method embodiments.

[0103] The embodiments described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation to the technical solutions provided by the embodiments of the present application. Those skilled in the art know that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0104] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation to the embodiments of the present application, and may include more or fewer steps than shown in the figures, or combine some steps, or different steps.

[0105] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0106] Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, and their appropriate combinations.

[0107] As used in the specification of this application and the above-mentioned drawings, the terms "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0108] It should be understood that in this application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that three relationships can exist. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist at the same time. Among them, A and B can be singular or plural. The character " / " generally means that the associated objects before and after are in an "or" relationship. "At least one (one) of the following" or a similar expression means any combination of these items, including any combination of single item (one) or plural items (ones). For example, at least one (one) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0109] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the above division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical or other forms.

[0110] The units described above as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0111] In addition, in each embodiment of the present application, each functional unit can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0112] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present application. The foregoing storage medium includes: various media that can store programs such as USB flash drives, mobile hard disks, read-only memories (ROM for short), random access memories (RAM for short), magnetic disks, or optical discs.

[0113] The preferred embodiments of the embodiments of the present application have been described above with reference to the accompanying drawings, and thus do not limit the scope of the rights of the embodiments of the present application. Any modifications, equivalent replacements, and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the rights of the embodiments of the present application.

Claims

1. A traffic accident warning method based on the fusion of radar and vision for object imaging, characterized in that, The method includes the following steps: Obtain camera acquisition data and radar acquisition data of the area to be warned according to a preset sampling period; Preprocess the camera acquisition data based on an object imaging algorithm to obtain first target data; Process the radar acquisition data to obtain second target data and distance data, angle data, and speed data corresponding to the second target data; Project the second target data onto the camera acquisition data to obtain a target region of interest corresponding to the second target data; Fuse the target region of interest and the first target data to obtain a target fusion result; Conduct traffic warning according to the target fusion result, the distance data, the angle data, and the speed data.

2. The method according to claim 1, characterized in that, The preprocessing of the camera acquisition data based on the object imaging algorithm to obtain first target data includes: Input the camera acquisition data into a preset feature filter for feature extraction; Determine the corresponding target image of interest in the camera acquisition data based on the extracted features as the first target data.

3. The method according to claim 1, wherein The processing of the radar acquisition data to obtain second target data and distance data, angle data, and speed data corresponding to the second target data includes: Parse the radar acquisition data to obtain first radar point cloud data; Conduct clustering processing on the first radar point cloud data to obtain second radar point cloud data; Conduct target screening according to the second radar point cloud data to obtain the second target data and distance data, angle data, and speed data corresponding to the second target data.

4. The method according to claim 3, wherein The conducting of target screening according to the second radar point cloud data to obtain the second target data and distance data, angle data, and speed data corresponding to the second target data includes: Determine the current type of the current target according to the second radar point cloud data; Determine that the current type is a preset type, and delete the second radar point cloud data corresponding to the current target. The preset types include an empty target type, a false target type, and a non-dangerous target type; Determine the second target data and distance data, angle data, and speed data corresponding to the second target data from the remaining second radar point cloud data.

5. The method according to claim 1, wherein The projecting of the second target data onto the camera acquisition data to obtain a target region of interest corresponding to the second target data includes: Construct a radar coordinate system corresponding to the second target data; Construct a pixel coordinate system corresponding to the camera acquisition data; Determine the conversion relationship between the radar coordinate system and the pixel coordinate system according to geometric and projection principles; Project the second target data into the pixel coordinate system according to the conversion relationship to obtain a target region of interest corresponding to the second target data.

6. The method according to claim 1, characterized in that, The fusing of the target region of interest and the first target data to obtain a target fusion result includes: Use the Hungarian algorithm to determine the matching pairs formed by the target region of interest and the target regions corresponding to the first target data; Calculate the intersection over union of the target region of interest and the target regions corresponding to the first target data in the matching pairs; When the intersection over union is greater than the intersection over union threshold, generate the target fusion result according to the target region of interest corresponding to the intersection over union and the target region corresponding to the first target data.

7. The method according to claim 6, characterized in that, The step of using the Hungarian algorithm to determine the matching pairs formed by the target region of interest and the target region corresponding to the first target data includes: Obtain the first centroid coordinates of the target region of interest and the second centroid coordinates of the target region corresponding to the first target data; According to the first centroid coordinates and the second centroid coordinates, use the Hungarian algorithm to perform the association operation between the target region of interest and the target region corresponding to the first target data to obtain the matching pairs.

8. A traffic accident warning device based on the fusion of radar and vision for object imaging, characterized in that, The device includes: A first module, configured to obtain camera acquisition data and radar acquisition data of a region to be warned according to a preset sampling period; A second module, configured to preprocess the camera acquisition data based on an object imaging algorithm to obtain first target data; A third module, configured to process the radar acquisition data to obtain second target data and the distance data, angle data, and speed data corresponding to the second target data; A fourth module, configured to project the second target data onto the camera acquisition data to obtain the target region of interest corresponding to the second target data; A fifth module, configured to fuse the target region of interest and the first target data to obtain a target fusion result; A sixth module, configured to perform traffic warning according to the target fusion result, the distance data, the angle data, and the speed data.

9. A computer device, characterized in that, Comprising: At least one processor; At least one memory, configured to store at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, The computer program, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images