Traffic risk real-time alarm method and system

Abstract

The invention relates to the technical field of traffic safety, and discloses a traffic risk real-time alarm method and system. The traffic risk real-time alarm method comprises the following steps: determining a traffic event alarm area and a first traffic participant detection frame according to traffic perception information sent by a multi-source perception device in real time; determining a target traffic participant detection frame according to the first traffic participant detection frame; based on the state information of the target traffic participant detection frame at the current time, predicting the position information of the target traffic participant detection frame at the future time; and carrying out traffic risk identification and alarm according to the position information of the target traffic participant detection frame at the current time, the position information of the target traffic participant detection frame at the future time and the traffic event alarm area. Through the scheme of the invention, missed identification and wrong identification of traffic risk events are avoided, the reliability and accuracy of traffic risk identification and alarm are improved, the alarm time delay is reduced, and invalid alarm is avoided.

Description

Technical Field

[0001] This invention relates to the field of traffic safety technology, and in particular to a method and system for real-time traffic risk warning. Background Technology

[0002] Various traffic incidents seriously affect traffic safety and traffic efficiency. Installing traffic safety warning systems can effectively reduce the incidence of traffic incidents, protect the lives and property of traffic participants (including pedestrians, non-motorized vehicles, and motorized vehicles), and improve road traffic efficiency.

[0003] Traditional traffic safety alarm systems rely on information from a single sensing device to identify traffic events, which can easily lead to blind spots, missed or false identifications of traffic events, resulting in low reliability of traffic event identification. Furthermore, the data transmission and calculation in the entire alarm process introduce high alarm latency, which can cause invalid alarms when traffic participants are traveling at high speeds. Summary of the Invention

[0004] In view of this, the present invention proposes a real-time traffic risk warning method and system, which solves the problems of low reliability of traffic event identification and long warning delay in traditional traffic safety warning systems.

[0005] On one hand, embodiments of the present invention provide a real-time traffic risk warning method, which includes: Based on the traffic sensing information sent in real time by multi-source sensing devices, determine the traffic incident alarm area and the first traffic participant detection box; Based on the first traffic participant detection box, determine the target traffic participant detection box; Based on the current state information of the target traffic participant detection box, predict the location information of the target traffic participant detection box in the future. Traffic risk identification and alerts are performed based on the current location information of the target traffic participant detection box, the future location information of the target traffic participant detection box, and the traffic event alarm area.

[0006] In some implementations, determining the traffic incident alarm area and the first traffic participant detection box based on traffic sensing information sent in real time by multi-source sensing devices includes: Based on real-time traffic environment images sent by visual sensors, determine the traffic incident warning area and the first traffic participant detection box; Based on the traffic participant perception information sent by the radar sensor and / or the traffic participant perception information sent by the vehicle-mounted perception device, a first traffic participant detection box is determined within the coverage area of ​​the traffic environment image.

[0007] In some implementations, determining the target traffic participant detection box based on the first traffic participant detection box includes: Calculate the crossover ratio (CROR) of the first traffic participant detection boxes determined by any two sensing devices in the multi-source sensing devices to determine candidate boxes; Filter out the target traffic participant detection boxes from all candidate boxes.

[0008] In some implementations, the cross-union ratio (CUI) of the first traffic participant detection boxes determined by any two sensing devices in the multi-source sensing devices is calculated to determine the candidate boxes, including: In response to a cross-union ratio (CUNR) that is less than a first CUNR threshold, the first traffic participant detection box that participated in the CUNR calculation is identified as a candidate box. In response to the intersection-to-union ratio (CUI) being not less than the first CUI threshold, a traffic participant fusion box is determined based on the first traffic participant detection boxes corresponding to any two sensing devices, and the two first traffic participant detection boxes and the traffic participant fusion box that participate in the fusion are determined as candidate boxes.

[0009] In some implementations, selecting the target traffic participant detection box from all candidate boxes includes: Determine the confidence level of the candidate boxes for the target traffic participant, and then filter out the target traffic participant detection boxes from all candidate boxes.

[0010] In some implementations, the confidence level for determining a candidate bounding box as a target traffic participant includes: Based on the initial confidence of the candidate box and the recognition capability weight of the multi-source sensing device corresponding to the candidate box, the confidence of the candidate box as the target traffic participant is determined.

[0011] In some implementations, determining the confidence level of a candidate bounding box as a target traffic participant, and then filtering out the target traffic participant detection box from all candidate bounding boxes, includes: Select the candidate box with the highest confidence from all candidate boxes; Based on the intersection-union ratio of the candidate box with the highest confidence and the other candidate boxes, the target traffic participant detection box is selected from all candidate boxes.

[0012] In some implementations, predicting the location information of the target traffic participant detection box in the future time based on the state information of the target traffic participant detection box at the current time includes: predicting the location information of the target traffic participant detection box in the future time based on the LSTM algorithm and the state information of the target traffic participant detection box at the current time.

[0013] In some implementations, traffic risk identification and alerting based on the current location information of the target traffic participant detection box, the future location information of the target traffic participant detection box, and the traffic event alert area includes: Traffic risk identification and alerts are performed based on the current location information of the target traffic participant detection box, the future location information of the target traffic participant detection box, the traffic light information sent by the traffic light acquisition unit, and the traffic event alarm area.

[0014] On the other hand, embodiments of the present invention also provide a real-time traffic risk warning system, which includes a computing device comprising: At least one processor; and The memory stores a computer program that can run on a processor, which executes the traffic risk real-time warning method described above when executing the program.

[0015] The present invention has at least the following beneficial effects: This invention provides a real-time traffic risk warning method and system. Based on traffic sensing information transmitted in real time by multi-source sensing devices, it determines a traffic event warning area and a first traffic participant detection box. From the first traffic participant detection box, it filters out a target traffic participant detection box. Based on the current state information of the target traffic participant detection box, it predicts the future location information of the target traffic participant detection box. Finally, based on the current location information, the future location information, and the traffic event warning area, it performs traffic risk identification and warning. This invention, through the above technical solution, avoids missed and false identification of traffic risk events, improves the reliability and accuracy of traffic risk identification and warning, and also reduces warning latency, avoiding invalid warnings. The multi-source sensing devices include at least a visual sensor. This invention's technical solution avoids missed and false identification of traffic risk events, improves the reliability and accuracy of traffic risk identification and warning, and also reduces warning latency, avoiding invalid warnings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of a real-time traffic risk warning method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the traffic event warning area determined by the real-time traffic risk warning method provided in this embodiment of the invention; Figure 3 This is a flowchart of a method for determining a target traffic participant detection box in a real-time traffic risk warning method provided in an embodiment of the present invention; Figure 4 A flowchart illustrating another method for determining the target traffic participant detection box in the real-time traffic risk warning method provided in this embodiment of the invention; Figure 5 A flowchart of another method for determining the target traffic participant detection box in the real-time traffic risk warning method provided in the embodiments of the present invention; Figure 6 A flowchart of another real-time traffic risk warning method provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of a real-time traffic risk warning system provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of another traffic risk real-time warning system provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.

[0019] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two entities or parameters with the same name but different names. It is clear that "first" and "second" are only for the convenience of expression and should not be construed as limiting the embodiments of the present invention. Subsequent embodiments will not explain this in detail.

[0020] Various traffic incidents seriously affect traffic safety and traffic efficiency. Installing traffic safety warning systems can effectively reduce the incidence of traffic incidents, protect the lives and property of traffic participants (including pedestrians, non-motorized vehicles, and motorized vehicles), and improve road traffic efficiency.

[0021] When relevant traffic safety warning systems issue traffic safety warnings, there is a problem of low reliability in traffic event identification. The inventors analyzed the reasons for the low reliability of traffic event identification as follows: On the one hand, since event perception relies on a single roadside information source, such as a roadside camera or millimeter-wave radar, it is easy to generate blind spots, thus causing missed identification of traffic events; on the other hand, under adverse working conditions, the perception accuracy of a single sensing device will decrease significantly, resulting in missed or false identification of events; furthermore, the data transmission and calculation in the entire alarm process bring high alarm latency. When traffic participants are moving at high speeds, the high latency will cause the device to issue invalid alarms (for example, when a non-motorized vehicle is identified as running a red light, the non-motorized vehicle may have already reached the other side by the time an alarm is issued, resulting in an invalid alarm).

[0022] In view of the above, in order to solve the above technical problems, the present invention will be described in detail below with reference to embodiments and accompanying drawings.

[0023] The first aspect of this invention provides a real-time traffic risk warning method, such as... Figure 1 As shown, the method specifically includes steps S10 to S40.

[0024] S10. Based on the traffic perception information sent in real time by the multi-source perception devices, determine the traffic incident alarm area and the first traffic participant detection box.

[0025] In this embodiment of the invention, the multi-source sensing device includes at least a visual sensor (e.g., a camera) from a roadside sensing device. The visual sensor can acquire traffic environment images, and based on these images acquired in real time, a traffic event warning area and a first traffic participant detection box can be determined.

[0026] In some examples, multi-source sensing devices may include not only visual sensors in roadside sensing devices, but also traffic light acquisition units in roadside sensing devices. Traffic light data can be acquired through the traffic light acquisition unit.

[0027] In some examples, multi-source sensing devices may include not only the traffic light acquisition unit and visual sensors in roadside sensing devices, but also radar sensors (such as lidar, millimeter-wave radar, etc.) in roadside sensing devices. Radar sensors can perceive information about traffic participants in the traffic environment. Traffic participants can include one or more of pedestrians, motor vehicles, and non-motorized vehicles.

[0028] In some examples, multi-source sensing devices may include not only visual sensors in roadside sensing devices but also vehicle-side sensing devices. Vehicle-side sensing devices may include one or more of the following: vehicle GPS (Global Positioning System) and vehicle status sensors such as throttle opening, brake pedal opening, and steering wheel angle. The vehicle's location information can be sensed via GPS, while vehicle status information, such as throttle opening, brake pedal opening, and steering wheel angle, can be sensed via vehicle status sensors.

[0029] It should be noted that the visual sensors, radar sensors, traffic light collectors, and vehicle GPS devices included in multi-source sensing devices can all be referred to as sensing devices. In some examples, multi-source sensing devices may include not only different types of sensing devices but also sensing devices of the same type. For example, multi-source sensing devices may include visual sensor vs1, visual sensor vs2, ..., visual sensor vsn, radar sensor rs1, radar sensor rs2, ..., radar sensor rn, etc.

[0030] In some examples, traffic perception information may include traffic light data collected by the traffic light acquisition unit and traffic environment images collected by the visual sensor, as well as traffic participant information perceived by the radar sensor, and vehicle location and status information perceived by the vehicle perception device. Traffic participant information may include the location information of traffic participants; therefore, corresponding detection boxes can be generated based on the location information and preset dimensions of the traffic participants. Vehicles, as traffic participants, can also have corresponding detection boxes generated based on their location information and preset vehicle dimensions. The first traffic participant detection box includes, but is not limited to, detection boxes generated based on the location information of traffic participants and detection boxes generated based on the location information of vehicles.

[0031] Traffic incident warning areas are located on traffic environment images. A traffic incident can be understood as an abnormal behavior that occurs at a specific time and in a specific space. Therefore, it is necessary to mark the incident warning area within the perception range of the camera.

[0032] In this embodiment of the invention, at least based on several traffic perception information sent in real time by several visual sensors in a multi-source sensing device, the traffic event alarm area under each frame of traffic environment image and the first traffic participant detection box within the coverage area of ​​that frame of traffic environment image can be determined.

[0033] S20. Determine the target traffic participant detection box based on the first traffic participant detection box.

[0034] The same traffic participant can be detected by one or more sensing devices in a multi-source sensing system. Therefore, in this embodiment of the invention, several first traffic participant detection boxes determined by traffic sensing information sent in real time by several sensing devices can be fused and filtered to remove duplicate first traffic participant detection boxes and incorrectly identified first traffic participant detection boxes, thereby obtaining target traffic participant detection boxes. Each target traffic participant detection box can represent a real traffic participant. Through the above scheme, this embodiment of the invention can accurately identify real traffic participants, improve the accuracy and recall rate of traffic participant identification, and lay the foundation for subsequent accurate traffic risk identification.

[0035] In this embodiment of the invention, after the target traffic participant is identified, the traffic participant can be efficiently tracked based on the SORT (SimpleOnline and Realtime Tracking) algorithm. This associates the target participant detection box determined in the current frame with the target participant detection box determined in the previous frame, thereby updating the status of each traffic participant (including newly appearing traffic participants, existing traffic participants, and traffic participants that have left the area) in real time and assigning a unique ID, thus achieving continuous tracking across frames.

[0036] S30. Based on the state information of the target traffic participant detection box at the current time, predict the location information of the target traffic participant detection box at a future time.

[0037] In this embodiment of the invention, the position information of the target traffic participant detection box in the future time can be predicted based on the Long Short-Term Memory (LSTM) network algorithm and the state information of the target traffic participant detection box at the current time.

[0038] The state information of the target traffic participant detection box at the current time may include at least the target traffic participant detection box's position, speed, and acceleration at the current time.

[0039] The future time can be set based on the actual scenario. For example, it can be in 10ms increments to predict the location information of the target traffic participant detection box in the next 10ms, 20ms, 30ms, 40ms and 50ms; it can be in 100ms increments to predict the location information of the target traffic participant detection box in the next 100ms, 200ms, 300ms, 400ms and 500ms; it can be in 50ms increments to predict the location information of the target traffic participant detection box in the next 50ms, 100ms, 150ms, 200ms and 250ms; or it can be in 1s increments to predict the location information of the target traffic participant detection box in the next 1s, 2s, 3s, 4s and 5s.

[0040] S40. Based on the location information of the target traffic participant detection box at the current time, the location information of the target traffic participant detection box at the future time, and the traffic event alarm area, traffic risk identification and alarm are performed.

[0041] In this embodiment of the invention, risk identification can be performed by combining the target traffic participant's current location information and future location information to determine whether they appear within the traffic incident warning area. In some examples, if the target traffic participant's current location information is within the traffic incident warning area, but their future location information is no longer within the warning area, it indicates that a dangerous behavior has occurred, i.e., a traffic risk has occurred, but there is no current traffic risk, therefore no warning is needed. In some examples, if the target traffic participant's current location information is within the traffic incident warning area, and their future location information is also within the warning area, it indicates that a dangerous behavior exists, and a warning is required. In some examples, if the target traffic participant's current location information is not within the warning area, but their future location information is within the warning area, then a warning is required.

[0042] In this embodiment of the invention, since each target traffic participant detection box can represent a real traffic participant, the accuracy of the location information of the target traffic participant detection box at the current time is ensured. Therefore, by combining the location information of the target traffic participant detection box at the current time with the location information of the target traffic participant at a future time to determine whether it appears in the traffic event alarm area, risk identification and alarm can be performed, which can greatly improve the accuracy of alarms, reduce alarm latency, and avoid the occurrence of invalid alarms.

[0043] This invention, through embodiments thereof, can determine a traffic event alarm region and a first traffic participant detection box based on traffic sensing information transmitted in real time by multi-source sensing devices. From the first traffic participant detection box, a target traffic participant detection box is selected. Based on the current state information of the target traffic participant detection box, its future location information is predicted. Finally, based on the current location information, the future location information, and the traffic event alarm region, traffic risk identification and alarm are performed. This invention, through the above technical solution, avoids missed and false identification of traffic risk events, improves the reliability and accuracy of traffic risk identification and alarm, reduces alarm latency, and avoids invalid alarms.

[0044] In some embodiments of the present invention, such as Figure 1 The S10 shown (determining the traffic event warning area and the first traffic participant detection box based on the traffic perception information sent in real time by the multi-source perception device) may include: determining the traffic event warning area and the first traffic participant detection box based on the traffic environment image sent in real time by the visual sensor; determining the first traffic participant detection box located within the coverage area of ​​the traffic environment image based on the traffic participant perception information sent by the radar sensor and / or the traffic participant perception information sent by the vehicle-side perception device.

[0045] Specifically, since traffic incidents can be understood as abnormal behaviors occurring at a specific time and in a specific space, traffic environment images are calibrated to determine traffic incident warning areas. There are various calibration methods. For example, traffic environment images can be calibrated using object detection models (e.g., YOLOv5, YOLOv8, or YOLOv10) to obtain traffic incident warning areas. Alternatively, traffic incident warning areas can be manually drawn within the traffic environment image. Figure 2 The image shown is a schematic diagram illustrating the traffic incident warning area determined by calibrating a traffic environment image. Figure 2 The colored boxes and the roughly cross-shaped blue lines in the image represent traffic incident warning areas.

[0046] After identifying the traffic incident warning area, the first traffic participant detection box located within the coverage area of ​​the traffic environment image can be determined based on traffic participant perception information sent by radar sensors and / or vehicle-mounted perception devices. This enables the association between the traffic incident warning area and traffic participant perception information at the same time.

[0047] In some embodiments of the present invention, when determining the first traffic participant detection box within the coverage area of ​​the traffic environment image based on the traffic participant perception information sent by the radar sensor and / or the traffic participant perception information sent by the vehicle-mounted perception device, the traffic participant perception information sent by several radar sensors and / or the traffic participant perception information sent by several vehicle-mounted perception devices can be converted into traffic participant perception information in a 2D image coordinate system. Based on the traffic participant perception information in all 2D image coordinate systems, the first traffic participant detection box within the coverage area of ​​each frame of the traffic environment image is determined.

[0048] It should be noted that, in this embodiment of the invention, the 2D image coordinate system is the same as the coordinate system of the traffic environment image. The traffic participant information sensed by the vehicle sensing device refers to the vehicle's own information.

[0049] Since the traffic participant perception information sensed by radar sensors (such as lidar and millimeter-wave radar) is in the sensor coordinate system, while the traffic participant perception information sensed by vehicle-mounted sensing devices is in the vehicle sensor coordinate system or the world coordinate system (for example, vehicle-mounted GPS senses the location information of traffic participants in the world coordinate system, while vehicle status sensors sense the vehicle status information in the sensor coordinate system), information from different coordinate systems can be converted to the same coordinate system to ensure the accuracy of subsequent risk identification. Because traffic environment images contain map data, the traffic participant perception information sensed by all sensors can be converted into traffic participant perception information in a 2D image coordinate system.

[0050] In this embodiment of the invention, the traffic participant perception information may include the traffic participant location information. The following describes the coordinate system transformation process of the traffic participant location information, taking the traffic participant location information, the radar sensor as LiDAR, and the vehicle-side perception device as GPS as an example.

[0051] Since the location information of traffic participants sensed by LiDAR is in the LiDAR coordinate system, while the location information of the vehicle sensed by the vehicle-mounted GPS is in the world coordinate system, it is necessary to first convert the location information of traffic participants sensed by LiDAR from the LiDAR coordinate system to the world coordinate system, and then from the world coordinate system to the 2D image coordinate system. Similarly, it is necessary to convert the vehicle location information sensed by the vehicle-mounted GPS from the world coordinate system to the 2D image coordinate system.

[0052] Specifically, the location information of traffic participants in the sensor coordinate system (referring to the lidar coordinate system in this example) can be converted into the location information of traffic participants in the world coordinate system according to formula (1). Based on formula (2), the location information of traffic participants in the world coordinate system can be converted into the location information of traffic participants in the 2D image coordinate system.

[0053] (1) Where (Xw, Yw, Zw) are world coordinates, (Xc, Yc, Zc) are sensor coordinates, and Hw is the calibration transformation matrix.

[0054] in, .

[0055] in, , For the sensor's extrinsic parameters, This is a rotation matrix, representing the sensor mounting angle. This is a translation vector, representing the offset of the installation position.

[0056] (2) in,

[0057] in, For the rotation matrix of the roadside vision sensor, Let u and v be the translation vector of the roadside vision sensor, and u and v be the x and y coordinates of the transformed image coordinate system. When u and v simultaneously satisfy... The data is then retained, where, l , w These represent the length and width pixel counts of the traffic environment image, respectively.

[0058] In this embodiment of the invention, by establishing a left-hand transformation relationship, the traffic participant perception information sent by the radar sensor and the vehicle-mounted perception device can be transformed into the coordinate system of the traffic environment image. This allows for the precise selection of traffic participant detection boxes within the coverage area of ​​the traffic environment image; these selected detection boxes constitute the first traffic participant detection box. This lays the foundation for subsequent precise data fusion.

[0059] In some embodiments of the present invention, step S20 (determining the target traffic participant detection box based on the first traffic participant detection box) may include: Reference Figure 1 and Figure 3Step S20 (determining the target traffic participant detection box based on the first traffic participant detection box) in the real-time traffic risk warning method provided in this embodiment of the invention may include S201 and S202.

[0060] S201. Calculate the crossover ratio (CROR) of the first traffic participant detection boxes determined by any two sensing devices in the multi-source sensing devices to determine the candidate boxes.

[0061] S202. Filter out the target traffic participant detection box from all candidate boxes.

[0062] As mentioned above, in this embodiment of the invention, the multi-source sensing device may include not only sensing devices of different types, but also sensing devices of the same type. For example, in some examples, the multi-source sensing device may include a visual sensor vs1, a radar sensor rs1, and a signal collector sc1. For example, in still other examples, the multi-source sensing device may include visual sensors vs1, vs2, ..., visual sensors vsn, and radar sensors rs1, rs2, ..., rn. Based on this, in step S201, any two sensing devices in the multi-source sensing device can be combined pairwise to obtain several sensing device groups. The crossover ratio (CROR) of the first traffic participant detection boxes corresponding to the two sensing devices in each sensing device group is calculated to determine the candidate boxes.

[0063] The calculation process for the intersection-union ratio (IUGR) can be determined based on the actual scenario. In some scenarios, where each sensing device in the sensing device group determines only one first traffic participant detection box, the IUGR of the first traffic participant detection boxes determined by the two sensing devices can be calculated separately. In some scenarios, where one sensing device in the sensing device group determines one first traffic participant detection box, and another sensing device determines multiple first traffic participant detection boxes, the IUGR of the first traffic participant detection box determined by one sensing device can be calculated separately with each first traffic participant detection box determined by the other sensing device, resulting in multiple IUGRs. In some scenarios, where both sensing devices in the sensing device group determine multiple first traffic participant detection boxes, all first traffic participant detection boxes determined by one sensing device are iterated through sequentially, and the IUGR of each of the traversed first traffic participant detection boxes is calculated separately with each of the first traffic participant detection boxes determined by the other sensing device, resulting in several IUGRs.

[0064] It should be noted that in the above application scenarios, the two sensing devices in the sensing device group can be different types of sensing devices or the same type of sensing devices, without specific limitations.

[0065] In this embodiment of the invention, candidate frames are determined by calculating the intersection-union ratio of the first traffic participant detection frames corresponding to any two sensing devices in the sensing device group. This achieves comprehensive detection coverage of traffic participant detection frames during traffic risk identification and early warning, avoids missing traffic events, and further improves the recall rate, accuracy, and reliability of traffic risk identification and early warning.

[0066] In step S202, based on the confidence level of the candidate boxes as target participant detection boxes, target traffic participant detection boxes with higher confidence levels can be selected from all candidate boxes. This further improves the accuracy of the target traffic participant detection boxes as real traffic participants, and further improves the accuracy and reliability of traffic risk identification and alarm.

[0067] In this embodiment of the invention, by combining any two sensing devices from the multi-source sensing devices in pairs to obtain several sensing device groups, and calculating the intersection-union ratio (IUR) of the first traffic participant detection boxes corresponding to the two sensing devices in each sensing device group, candidate boxes can be determined based on the calculated IUR. Based on the confidence of the candidate boxes as target participant detection boxes, target traffic participant detection boxes with higher confidence are selected from all candidate boxes. Thus, comprehensive detection coverage of traffic participant detection boxes is achieved in the process of traffic risk identification and early warning, avoiding missed identification events, further improving the accuracy of target traffic participant detection boxes as real traffic participants, and further improving the recall rate, accuracy and reliability of traffic risk identification and early warning.

[0068] In some embodiments of the present invention, the step S201 of calculating the cross-union ratio (CUP) of the first traffic participant detection boxes determined by any two sensing devices in the multi-source sensing devices to determine candidate boxes may include: in response to the CUP being less than a first CUP threshold, determining the first traffic participant detection boxes involved in the CUP calculation as candidate boxes; in response to the CUP being not less than the first CUP threshold, determining a traffic participant fusion box based on the first traffic participant detection boxes corresponding to any two sensing devices, and determining the two first traffic participant detection boxes involved in the fusion and the traffic participant fusion box as candidate boxes.

[0069] like Figure 4 As shown, candidate boxes can be determined through steps S2011 to S2014.

[0070] S2011. Calculate the intersection-union ratio of the detection frames of the first traffic participants corresponding to the two sensing devices in the sensing device group.

[0071] S2012. Determine whether the crossover-union ratio is less than the first crossover-union ratio threshold.

[0072] If the crossover-union ratio is less than the first crossover-union ratio threshold, then proceed to step S2014; if the crossover-union ratio is not less than the first crossover-union ratio threshold, then proceed to step S2013.

[0073] S2013. Based on the first traffic participant detection boxes corresponding to the two sensing devices respectively, determine the traffic participant fusion box, and determine the two first traffic participant detection boxes and the traffic participant fusion box that participate in the fusion as candidate boxes.

[0074] S2014. The first traffic participant detection box that participates in the intersection-union calculation is identified as a candidate box.

[0075] In step S2011, the crossover ratio between the two detection boxes can be calculated using formula (3).

[0076] (3) Where Bi and Bj represent the first traffic participant detection boxes determined by different multi-source sensing devices.

[0077] In step S2012, the first crossover-union ratio threshold can be set according to actual needs. For example, the first crossover-union ratio threshold can be set to 0.4, 0.5, 0.6 or 0.7, etc., but is not limited to this, and the first crossover-union ratio threshold can also be set to other values.

[0078] Compare the cross-union ratio (CUNR) with the first CUNR threshold. If the CUNR is not less than the first CUNR threshold, it means that the two detection boxes involved in the CUNR calculation are duplicate targets. Then, proceed to step S2013 to determine the traffic participant fusion box based on the first traffic participant detection boxes corresponding to any two sensing devices. The two first traffic participant detection boxes and the traffic participant fusion box involved in the fusion are determined as candidate boxes.

[0079] If the crossover ratio (CRR) is less than the first CRR threshold, it means that the two detection boxes involved in the CRR calculation are not the same target. Then, step S2014 is executed to determine the two first traffic participant detection boxes involved in the CRR calculation as candidate boxes.

[0080] In step S2013, the information of the traffic participant fusion box can be determined by the first traffic participant detection boxes involved in the fusion. For example, the average of the center point coordinates of the two first traffic participant detection boxes involved in the fusion can be determined as the center point coordinates of the traffic participant fusion box, the average of the lengths of the two first traffic participant detection boxes involved in the fusion can be determined as the length of the traffic participant fusion box, and the average of the heights of the two first traffic participant detection boxes involved in the fusion can be determined as the height of the traffic participant fusion box.

[0081] In this embodiment of the invention, based on the first traffic participant detection boxes corresponding to any two of the multi-source sensing devices, the cross-union ratio (CUI) can be determined, and the CUI is compared with a first CUI threshold. If the CUI is less than the first CUI threshold, the two first traffic participant detection boxes involved in the CUI calculation are determined as candidate boxes. If the CUI is not less than the first CUI threshold, a traffic participant fusion box is determined based on the first traffic participant detection boxes corresponding to any two sensing devices, and the two first traffic participant detection boxes involved in the fusion and the traffic participant fusion box are determined as candidate boxes. This achieves comprehensive detection coverage of traffic participant detection boxes during traffic risk identification and early warning, avoids missed identification events, further improves the accuracy of target traffic participant detection boxes as real traffic participants, and further improves the recall rate, accuracy, and reliability of traffic risk identification and warning.

[0082] In this embodiment of the invention, step S202, which involves filtering out the target traffic participant detection box from all candidate boxes, may include: determining the confidence level of a candidate box as a target traffic participant, so as to filter out the target traffic participant detection box from all candidate boxes.

[0083] More specifically, such as Figure 4 As shown, the target traffic participant detection box can be determined through steps S2021 and S2022.

[0084] S2021. Based on the initial confidence of the candidate box and the recognition capability weight of the multi-source sensing device corresponding to the candidate box, determine the confidence of the candidate box as the target traffic participant.

[0085] S2022. Based on the confidence level of the candidate boxes as target traffic participants, determine the target traffic participant detection box from all candidate boxes.

[0086] In step S2021, for a first traffic participant detection box in a candidate box, the confidence level of the first traffic participant detection box as a target traffic participant can be determined based on its initial confidence level and the recognition capability weight of the multi-source sensing device corresponding to the first traffic participant detection box. For a first traffic participant detection box participating in fusion in a candidate box, the confidence level of the first traffic participant detection box participating in fusion as a target traffic participant can be determined based on its initial confidence level, the area ratio of the first traffic participant detection box participating in fusion to the corresponding traffic participant fusion box, and the recognition capability weight of the multi-source sensing device corresponding to the first traffic participant detection box participating in fusion. For a traffic participant fusion box determined in a candidate box, the confidence level of the fusion box as a target traffic participant can be determined based on the confidence levels of all first traffic participant detection boxes participating in fusion as target traffic participants.

[0087] In this embodiment of the invention, the confidence level of a candidate box as a target traffic participant is determined by the initial confidence level of the candidate box and the recognition capability weight of the multi-source sensing device corresponding to the candidate box. This can improve the accuracy of the determined candidate box confidence level, thereby further improving the accuracy of traffic risk identification and alarm in the process of traffic risk identification and alarm.

[0088] The following example illustrates the process of determining the confidence level of a candidate bounding box for a target traffic participant. It should be understood that the following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0089] For the first traffic participant detection box in the candidate box, the detection box is obtained directly by a single multi-source sensing device and does not participate in the fusion. Therefore, the confidence of the first traffic participant detection box as the target traffic participant can be calculated based on formula (4).

[0090] (4) in, The confidence level of the traffic participant detection box is used as the target traffic participant. The initial confidence score is the detection confidence score of the traffic participant detection box. This initial confidence score is the algorithm detection confidence score, which can be given based on the target detection algorithm deployed on the multi-source sensing device (such as YOLOv5, YOLOv8, YOLOv10, YOLOv12, etc.). This represents the recognition capability weight of multi-source sensing devices, i.e., the ability of sensors to detect objects.

[0091] The recognition capability weight of the multi-source sensing device can be determined based on the operating conditions (such as weather, pedestrian flow, and vehicle flow) that different types of sensing devices are adapted to. If a sensing device has a better recognition capability under a certain operating condition, its recognition capability weight under that condition can be set to a larger value, such as greater than or equal to 0.5. If a sensing device's recognition capability deteriorates under a certain operating condition, its recognition capability weight under that condition can be set to a smaller value, such as less than 0.5.

[0092] For the first traffic participant detection box participating in the fusion in the candidate box, the confidence level of the traffic participant detection box participating in the fusion as the target traffic participant can be calculated based on Formula 5.

[0093] For the traffic participant fusion box in the candidate box, the confidence of all traffic participant detection boxes participating in the fusion as the target traffic participant can be calculated based on Formula 5, and the confidence of the traffic participant fusion box as the target traffic participant can be calculated based on Formula 6.

[0094] (5) (6) Where P(Object) represents the fusion bounding box of the traffic participant, and N represents the number of multi-source sensing devices participating in the fusion. The confidence level of the detection frames for traffic participants participating in the fusion. The initial confidence score is the detection confidence score of the traffic participants participating in the fusion. This initial confidence score is the algorithm detection confidence score, which can be given based on the target detection algorithm (such as YOLOv5, YOLOv8, YOLOv10, YOLOv12, etc.) deployed on the multi-source sensing device. This represents the ratio of the area of ​​the traffic participant detection box (det) participating in the fusion to the area of ​​the fused traffic participant box (object) obtained through the fusion process. Weights for the recognition capabilities of multi-source sensing devices.

[0095] In step S2022, based on the confidence level of the candidate boxes as target traffic participants, targets that are repeatedly detected by the multi-source sensing devices and targets that are misidentified by the multi-source sensing devices can be removed from several candidate boxes, thereby filtering out the real targets, i.e. the target traffic participant detection boxes, which can improve the accuracy of subsequent risk identification and alarm.

[0096] In this embodiment of the invention, the confidence level of a candidate box as a target traffic participant is determined based on the initial confidence level of the candidate box and the recognition capability weight of the multi-source sensing device corresponding to the candidate box. Based on the confidence level of the candidate box as a target traffic participant, a target traffic participant detection box is determined. This can improve the accuracy of the determined candidate box confidence levels, filter out genuine traffic participants, and thus further improve the accuracy of traffic risk identification and alerting.

[0097] In some embodiments of the present invention, determining the confidence level of a candidate box as a target traffic participant, so as to filter out the target traffic participant detection box from all candidate boxes, may include: taking out the candidate box with the highest confidence level from all candidate boxes; and filtering out the target traffic participant detection box from all candidate boxes based on the intersection-union ratio of the candidate box with the highest confidence level and the remaining candidate boxes.

[0098] In some embodiments of the present invention, such as Figure 5 As shown, a specific method for filtering out the target traffic participant detection box from all candidate boxes includes steps S501 to S504.

[0099] S501. Extract the candidate box with the highest confidence from all candidate boxes.

[0100] S502. Calculate the intersection-union ratio (IUR) of the candidate box with the highest confidence score with all remaining candidate boxes, and delete candidate boxes whose IUR is greater than the second IUR threshold.

[0101] S503, Determine if there are any remaining candidate boxes.

[0102] If yes, return to step S501 to continue selecting the candidate box with the highest confidence from all remaining candidate boxes. If no, proceed to step S504.

[0103] S504. Determine all candidate boxes with the highest confidence as target participant detection boxes.

[0104] In this embodiment of the invention, all candidate boxes are sorted in descending order of confidence to obtain a first detection box set D, and a second detection box set R is initialized. The candidate box B with the highest confidence is taken from the first detection box set D and added to the second detection box set R. All remaining candidate boxes bi in the first detection box set are traversed, and the intersection-union ratio (IoU) (B, bi_n) between the traversed candidate box bi_n and the candidate box with the highest confidence is calculated. Candidate boxes bi_n with an IoU higher than a second IoU threshold are removed from the first detection box set D. It is determined whether there are still candidate boxes in the first detection box set D. If there are still candidate boxes in the first detection box set D, the process of taking the candidate box B with the highest confidence from the first detection box set D and adding it to the second detection box set R is repeated until the first detection box set D is empty, resulting in the second detection box set R. All candidate boxes in the second detection box set R are identified as target traffic participant detection boxes.

[0105] Through the embodiments of the present invention, duplicate and incorrectly identified first traffic participant detection boxes can be removed, ensuring that the remaining first traffic participant detection boxes are all true traffic participant detection boxes. This avoids missed and false identification of traffic participants, improves the accuracy and recall rate of traffic participant identification, and thus improves the accuracy and reliability of traffic risk identification and alarm.

[0106] In some embodiments of the present invention, such as Figure 6 As shown, the real-time traffic risk warning method provided in this embodiment of the invention may include steps S600 to S630.

[0107] S600: Based on the traffic perception information sent in real time by the multi-source sensing devices, determine the traffic incident alarm area and the first traffic participant detection box.

[0108] S610. Determine the target traffic participant detection box based on the first traffic participant detection box.

[0109] S620: Based on the LSTM algorithm, predict the location information of the target traffic participant detection box in the future time according to the state information of the target traffic participant detection box at the current time.

[0110] S630. Based on the location information of the target traffic participant detection box at the current time, the location information of the target traffic participant detection box at a future time, and the traffic event alarm area, traffic risk identification and alarm are performed.

[0111] In the present invention as Figure 6 In the illustrated embodiment, the specific implementation methods of steps S600 and S610 are as follows: Figure 1Steps S10 and S20 in the illustrated embodiment are the same; therefore, the specific implementation of steps S600 and S610 will not be described in detail here.

[0112] In step S620, the position information of the target traffic participant detection box at a future time can be predicted based on the Long Short-Term Memory (LSTM) algorithm and the current state information of the target traffic participant detection box. In some examples, the current state information of the target traffic participant detection box may include its position, velocity, and acceleration at the current time. In some examples, if the target traffic participant detection box is a vehicle detection box, its current state information may include not only its position, velocity, and acceleration at the current time, but also a vehicle state vector uploaded by the vehicle. This vehicle state vector may contain vehicle state data uploaded by the vehicle, such as throttle opening, steering wheel angle, and brake pedal opening.

[0113] The future time can be set based on the actual scenario. For example, it can be predicted in 10ms increments for the target traffic participant detection box in the next 10ms, 20ms, 30ms, 40ms and 50ms; in 100ms increments for the target traffic participant detection box in the next 100ms, 200ms, 300ms, 400ms and 500ms; in 50ms increments for the target traffic participant detection box in the next 50ms, 100ms, 150ms, 200ms and 250ms; or in 1s increments for the target traffic participant detection box in the next 1s, 2s, 3s, 4s and 5s.

[0114] In this embodiment of the invention, roadside data is used to provide a global perspective, while vehicle-side data is used to provide a refined individual perspective. By using the location and operational status information of the target traffic participants at the current time, traffic risks can be identified more accurately, further improving the accuracy of traffic risk identification and warning.

[0115] The following specific examples illustrate the process of predicting the location information of the target traffic participant detection box at future time. It should be understood that the following examples are only used to explain and illustrate the present invention, and are not intended to limit the present invention.

[0116] Example 1 Based on the LSTM algorithm shown in Formulas 7 and 8, and combined with the state information of the target traffic participant detection box at the current time, the position information of the target traffic participant detection box at the future time is predicted.

[0117] (7) (8) in, .

[0118] Among them, h t h is the model's hidden state variable at the current time t. t-1 Let h be the model's hidden state variable at time t-1. t+τ-1 for The model's hidden state variables at time t. x represents the location information of the target traffic participant detection box at a future time. t This represents the state information of the target traffic participant detection box at the current time t. This indicates the position of the target traffic participant detection box at the current time t. This represents the velocity of the target traffic participant detection box at the current time t. This represents the acceleration of the target traffic participant detection box at the current time t.

[0119] In this embodiment of the invention, the target traffic participant detection box is predicted in the future based on the state information of the target traffic participant detection box at the current time using the LSTM algorithm. The risk identification and alarm are then performed by combining the target traffic participant's current location information and future location information with the traffic event alarm area. This can greatly reduce alarm latency and avoid invalid alarms.

[0120] Example 2 Based on the LSTM algorithm shown in Formulas 9 and 10, and combining the location information and running status information of the target traffic participant detection box at the current time, the location information of the target traffic participant detection box at the future time is predicted.

[0121] (9) (10) in, .

[0122] in, Let be the model's hidden state variables at the current time t. Let be the model's hidden state variables at time t-1. for The model's hidden state variables at time t. This indicates the location information of the target traffic participant detection box at a future time. This represents the state information of the target traffic participant detection box at the current time t. This indicates the position of the target traffic participant detection box at the current time t. This represents the velocity of the target traffic participant detection box at the current time t. This represents the acceleration of the target traffic participant detection box at the current time t. This represents the vehicle state vector.

[0123] In this embodiment of the invention, by using the LSTM algorithm, the position information of the target traffic participant detection box in the future time is predicted based on the state information of the target traffic participant detection box at the current time. Based on the position information and running status information of the target traffic participant at the current time, traffic risk identification can be performed more accurately, thereby further improving the accuracy of traffic risk identification and alarm.

[0124] In step S630, risk identification can be performed by combining the target traffic participant's current location information and its future location information to determine if it appears within the traffic incident warning area. For example, if the target traffic participant's current location information is within the traffic incident warning area, but its future location information is no longer within the warning area, it indicates that a dangerous behavior has occurred, i.e., a traffic risk has occurred, but there is no current traffic risk, therefore no warning is needed. Conversely, if the target traffic participant's current location information is within the traffic incident warning area, and its future location information is also within the warning area, it indicates that a dangerous behavior exists, and a warning is required. Similarly, if the target traffic participant's current location information is not within the warning area, but its future location information is, it indicates that a dangerous behavior exists, and a warning is required. Through this embodiment of the invention, by combining the target traffic participant's current location information and its future location information to determine if it appears within the traffic incident warning area for risk identification and warning, warning latency can be reduced.

[0125] In this embodiment of the invention, based on traffic sensing information transmitted in real time by multi-source sensing devices, a traffic event alarm area and a first traffic participant detection box are determined; based on the first traffic participant detection box, a target traffic participant detection box is determined; based on the LSTM algorithm, the position information of the target traffic participant detection box in the future time is predicted according to the current state information of the target traffic participant detection box; traffic risk identification and alarm are performed based on the current position information of the target traffic participant detection box, the future position information of the target traffic participant detection box, and the traffic event alarm area. This can avoid missed and false identification of traffic risk events, improve the reliability and accuracy of traffic risk identification and alarm, reduce alarm latency, and avoid invalid alarms.

[0126] In some embodiments of the present invention, step S630 (identifying and alerting traffic risks based on the location information of the target traffic participant detection box at the current time, the location information of the target traffic participant detection box at a future time, and the traffic event alarm area) in the real-time traffic risk alarm method provided by the present invention may include: identifying and alerting traffic risks based on the location information of the target traffic participant detection box at the current time, the location information of the target traffic participant detection box at a future time, the traffic light information sent by the traffic light acquisition unit, and the traffic event alarm area.

[0127] Traffic risk identification can be divided into two categories: behavioral risk identification and status-based risk identification. Behavioral risk identification includes pedestrians running red lights and warnings for large trucks turning right, which can be based on real-time and predicted trajectory data. Traffic risk identification is performed using traffic light data and calibrated traffic incident warning areas, along with real-time and predicted trajectory data. When traffic light data and calibrated traffic incident warning areas meet the preset first warning condition, an alarm is issued promptly. Status-based risks mainly include non-motorized vehicles not wearing helmets and motorized vehicles not wearing seat belts. These can be identified using image recognition algorithms (such as YOLOv12) when they appear in the traffic incident warning area. When the identification result triggers the preset second warning condition, an alarm is issued promptly.

[0128] In some examples, when issuing traffic risk alerts, different priorities can be configured for different traffic events, and alerts can be issued according to the priority. For example, high-priority risks will be alerted first.

[0129] In some examples, when issuing traffic risk warnings, the warning information can be sent to warning devices, which can be one or more of the following: LED lights, roadside displays, vehicle-mounted displays, sound columns, illuminated road studs, etc. The warning information can be sent to traffic participants through sound, light, electricity and other means.

[0130] In this embodiment of the invention, traffic risk identification and alarm are performed based on the current location information of the target traffic participant detection box, the future location information of the target traffic participant detection box, the traffic light information sent by the traffic light acquisition unit, and the traffic event alarm area. This can improve the coverage of traffic risk identification and early warning, avoid missed or false identification of traffic risk events, improve the reliability and accuracy of traffic risk identification and alarm, and reduce alarm latency to avoid invalid alarms.

[0131] Based on the same inventive concept, according to another aspect of the present invention, embodiments of the present invention also provide a real-time traffic risk warning system. For example... Figure 7 As shown, the real-time traffic risk warning system 1 includes a computing device 10, which includes a processor and a memory. The memory stores a computer program that can run on the processor. When the processor executes the program, it performs the steps of the method described above.

[0132] The memory, as a non-volatile storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods described in the embodiments of this application. The processor executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in the memory, thereby implementing the methods described in the above embodiments.

[0133] The memory may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the device, etc. Furthermore, the memory may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the local module via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0134] The computing device of this invention determines a traffic event warning area and a first traffic participant detection box based on traffic perception information sent in real time by multi-source sensing devices. Based on several first traffic participant detection boxes determined by the multi-source sensing devices, a target traffic participant detection box is fused and filtered. Using the current state information of the target traffic participant detection box, the future location information of the target traffic participant detection box is predicted. This technical solution, combining the current location information, the future location information, and the traffic event warning area, for traffic risk identification and warning, avoids missed and false identification of traffic risk events, improves the reliability of traffic risk identification and warning, reduces warning latency, and avoids invalid warnings.

[0135] In some embodiments of the present invention, such as Figure 8 As shown, the real-time traffic risk warning system 1 includes, in addition to the computing device 10, a multi-source sensing device 20, a transmission device 30, and an alarm device 40.

[0136] In this embodiment of the invention, the multi-source sensing device 20 includes roadside sensing devices and vehicle-side sensing devices. The roadside sensing devices include cameras, millimeter-wave radar, lidar, traffic light data acquisition cards, etc. The vehicle-side sensing devices include vehicle GPS and vehicle status sensors (used to sense throttle opening, brake pedal opening, steering wheel angle, etc.).

[0137] The transmission device 30 may include an intersection switch, an RSU (Road Side Unit), and an OBU (Onboard Unit). Sensing information from the roadside sensing devices is aggregated to the intersection switch via Ethernet. Vehicle-mounted sensing devices transmit information to the RSU via the OBU, and then back to the intersection switch via Ethernet. Traffic participant sensing information detected by both the roadside and vehicle-mounted sensing devices is transmitted to the computing device 10 via the intersection switch.

[0138] The computing device 10 can be an edge MEC (Mobile Edge Computing) device. This device is used to determine the traffic event warning area and the first traffic participant detection box based on traffic perception information sent in real time by the multi-source sensing devices. Based on several first traffic participant detection boxes determined by the multi-source sensing devices, the target traffic participant detection box is fused and filtered. Using the current state information of the target traffic participant detection box, the future location information of the target traffic participant detection box is predicted. Combining the current location information, the future location information, and the traffic event warning area, traffic risk identification and warning are performed, and when an alarm is issued, the alarm information is sent to the alarm device.

[0139] The warning device 40 may be one or more of the following: LED (Light Emitting Diode) lights, roadside displays, vehicle-mounted displays, sound columns, luminous road studs, etc., used to alert traffic participants to traffic risks and to pay attention to safety through sound, light, electricity and other means.

[0140] Finally, it should be noted that those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium for the program can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. The above computer program embodiments can achieve the same or similar effects as any of the corresponding foregoing method embodiments.

[0141] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the functionality of various illustrative components, blocks, modules, circuits, and steps has been generally described. Whether this functionality is implemented as software or as hardware depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the functionality in various ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the embodiments disclosed herein.

[0142] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. The sequence numbers of the disclosed embodiments of this invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0143] It should be understood that, as used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, “and / or” refers to any and all possible combinations of one or more of the associated listed items.

[0144] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A method for real-time traffic risk warning, comprising: Based on the traffic sensing information sent in real time by multi-source sensing devices, determine the traffic incident alarm area and the first traffic participant detection box; Based on the first traffic participant detection box, determine the target traffic participant detection box; Based on the current state information of the target traffic participant detection box, predict the location information of the target traffic participant detection box in the future. Traffic risk identification and alerts are performed based on the current location information of the target traffic participant detection box, the future location information of the target traffic participant detection box, and the traffic event alarm area.

2. The method according to claim 1, characterized in that, Based on the traffic sensing information sent in real time by multi-source sensing devices, the traffic incident alarm area and the first traffic participant detection box are determined as follows: Based on real-time traffic environment images sent by visual sensors, determine the traffic incident warning area and the first traffic participant detection box; Based on the traffic participant perception information sent by the radar sensor and / or the traffic participant perception information sent by the vehicle-mounted perception device, a first traffic participant detection box located within the coverage area of ​​the traffic environment image is determined.

3. The method according to claim 1, characterized in that, Based on the first traffic participant detection box, the target traffic participant detection box is determined to include: Calculate the crossover ratio (CROR) of the first traffic participant detection boxes determined by any two sensing devices in the multi-source sensing devices to determine candidate boxes; Filter out the target traffic participant detection boxes from all candidate boxes.

4. The method according to claim 3, characterized in that, Calculate the crossover ratio (CROR) of the first traffic participant detection boxes determined by any two sensing devices in the multi-source sensing device group to determine the candidate boxes, including: In response to the intersection-union ratio being less than a first intersection-union ratio threshold, the first traffic participant detection box that participated in the intersection-union ratio calculation is determined as the candidate box; In response to the intersection-to-union ratio (CUI) being not less than the first CUI threshold, a traffic participant fusion box is determined based on the first traffic participant detection boxes corresponding to any two sensing devices, and the two first traffic participant detection boxes and the traffic participant fusion box are determined as the candidate boxes.

5. The method according to claim 3, characterized in that, From all candidate boxes, the target traffic participant detection boxes are selected as follows: The confidence level of the candidate boxes as target traffic participants is determined in order to filter out the target traffic participant detection boxes from all candidate boxes.

6. The method according to claim 5, characterized in that, The confidence level for determining the candidate bounding box as the target traffic participant includes: Based on the initial confidence level of the candidate box and the recognition capability weight of the multi-source sensing device corresponding to the candidate box, the confidence level of the candidate box as the target traffic participant is determined.

7. The method according to claim 5, characterized in that, Determining the confidence level of the candidate bounding boxes as target traffic participants, and then filtering out the target traffic participant detection boxes from all candidate bounding boxes, includes: Select the candidate box with the highest confidence from all candidate boxes; Based on the intersection-union ratio of the candidate box with the highest confidence and the other candidate boxes, the target traffic participant detection box is selected from all candidate boxes.

8. The method according to claim 1, characterized in that, The step of predicting the location information of the target traffic participant detection box in the future time based on the current state information of the target traffic participant detection box includes: Based on the LSTM algorithm, the position information of the target traffic participant detection box at future times is predicted according to the state information of the target traffic participant detection box at the current time.

9. The method according to claim 8, characterized in that, Based on the location information of the target traffic participant detection box at the current time, the location information of the target traffic participant detection box at a future time, and the traffic event alarm area, traffic risk identification and alarming include: Traffic risk identification and alerts are performed based on the current location information of the target traffic participant detection box, the future location information of the target traffic participant detection box, the traffic light information sent by the traffic light acquisition unit, and the traffic event alarm area.

10. A real-time traffic risk warning system, characterized in that, Includes a computing device, the computing device comprising: At least one processor; and A memory storing a computer program executable on the processor, wherein the processor, when executing the program, performs the steps of the method as described in any one of claims 1 to 8.

Images