Collision risk level determination method and device, storage medium, electronic equipment and product

Abstract

The invention discloses a collision risk level determination method and device, a storage medium and electronic equipment, and relates to the technical field of artificial intelligence, and the collision risk level determination method comprises the steps: receiving image data shot by a plurality of cameras on target protection equipment, the image data is used for recording position change information of a plurality of vehicles in a preset protection area of the target protection equipment and state change information of each vehicle; establishing a three-dimensional point cloud map between the target protection equipment and the plurality of vehicles according to the image data and inertia correction data of the target protection equipment corresponding to the image data; detecting the image data and the three-dimensional point cloud map through a target detection algorithm to obtain relative position parameters of the vehicle and target protection equipment and real-time characteristic parameters of a vehicle door; and determining a collision risk level of collision between the target protection equipment and the vehicle based on the relative position parameter and the real-time characteristic parameter.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a method, apparatus, storage medium, electronic device, and product for determining collision risk level. Background Technology

[0002] In modern urban traffic environments, electric bicycles have become an important choice for many city residents' daily travel. However, with the rapid increase in the number of electric bicycles, the safety hazards they face when riding on busy streets are becoming increasingly prominent, among which "door-opening accidents" are particularly serious. "Door-opening accidents" occur when an electric bicycle rider passes a parked vehicle and a passenger suddenly opens the door. The rider often has too little reaction time or obstructed vision to avoid the door, resulting in a direct collision and even secondary accidents, seriously threatening the rider's life. Traditional safety technologies, such as onboard blind spot monitoring systems and dashcams, are inadequate in dealing with "door-opening accidents." Onboard blind spot monitoring systems mainly focus on managing the vehicle's own blind spots and cannot anticipate sudden door openings from stationary vehicles on the roadside; while dashcams can save accident scene videos for post-accident analysis, they lack real-time warning functions and have limited effectiveness in preventing immediate danger. In addition, although ultrasonic or radar sensors can detect objects, their performance degrades significantly in rainy or snowy weather, and they can only sense the presence of objects but cannot analyze specific behaviors, such as the opening process of a car door.

[0003] Therefore, in related technologies, there is no effective solution yet for the risk of "door-opening kill" caused by a vehicle door suddenly opening, which cannot be predicted in real time and accurately. Summary of the Invention

[0004] This application provides a method, apparatus, storage medium, electronic device, and product for determining collision risk level, in order to at least solve the problem in the related art of being unable to accurately predict the risk of "door-opening kill" caused by a vehicle suddenly opening its door in real time.

[0005] According to one embodiment of this application, a method for determining a collision risk level is provided, comprising: receiving image data captured by multiple cameras on a target protection device, wherein the image data is used to record position change information of multiple vehicles within a preset protection area of ​​the target protection device and state change information of each vehicle; establishing a three-dimensional point cloud map between the target protection device and the multiple vehicles based on the image data and the inertial correction data of the target protection device corresponding to the image data; detecting the image data and the three-dimensional point cloud map using a target detection algorithm to obtain the relative position parameters between the vehicles and the target protection device and the real-time feature parameters of the vehicle doors; and determining the collision risk level of a collision between the target protection device and the vehicles based on the relative position parameters and the real-time feature parameters.

[0006] In an exemplary embodiment, a target detection algorithm is used to detect image data and a 3D point cloud map to obtain real-time feature parameters of a vehicle door. This includes: detecting image data to obtain key points of the vehicle door, wherein multiple key points include at least one of the following: door hinge, door handle, and door edge; determining the positional changes of the key points between consecutive image frames, and calculating the motion angle and motion speed of the door hinge based on the positional changes; and determining the real-time feature parameters of the vehicle door based on the motion angle and motion speed.

[0007] In one exemplary embodiment, determining the real-time characteristic parameters of a vehicle door based on its motion angle and motion speed includes: determining that the real-time characteristic parameters of the vehicle door are in an open state when the absolute value of the motion angle is greater than a preset angle threshold and the motion speed is greater than a preset speed threshold; determining that the real-time characteristic parameters of the vehicle door are in an about-to-open state when the absolute value of the motion angle is less than the preset angle threshold and the motion speed is close to zero; and determining that the real-time characteristic parameters of the vehicle door are in a stationary state when the absolute value of the motion angle is less than the preset angle threshold and the motion speed is less than the preset speed threshold.

[0008] In an exemplary embodiment, the relative position parameters between the vehicle and the target protection device are obtained by detecting image data and a three-dimensional point cloud map using a target detection algorithm, including: detecting the three-dimensional point cloud map to determine the first position coordinates of the vehicle in three-dimensional space and the second position coordinates of the target protection device in three-dimensional space; and determining the relative position parameters based on the first position coordinates and the second position coordinates.

[0009] In one exemplary embodiment, determining the collision risk level of a collision between a target protective device and a vehicle based on relative position parameters and real-time feature parameters includes: calculating a collision risk value between the target protective device and the vehicle based on the relative position parameters and real-time feature parameters; determining the risk level as low risk if the collision risk value is less than or equal to a first preset risk threshold; determining the risk level as medium risk if the collision risk value is greater than the first preset risk threshold but less than a second preset risk threshold; and determining the risk level as high risk if the collision risk value is greater than or equal to the second preset risk threshold.

[0010] In an exemplary embodiment, after determining the collision risk level of a collision between the target protective device and the vehicle based on relative position parameters and real-time feature parameters, the method further includes: executing a first response strategy when the risk level is low, wherein the first response strategy is implemented by issuing a light flashing command to a mobile device associated with the target protective device; executing a second response strategy when the risk level is medium, wherein the second response strategy is implemented by issuing a voice prompt command to a mobile device associated with the target protective device; and executing a third response strategy when the risk level is high, wherein the third response strategy is implemented by issuing a stop command to a mobile device associated with the target protective device.

[0011] According to another aspect of the embodiments of this application, a collision risk level determination device is also provided, comprising: a receiving module, configured to receive image data captured by multiple cameras on a target protection device, wherein the image data is used to record position change information of multiple vehicles within a preset protection area of ​​the target protection device and state change information of each vehicle; a building module, configured to build a three-dimensional point cloud map between the target protection device and the multiple vehicles based on the image data and the inertial correction data of the target protection device corresponding to the image data; a detection module, configured to detect the image data and the three-dimensional point cloud map using a target detection algorithm to obtain the relative position parameters between the vehicles and the target protection device and the real-time feature parameters of the vehicle doors; and a determination module, configured to determine the collision risk level of a collision between the target protection device and the vehicles based on the relative position parameters and the real-time feature parameters.

[0012] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the above-described method for determining the collision risk level when it is run.

[0013] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the collision risk level determination method through the computer program.

[0014] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program and a method for determining the above-mentioned collision risk level when the computer program is executed by a processor.

[0015] In this embodiment, image data captured by multiple cameras on a target protection device is received. This image data records the positional changes and state changes of multiple vehicles within a preset protection area of ​​the target protection device. A three-dimensional point cloud map between the target protection device and the multiple vehicles is established based on the image data and the corresponding inertial correction data of the target protection device. A target detection algorithm is used to detect the image data and the three-dimensional point cloud map to obtain the relative position parameters between the vehicles and the target protection device, as well as the real-time feature parameters of the vehicle doors. The collision risk level between the target protection device and the vehicles is determined based on the relative position parameters and the real-time feature parameters. This technical solution solves the problem of the inability to accurately predict the "door-opening kill" risk caused by sudden vehicle door opening in real time. Furthermore, by fusing multi-camera visual and inertial data, the relative position of the vehicles and the state of the doors are analyzed in real time to assess the collision risk level and provide accurate collision risk prediction. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the hardware environment for a method for determining a collision risk level according to an embodiment of this application;

[0019] Figure 2 This is a flowchart of a method for determining the collision risk level according to an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of a smart helmet system for preventing door opening and killing based on three-dimensional vision and AI prediction, according to an embodiment of this application.

[0021] Figure 4 This is a schematic flowchart of a door-opening-killing early warning method based on 3D vision and AI prediction according to an embodiment of this application;

[0022] Figure 5 This is a structural block diagram of a collision risk level determination device according to an embodiment of this application. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, apparatus, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, apparatus, or devices.

[0025] According to one aspect of the embodiments of this application, a method for determining a collision risk level is provided. This method for determining a collision risk level is widely applicable to whole-house intelligent digital control application scenarios such as smart homes, smart home ecosystems, and intelligence house ecosystems. Optionally, in this embodiment, the above-mentioned method for determining a collision risk level can be applied to, for example... Figure 1 The hardware environment shown consists of terminal device 102 and server 104. Figure 1 This is a schematic diagram of the hardware environment for a method for determining a collision risk level according to an embodiment of this application, such as... Figure 1 As shown, server 104 is connected to terminal device 102 via a network and can be used to provide services (such as application services) to the terminal or clients installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0026] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth. The terminal device 102 may not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projector, smart TV, smart clothes rack, smart curtains, smart audio-visual equipment, smart socket, smart speaker, smart speaker box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart robot vacuum cleaner, smart window cleaning robot, smart mopping robot, smart air purifier, smart steam oven, smart microwave oven, smart water heater, smart air purifier, smart water dispenser, smart door lock, etc.

[0027] This embodiment provides a method for determining the collision risk level, applied to the aforementioned terminal device. Figure 2 This is a flowchart of a method for determining the collision risk level according to an embodiment of this application. The process includes the following steps:

[0028] Step S202: Receive image data captured by multiple cameras on the target protection device, wherein the image data is used to record the position change information of multiple vehicles within the preset protection area of ​​the target protection device and the status change information of each vehicle;

[0029] Step S204: Establish a three-dimensional point cloud map between the target protection device and multiple vehicles based on the image data and the inertial correction data of the target protection device corresponding to the image data;

[0030] Step S206: The image data and the three-dimensional point cloud map are detected by the target detection algorithm to obtain the relative position parameters of the vehicle and the target protection equipment and the real-time feature parameters of the vehicle door.

[0031] Step S208: Determine the collision risk level of the target protection device colliding with the vehicle based on the relative position parameters and the real-time feature parameters.

[0032] Through the above steps, image data captured by multiple cameras on the target protection device is received. This image data records the positional changes and state changes of multiple vehicles within a preset protection area of ​​the target protection device. A three-dimensional point cloud map between the target protection device and the multiple vehicles is constructed based on the image data and the corresponding inertial correction data of the target protection device. A target detection algorithm is used to detect the image data and the three-dimensional point cloud map to obtain the relative position parameters between the vehicles and the target protection device, as well as the real-time feature parameters of the vehicle doors. Based on the relative position parameters and the real-time feature parameters, the collision risk level between the target protection device and the vehicles is determined. This technical solution solves the problem of the inability to accurately predict the "door-opening kill" risk caused by sudden vehicle door opening in real time. Furthermore, by fusing multi-camera visual and inertial data, the relative position of the vehicles and the state of the doors are analyzed in real time to assess the collision risk level and provide accurate collision risk prediction.

[0033] In an exemplary embodiment, a target detection algorithm is used to detect image data and a 3D point cloud map to obtain real-time feature parameters of a vehicle door. This includes: detecting image data to obtain key points of the vehicle door, wherein multiple key points include at least one of the following: door hinge, door handle, and door edge; determining the positional changes of the key points between consecutive image frames, and calculating the motion angle and motion speed of the door hinge based on the positional changes; and determining the real-time feature parameters of the vehicle door based on the motion angle and motion speed.

[0034] Optionally, when a cyclist wearing a smart helmet is riding on a city road, the helmet captures images of the road ahead using two high-resolution cameras positioned at the front and sides. When a parked car enters the helmet's field of view, the system first identifies the car door hinges, handles, and edges as key points using a target detection algorithm. Then, by analyzing several consecutive frames of images, the system calculates the positional change of the door hinges relative to the car body, thus determining the door's opening angle and speed. If a sudden increase in the door hinge angle or speed is detected, the system quickly determines that the door is opening or about to open. This method, based on key point detection and 3D dynamic analysis, enables the smart helmet to accurately identify and predict "door-opening-kill" events in complex urban environments, effectively improving the safety of electric bicycle riders.

[0035] In one exemplary embodiment, determining the real-time characteristic parameters of a vehicle door based on its motion angle and motion speed includes: determining that the real-time characteristic parameters of the vehicle door are in an open state when the absolute value of the motion angle is greater than a preset angle threshold and the motion speed is greater than a preset speed threshold; determining that the real-time characteristic parameters of the vehicle door are in an about-to-open state when the absolute value of the motion angle is less than the preset angle threshold and the motion speed is close to zero; and determining that the real-time characteristic parameters of the vehicle door are in a stationary state when the absolute value of the motion angle is less than the preset angle threshold and the motion speed is less than the preset speed threshold.

[0036] Optionally, an electric bicycle rider wears a smart helmet while riding. The helmet has preset angle thresholds of 3° and speed thresholds of 0.3 m / s to distinguish between a stationary, about-to-open, and open car door. When a parked car appears in the helmet's side-mounted camera's field of view, the movement of the door hinges is monitored. At this point, the hinge's angle of movement is 0°, and its speed is also 0 m / s, indicating the door is stationary. Shortly after, a passenger inside the car reaches out to open the door. Although the door hasn't moved, the passenger's hand movement causes a slight disturbance in the air near the hinge, detecting a slight increase in the hinge's speed to 0.2 m / s. The angle of movement remains 0°, which is below the threshold for the open state, but the near-zero speed change indicates an impending action. At this point, the door is determined to be about to open. Just as the rider notices the movement ahead, the passenger suddenly pulls open the door, causing the hinge's angle of movement to rapidly increase to 7° and the speed to jump to 0.5 m / s, exceeding the preset thresholds. The smart helmet immediately detected that the car door was open. Through this real-time feature parameter judgment based on specific thresholds, the smart helmet effectively provides cyclists with accurate and timely safety warnings in complex urban traffic environments, reducing the risk of accidents such as "door-opening accidents" and protecting the lives of cyclists.

[0037] In an exemplary embodiment, the relative position parameters between the vehicle and the target protection device are obtained by detecting image data and a three-dimensional point cloud map using a target detection algorithm, including: detecting the three-dimensional point cloud map to determine the first position coordinates of the vehicle in three-dimensional space and the second position coordinates of the target protection device in three-dimensional space; and determining the relative position parameters based on the first position coordinates and the second position coordinates.

[0038] Optionally, an electric bicycle rider wearing a smart helmet is traversing city streets. Suddenly, a parked vehicle enters the helmet's field of vision. The helmet's internal camera and LiDAR activate, capturing and constructing a 3D point cloud map of the vehicle's surroundings. An object detection algorithm identifies the vehicle's 3D bounding box on the point cloud map, determining the vehicle's precise position coordinates (x1, y1, z1) in 3D space. Simultaneously, the 3D position coordinates (x2, y2, z2) of the smart helmet (i.e., the rider) are also determined. Next, based on the 3D position coordinates of the vehicle and helmet, the relative distance and orientation between them are calculated. For example, by calculating the values ​​of (x1-x2), (y1-y2), and (z1-z2), the lateral, longitudinal, and vertical distances between the vehicle and the rider can be accurately determined. Through this method of calculating relative position parameters combining a 3D point cloud map and an object detection algorithm, the smart helmet can provide riders with accurate and timely safety warnings, effectively reducing traffic accidents such as "door-opening accidents."

[0039] In one exemplary embodiment, determining the collision risk level of a collision between a target protective device and a vehicle based on relative position parameters and real-time feature parameters includes: calculating a collision risk value between the target protective device and the vehicle based on the relative position parameters and real-time feature parameters; determining the risk level as low risk if the collision risk value is less than or equal to a first preset risk threshold; determining the risk level as medium risk if the collision risk value is greater than the first preset risk threshold but less than a second preset risk threshold; and determining the risk level as high risk if the collision risk value is greater than or equal to the second preset risk threshold.

[0040] Optionally, a cyclist is wearing a smart helmet. A parked car ahead suddenly attracts the helmet's attention, and the passengers inside are preparing to open the door. If, at a certain moment, the door has just begun to move, with an angle less than 5° and a speed of 0.2 m / s, and the relative distance between the car and the cyclist is greater than 15 meters, the calculated collision risk value is less than the first preset risk threshold (set to 0.1), therefore the current risk level is judged as low. As time progresses, the door's movement accelerates, reaching an angle of 10° and a speed of 0.5 m / s, while the relative distance between the car and the cyclist shortens to 10 meters, but not yet reaching a dangerous distance. At this point, the calculated collision risk value is between the first preset risk threshold (0.1) and the second preset risk threshold (set to 0.5), therefore it is judged as medium risk. Finally, the door's opening speed further increases, the angle exceeds 20°, and the relative distance between the car and the cyclist rapidly shortens to within 3 meters. At this point, the calculated collision risk value exceeds the second preset risk threshold (0.5), and the risk level is immediately determined to be high risk. Through this graded early warning mechanism, the smart helmet can not only effectively identify and assess the collision risk of "door-opening collisions," but also take appropriate early warning measures according to the severity of the risk, greatly improving the safety of cyclists in urban traffic environments and the practicality of the early warning system.

[0041] In an exemplary embodiment, after determining the collision risk level of a collision between the target protective device and the vehicle based on relative position parameters and real-time feature parameters, the method further includes: executing a first response strategy when the risk level is low, wherein the first response strategy is implemented by issuing a light flashing command to a mobile device associated with the target protective device; executing a second response strategy when the risk level is medium, wherein the second response strategy is implemented by issuing a voice prompt command to a mobile device associated with the target protective device; and executing a third response strategy when the risk level is high, wherein the third response strategy is implemented by issuing a stop command to a mobile device associated with the target protective device.

[0042] Optionally, a cyclist wears a smart helmet and a smartwatch linked to the helmet system on their wrist. The helmet detects a passenger preparing to open the door of a parked vehicle. The smart system detects that the door is just beginning to open, but the angle and speed are within a safe range, and the cyclist is still a sufficient distance away, classifying the situation as low-risk. At this point, the first response strategy is executed: a command is sent via Bluetooth to the cyclist's watch, causing the LED light on the watch to flash slowly, alerting the cyclist to a potential risk ahead, but without requiring immediate action. As the door's opening angle gradually increases and the speed accelerates, but still does not reach an emergency level, the situation is classified as medium-risk. At this point, the second response strategy is executed: the smart system plays a pre-recorded voice warning through the watch's speaker: "Caution ahead, door may be opening, please slow down and observe." This warning method is more direct; by conveying information through voice, the cyclist can immediately understand the risk and take measures such as slowing down or adjusting their route to avoid potential danger. When the door's opening angle and speed exceed a preset danger threshold, and the distance to the cyclist rapidly decreases to within a few meters, the system assesses the risk level as high-risk. At this point, the smart helmet immediately executes the third response strategy, sending an emergency stop command directly to the rider's smartwatch and the electric bicycle's control system. This integrated emergency response ensures that the device can act quickly to protect the rider's safety in dangerous situations. Through the implementation of the above-mentioned tiered warning and response strategies, the smart helmet and the rider's associated mobile devices jointly construct an efficient and hierarchical safety protection network, providing appropriate warnings and emergency avoidance measures at different risk levels, significantly improving safety in urban cycling environments.

[0043] Optionally, after receiving image data captured by multiple cameras on the target protection device, the method further includes: determining the three-dimensional position information and volume information of the vehicle in consecutive image frames; calculating the intersection-over-union ratio (IoU) between the three-dimensional positions of the vehicle in adjacent image frames based on the three-dimensional position information and volume information; and determining whether the vehicle is in an occluded state based on the value corresponding to the IoU. If the value corresponding to the IoU is less than a preset occlusion threshold, the vehicle is determined to be in an occluded state; if the value corresponding to the IoU is greater than or equal to the preset occlusion threshold, the vehicle is determined not to be in an occluded state.

[0044] To better understand the process of determining the collision risk level described above, the following description, in conjunction with optional embodiments, further illustrates the process of determining the collision risk level, but is not intended to limit the technical solutions of the embodiments of this application.

[0045] Among related technologies, "door-opening kills" cannot be effectively predicted. Traditional vehicle blind spot monitoring only targets the car's own blind spots and cannot identify the sudden opening of doors by stationary vehicles on the roadside. Ordinary cycling dashcams only have recording functions and lack active warning capabilities. Ultrasonic / radar sensors are easily affected by environmental interference (such as rain and snow) and have difficulty distinguishing between car door openings and other obstacles. In cycling scenarios, electric bicycles travel at relatively high speeds (20-30 km / h), and the rider's field of vision is easily obstructed. Roadside parking is dense, and there are various types of vehicles (cars / trucks / ride-hailing vehicles, etc.), requiring high-precision identification. The reaction time from door opening to collision is usually only 1-2 seconds, requiring early warning.

[0046] To address the aforementioned issues, this application proposes an optional embodiment of a door-opening-and-kill warning method based on 3D vision and AI prediction. This method first involves the synchronous acquisition of data from multi-view cameras and an IMU (Inertial Measurement Unit), using FPGA (Field-Programmable Gate Array) technology to ensure real-time and accurate image processing. Then, visual SLAM (Simultaneous Localization and Mapping) and stereo matching algorithms are used to process the acquired data, reconstructing a 3D model of the cycling environment in real time and accurately locating the spatial relationship between the vehicle and the cyclist. Next, an improved YOLO (You Only Look Once) model is used to detect and identify key features of the vehicle and doors, such as hinges, handlebars, and edges. Subsequently, the system uses an LSTM (Long Short-Term Memory) neural network to analyze the motion time-series data of the door hinges, predicting the probability of the door opening, and dynamically assessing the cyclist's collision risk by combining real-time relative position information. Finally, depending on the risk level, the smart helmet system takes corresponding early warning measures, including but not limited to visual prompts, tactile warnings, and even linkage with the electric bicycle's braking system, to prevent safety accidents such as "door-opening kills" and comprehensively improve the level of riding safety protection.

[0047] Optional, Figure 3 This is a schematic diagram of a smart helmet system for preventing door opening and killing based on three-dimensional vision and AI prediction according to an embodiment of this application. The system includes at least: a three-dimensional vision perception module 30, a deep learning intelligent analysis module 32, a collision risk assessment module 34, an early warning and execution module 36, and a system communication and control module 38.

[0048] Optionally, a 3D visual perception module 30 is included. This module consists of multi-view high-definition cameras and an IMU (Inertial Measurement Unit). The camera array can cover the front, left, right, and rear views, capturing real-time environmental images. The IMU is responsible for measuring the helmet's acceleration, angular velocity, and rotation angle to compensate for image shifts caused by helmet movement. By simultaneously exposing and acquiring data from multiple cameras, combined with the IMU's motion data, and utilizing visual SLAM technology, real-time 3D environment reconstruction is achieved, providing accurate spatial information for subsequent intelligent analysis.

[0049] Optional, a deep learning intelligent analysis module 32. Based on an improved YOLO model and LSTM neural network, where YOLO is used for real-time detection of vehicle and door features, and LSTM is used to analyze the temporal motion features of key points such as door hinges to predict the likelihood of door opening. This module can quickly identify and locate surrounding vehicles, paying particular attention to the state of doors, and predict potential "door-opening kill" behavior through deep learning algorithms, providing crucial information for collision risk assessment.

[0050] Optional, a collision risk assessment module 34. This module integrates data from visual perception and deep learning analysis, using a comprehensive evaluation algorithm to analyze real-time distance, relative speed, predicted door opening status, etc. It dynamically calculates the collision risk between the cyclist and surrounding vehicles, determines warning strategies based on different risk levels, and ensures the timeliness and effectiveness of warnings.

[0051] Optional, a warning and execution module 36. This module includes a warning signal generation unit and an external device communication interface, capable of sending warning signals to the rider and communicating with the electric bicycle's braking system in high-risk situations. Based on the collision risk assessment results, this module issues warnings to the rider through LED flashing lights, sound prompts, or vibration feedback. In extremely high-risk situations, it can actively control the electric bicycle to decelerate or brake suddenly, thereby avoiding or mitigating collisions.

[0052] Optional, a system communication and control module 38. This module ensures data flow synchronization and low-latency communication throughout the system, coordinates sensor data fusion, algorithm processing, and the distribution of early warning commands, and achieves efficient system-level collaboration.

[0053] Optional, Figure 4 This is a schematic flowchart of a door-opening-kill warning method based on 3D vision and AI prediction according to an embodiment of this application, which specifically includes the following steps:

[0054] Step 1: Synchronous Data Acquisition from Multiple Sensors. Hardware trigger signals are used to achieve synchronized exposure across multiple cameras, combined with IMU inertial data to compensate for helmet motion jitter. An optical distortion correction algorithm is employed to process the fisheye lens images, ensuring spatiotemporal alignment of multi-view visual data. Details are as follows:

[0055] (1) Hardware synchronization mechanism. The FPGA is used as the central controller to accurately generate synchronization signals to ensure that the three cameras start exposure at the same time and capture image data at the same moment. Then, the data is transmitted through the MIPI-CSI2 interface, which allows the cameras to transmit image data to the main processor in a high-speed and low-power manner, enabling the system to quickly collect and process a large amount of visual information.

[0056] (2) Image Data Processing. HDR (High Dynamic Range) image synthesis technology is used to address lighting issues in backlit environments or when passing through tunnels with significant brightness variations, ensuring clear image visibility even under poor lighting conditions, which aids in subsequent feature detection. A fisheye lens is used for the side-view camera to expand the field of view, but fisheye lenses suffer from severe geometric distortion. The fisheye module in the OpenCV library is used to correct distortion in real time on the captured images, and an equidistant projection model is used to recreate the real scene, improving the accuracy of image information.

[0057] Step Two: Real-time 3D Environment Reconstruction. Based on visual SLAM technology, a sparse 3D point cloud map is constructed through feature point matching and IMU data fusion. A stereo matching algorithm is used to generate a disparity map, which is then converted into a dense 3D environment model to calculate the spatial relationship between the vehicle and the rider in real time. Details are as follows:

[0058] (1) Visual Inertial Odometry (VIO): The front-end ORB-SLAM3 mainly uses ORB (Oriented Fast and Rotated BRIEF) feature points for fast environmental feature extraction, and combines IMU (Inertial Measurement Unit) data to compensate for image blur caused by helmet movement, ensuring stable tracking and positioning of feature points in dynamic environments; the back-end g2o optimizes the position and attitude of feature points by minimizing the error function, constructs a local map, generates sparse 3D point cloud, and provides a basic 3D environmental model for the system.

[0059] (2) Dense 3D Reconstruction. The disparity map between the left and right images is calculated using the SGM (Semi-Global Matching) algorithm. The disparity map is then converted into a 3D point cloud. This step involves converting the disparity value of each pixel into depth distance, and then combining the intrinsic and extrinsic parameters of the camera to calculate the position of each pixel in 3D space, thereby generating a dense 3D environment model.

[0060] Step 3: Vehicle and Door Detection. An improved YOLO deep learning model simultaneously detects the vehicle's 3D bounding box and key points of the door hinges. Trained on labeled 3D data, this model directly outputs the vehicle's 3D bounding box information, including its position coordinates (x, y, z), dimensions (length, width, height), and orientation in 3D space, enabling the system to understand the vehicle's true spatial relationship with the rider. In addition to the vehicle itself, the model also detects the structural features of the doors, including key points such as the door hinges. By analyzing the motion of these key points, it further determines whether the door is open or about to open, providing crucial information for collision risk assessment.

[0061] Step 4: Multi-target tracking. By fusing vehicle appearance features with 3D motion trajectories, an improved DeepSORT algorithm is used to achieve continuous target tracking. 3D IOU matching and motion prediction are used to address the target association problem under occlusion conditions. Details are as follows:

[0062] (1) Integration of Appearance and Motion Features. Based on the detected 3D bounding box of the vehicle and the structural features of the doors, unique appearance features of each vehicle can be extracted, including but not limited to color, shape, and the relative position of the door hinges. These features help to establish a more stable association between targets. Through visual inertial odometry (VIO) and 3D environment reconstruction technology, the 3D motion trajectory of each target can be estimated in real time, including information such as velocity, direction, and acceleration, providing dynamic position reference for subsequent target tracking.

[0063] (2) Improved DeepSORT algorithm for target tracking. The DeepSORT algorithm integrates the feature extraction capabilities of deep learning with the traditional Kalman filter motion prediction model, enabling continuous tracking of various targets in multi-target environments, even under occlusion or high dynamic conditions, without easily losing targets. Specifically, in the target detection and tracking process, three-dimensional IOU (Intersection over Union) is used for target matching, taking into account the overlap of targets in three-dimensional space, rather than being limited to two-dimensional images, thereby improving the accuracy and robustness of tracking. When the target is partially occluded, resulting in incomplete visibility of appearance features, the improved DeepSORT algorithm can continue to track the target through motion prediction and three-dimensional IOU matching, maintaining the continuity of the target and the coherence of recognition.

[0064] Step 5: Door Opening Behavior Prediction. An LSTM neural network analyzes the temporal motion characteristics of key points on the door hinge, including displacement changes and angular velocities. The model learns the difference between normal parking and preparing to open the door, outputting the probability of the door opening in the future. Specifically, through continuous detection of key points on the door hinge, a series of temporal motion data are collected, including changes in hinge displacement and rotational angular velocity. This data constitutes the main input to the LSTM neural network, used to analyze the dynamic trend of the door. Based on the collected temporal motion characteristics, the LSTM model can output the probability of the door opening in the future. The predicted opening probability is monitored in real time. Once a set threshold is reached, an early warning mechanism is immediately activated, notifying the rider to take appropriate evasive measures to avoid potential "door-opening accidents."

[0065] Optionally, an LSTM neural network is a special type of recurrent neural network (RNN) that is well-suited for processing time-series data, especially sequences with long-term dependencies. Here, the LSTM model is trained to recognize subtle differences between two behavioral patterns: a vehicle in a normal parking state and a preparatory door opening state. By analyzing changes in the displacement and angular velocity of the hinge point, the model learns to capture behavioral cues that may lead to door opening.

[0066] Step Six: Dynamic Collision Risk Assessment. A collision risk value that changes over time is generated by combining the real-time distance between the cyclist and the vehicle door, the relative velocity vector, the probability of the door opening, and the road type coefficient. Specifically, high-precision 3D environmental perception technology is used to monitor the relative distance between the cyclist and the vehicle door in real time. Through a multi-target tracking algorithm, not only is the static position of the vehicle obtained, but the relative speed and direction between it and the cyclist are also accurately calculated. Combining the analysis results of the motion characteristics of the key points of the door hinge using an LSTM neural network, the probability of the door opening in the next time period is predicted. The weight coefficients of the risk assessment are automatically adjusted according to the road type of the current cycling environment (e.g., busy city streets, relatively quiet suburban roads) to reflect the differences in collision risk under different road environments. Based on the above parameters, complex mathematical models and algorithms, such as machine learning classifiers or Bayesian networks, are used to comprehensively analyze the influence of various factors on the probability of a collision, thereby generating a collision risk value that changes over time.

[0067] Step Seven: Tiered Warning Decision-Making. Tiered warning decision-making is a key safety feature in smart helmet systems. It implements different levels of response strategies based on the estimated Time To Collision (TTC) threshold, ensuring that riders receive appropriate warnings based on immediate risks and effectively avoid collisions. Details are as follows:

[0068] (1) Long-distance risk: Visual cue. When a potential collision risk is detected, but the TTC is long, the smart helmet will issue a soft visual warning through built-in LED lights or a display screen to alert the rider to the potential danger ahead, without requiring immediate emergency action. This warning method will not cause excessive tension to the rider, but will ensure that they remain alert to their surroundings.

[0069] (2) Mid-range risk: Tactile warning. When the TTC is shortened to a certain range, i.e. the risk level increases, the helmet's built-in vibration module will be activated to produce moderate-intensity tactile feedback. This direct physical feedback is more likely to attract the rider's attention than simple visual cues, prompting them to prepare to take evasive action immediately.

[0070] (3) Emergency Distance: Multimodal Strong Warning and Vehicle Linkage. In situations where the TTC is extremely short, i.e., facing an emergency collision risk, the highest level of warning response is immediately activated. At this time, in addition to strong visual and tactile warnings, the smart helmet will further enhance the warning effect through audible alarms, ensuring that the rider is immediately aware of the danger even in a highly focused driving state. Furthermore, the helmet system can link with the rider's e-bike to automatically activate the emergency braking or deceleration assist system, minimizing the impact of the collision or completely avoiding it.

[0071] In summary, this application receives real-time images from multiple cameras, corrects distortion in fisheye lens images, analyzes the images using an improved YOLO network, generates feature maps containing vehicle and door features, predicts the vehicle's 3D position and size, and simultaneously detects four key points of the door (upper and lower hinges, handle, and edge), calculates the hinge motion angle and speed, and determines whether the door is stationary, opening, or about to open. The output labels the 3D positions of all vehicles in the image, specifically marking vehicles with a risk of door opening, and transmits the detection results to the early warning system. In conclusion, through distortion correction and deep learning analysis of real-time images from multiple cameras, this application accurately predicts the 3D position of vehicles and the state of doors, identifies "door-opening hazards" in advance, and efficiently links with the early warning system, significantly enhancing the safety protection capabilities of electric bicycle riders.

[0072] Through the above description of the embodiments, those skilled in the art can clearly understand that the collision risk level determination method according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software device. This computer software device is stored in a storage medium (such as ROM / RAM, disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the collision risk level determination method of the various embodiments of this application.

[0073] This embodiment also provides a collision risk level determination device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0074] Figure 5 This is a structural block diagram of a collision risk level determination device according to an embodiment of this application; as shown... Figure 5 As shown, it includes:

[0075] The receiving module 52 is used to receive image data captured by multiple cameras on the target protection device, wherein the image data is used to record the position change information of multiple vehicles within the preset protection area of ​​the target protection device and the status change information of each vehicle.

[0076] The module 54 is used to establish a three-dimensional point cloud map between the target protection device and multiple vehicles based on the image data and the inertial correction data of the target protection device corresponding to the image data.

[0077] The detection module 56 is used to detect the image data and the three-dimensional point cloud map through a target detection algorithm to obtain the relative position parameters of the vehicle and the target protection equipment and the real-time feature parameters of the vehicle door.

[0078] The determination module 58 is used to determine the collision risk level of the target protection equipment colliding with the vehicle based on the relative position parameters and the real-time feature parameters.

[0079] The aforementioned device receives image data captured by multiple cameras on a target protection device. This image data records the positional changes and state changes of multiple vehicles within a preset protection area of ​​the target protection device. A three-dimensional point cloud map is constructed between the target protection device and the multiple vehicles based on the image data and the corresponding inertial correction data of the target protection device. A target detection algorithm is used to detect the image data and the three-dimensional point cloud map to obtain the relative position parameters between the vehicles and the target protection device, as well as the real-time feature parameters of the vehicle doors. Based on the relative position parameters and the real-time feature parameters, the collision risk level between the target protection device and the vehicles is determined. This technical solution solves the problem of the inability to accurately predict the "door-opening kill" risk caused by sudden vehicle door opening in real time. Furthermore, by fusing multi-camera visual and inertial data, the relative position of the vehicles and the state of the doors are analyzed in real time to assess the collision risk level and provide accurate collision risk prediction.

[0080] In an exemplary embodiment, the detection module described above is further configured to detect image data to obtain key points of the vehicle door, wherein the multiple key points include at least one of the following: door hinge, door handle, and door edge; determine the positional changes of the key points between consecutive image frames, and calculate the motion angle and motion speed of the door hinge based on the positional changes; and determine the real-time feature parameters of the vehicle door based on the motion angle and motion speed.

[0081] In an exemplary embodiment, the detection module is further configured to: determine that the real-time characteristic parameters of the vehicle door are in an open state when the absolute value of the motion angle is greater than a preset angle threshold and the motion speed is greater than a preset speed threshold; determine that the real-time characteristic parameters of the vehicle door are in an about-to-open state when the absolute value of the motion angle is less than a preset angle threshold and the motion speed is close to zero; and determine that the real-time characteristic parameters of the vehicle door are in a stationary state when the absolute value of the motion angle is less than a preset angle threshold and the motion speed is less than a preset speed threshold.

[0082] In an exemplary embodiment, the detection algorithm described above is further used to detect a three-dimensional point cloud map, determine the first position coordinates of the vehicle in three-dimensional space and the second position coordinates of the target protective device in three-dimensional space; and determine relative position parameters based on the first position coordinates and the second position coordinates.

[0083] In an exemplary embodiment, the determining module is further configured to calculate the collision risk value of a collision between the target protective equipment and the vehicle based on relative position parameters and real-time feature parameters; determine the risk level as low risk level if the collision risk value is less than or equal to a first preset risk threshold; determine the risk level as medium risk level if the collision risk value is greater than the first preset risk threshold and less than a second preset risk threshold; and determine the risk level as high risk level if the collision risk value is greater than or equal to the second preset risk threshold.

[0084] In an exemplary embodiment, the determining module further includes: an execution unit, configured to determine the collision risk level of a collision between the target protective device and the vehicle based on relative position parameters and real-time feature parameters; and, if the risk level is low, execute a first response strategy by issuing a flashing light command to a mobile device associated with the target protective device; if the risk level is medium, execute a second response strategy by issuing a voice prompt command to the mobile device associated with the target protective device; and if the risk level is high, execute a third response strategy by issuing a stop command to the mobile device associated with the target protective device.

[0085] Embodiments of this application also provide a storage medium including a stored program, wherein the program, when executed, performs any of the above-described methods for determining collision risk levels.

[0086] Optionally, in this embodiment, the storage medium may be configured to store program code for performing the following steps:

[0087] S1, receiving image data captured by multiple cameras on the target protection device, wherein the image data is used to record the position change information of multiple vehicles within the preset protection area of ​​the target protection device and the status change information of each vehicle;

[0088] S2, establish a three-dimensional point cloud map between the target protection device and multiple vehicles based on the image data and the inertial correction data of the target protection device corresponding to the image data;

[0089] S3, the image data and the three-dimensional point cloud map are detected by the target detection algorithm to obtain the relative position parameters of the vehicle and the target protection equipment and the real-time feature parameters of the vehicle door;

[0090] S4. Based on the relative position parameters and the real-time feature parameters, determine the collision risk level of the target protection equipment colliding with the vehicle.

[0091] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0092] Embodiments of this application also provide a computer program product, including a computer program, wherein the computer program, when executed by a processor, performs the steps in any of the above method embodiments.

[0093] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0094] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0095] S1, receiving image data captured by multiple cameras on the target protection device, wherein the image data is used to record the position change information of multiple vehicles within the preset protection area of ​​the target protection device and the status change information of each vehicle;

[0096] S2, establish a three-dimensional point cloud map between the target protection device and multiple vehicles based on the image data and the inertial correction data of the target protection device corresponding to the image data;

[0097] S3, the image data and the three-dimensional point cloud map are detected by the target detection algorithm to obtain the relative position parameters of the vehicle and the target protection equipment and the real-time feature parameters of the vehicle door;

[0098] S4. Based on the relative position parameters and the real-time feature parameters, determine the collision risk level of the target protection equipment colliding with the vehicle.

[0099] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0100] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0101] The collection, storage, use, processing, transmission, provision, and disclosure of financial data or user data involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0102] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used the solution.

[0103] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0104] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for determining collision risk level, characterized in that, include: Receive image data captured by multiple cameras on the target protection device, wherein the image data is used to record the position change information of multiple vehicles within the preset protection area of ​​the target protection device and the status change information of each vehicle; A three-dimensional point cloud map between the target protection device and multiple vehicles is established based on the image data and the inertial correction data of the target protection device corresponding to the image data. The image data and the three-dimensional point cloud map are detected by the target detection algorithm to obtain the relative position parameters of the vehicle and the target protection equipment and the real-time feature parameters of the vehicle door. The collision risk level of the target protection equipment colliding with the vehicle is determined based on the relative position parameters and the real-time feature parameters.

2. The method for determining the collision risk level according to claim 1, characterized in that, The image data and the 3D point cloud map are detected using an object detection algorithm to obtain real-time feature parameters of the vehicle doors, including: The image data is detected to obtain the key points of the vehicle door, wherein the multiple key points include at least one of the following: door hinge, door handle, and door edge; Determine the positional changes of the key points between consecutive image frames, and calculate the motion angle and speed of the door hinge based on the positional changes; The real-time characteristic parameters of the vehicle door are determined based on the motion angle and the motion speed.

3. The method for determining the collision risk level according to claim 2, characterized in that, The real-time characteristic parameters of the vehicle door are determined based on the motion angle and the motion speed, including: If the absolute value of the motion angle is greater than a preset angle threshold and the motion speed is greater than a preset speed threshold, the real-time characteristic parameters of the vehicle door are determined to be in an open state. When the absolute value of the motion angle is less than a preset angle threshold and the motion speed is close to zero, the real-time characteristic parameters of the vehicle door are determined to be in an imminent opening state. If the absolute value of the motion angle is less than a preset angle threshold and the motion speed is less than a preset speed threshold, the real-time characteristic parameters of the vehicle door are determined to be in a stationary state.

4. The method for determining the collision risk level according to claim 1, characterized in that, The image data and the 3D point cloud map are detected using a target detection algorithm to obtain the relative position parameters between the vehicle and the target protection equipment, including: The three-dimensional point cloud map is inspected to determine the first position coordinates of the vehicle in three-dimensional space and the second position coordinates of the target protective equipment in three-dimensional space. The relative position parameters are determined based on the first position coordinates and the second position coordinates.

5. The method for determining the collision risk level according to claim 1, characterized in that, Determining the collision risk level of the target protection equipment colliding with the vehicle based on the relative position parameters and the real-time feature parameters includes: The collision risk value of the target protection equipment colliding with the vehicle is calculated based on the relative position parameters and the real-time feature parameters. If the collision risk value is less than or equal to a first preset risk threshold, the risk level is determined to be a low risk level. If the collision risk value is greater than a first preset risk threshold and less than a second preset risk threshold, the risk level is determined to be a medium risk level. If the collision risk value is greater than or equal to the second preset risk threshold, the risk level is determined to be a high risk level.

6. The method for determining the collision risk level according to claim 5, characterized in that, After determining the collision risk level of the target protection device colliding with the vehicle based on the relative position parameters and the real-time feature parameters, the method further includes: When the risk level is low, a first response strategy is executed, wherein the first response strategy is to send a light flashing command to a mobile device associated with the target protection device; If the risk level is medium risk, a second response strategy is executed, wherein the second response strategy is to issue a voice prompt command to the mobile device associated with the target protection device; If the risk level is high, a third response strategy is executed, wherein the third response strategy is to issue a stop command to the mobile device associated with the target protection device.

7. A device for determining the collision risk level, characterized in that, include: The receiving module is used to receive image data captured by multiple cameras on the target protection device, wherein the image data is used to record the position change information of multiple vehicles within the preset protection area of ​​the target protection device and the status change information of each vehicle; A module is established to create a three-dimensional point cloud map between the target protection device and multiple vehicles based on the image data and the inertial correction data of the target protection device corresponding to the image data. The detection module is used to detect the image data and the three-dimensional point cloud map through a target detection algorithm to obtain the relative position parameters of the vehicle and the target protection equipment and the real-time feature parameters of the vehicle door; The determination module is used to determine the collision risk level of the target protection equipment colliding with the vehicle based on the relative position parameters and the real-time feature parameters.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the method for determining the collision risk level as described in any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for determining the collision risk level as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method for determining the collision risk level as described in any one of claims 1 to 6.

Images