Method and system for predicting vehicle collisions based on digital twin in smart rsu

Abstract

The method for predicting a vehicle collision based on a digital twin in a smart RSU according to the present invention may include the steps of: acquiring traffic data in real time from a smart RSU installed on a road; simulating the acquired traffic data on a digital twin to recreate a virtual road; inputting the results of the digital twin simulation into an artificial intelligence model to predict the probability of a vehicle collision on the virtual road; generating collision avoidance information through a reinforcement learning algorithm (PPO) based on the probability of a vehicle collision; and transmitting the collision avoidance information to a vehicle and other devices using V2X communication.

Description

Technology Field

[0001] A method and system for predicting vehicle collisions based on a digital twin in a smart RSU according to one embodiment relates to the field of intelligent traffic safety systems and V2X communication-based collision prediction technology that generate a virtual road environment based on real-time traffic data and determine the possibility of collision between vehicles in advance using artificial intelligence. Background Technology

[0003] Most existing traffic safety systems adopt cloud-based data analysis structures; however, they face limitations in achieving millisecond-level real-time responses due to network latency issues arising from the data collection to analysis process. Furthermore, most collision warning systems rely on physics-based models such as Time-To-Collision (TTC) or Time-To-Intersection (TTI). Since these models perform only simple mathematical calculations based on current vehicle speed and distance information, they are limited in reflecting sudden situations or surrounding traffic factors. For instance, while a high risk of collision exists if two vehicles have similar TTI values ​​close to an intersection, these physics models fail to account for complex scenarios such as changes in driving patterns or pedestrian intervention.

[0004] Edge computing technology based on Smart Roadside Units (RSUs) is attracting attention as a technical alternative to overcome these limitations. Smart RSUs integrate with various sensors such as LIDAR, RADAR, CCTV, weather sensors, and GNSS. By analyzing collected data in real time based on V2X (Vehicle-to-Everything) communication capabilities and edge processors, they can instantly identify information such as distance, speed, and trajectory between vehicles. Through this, collision risks can be detected in advance at a local level without network delay, and accidents can be prevented by sending warning messages to the relevant vehicle or surrounding vehicles via V2I and V2V communication, or by dynamically adjusting the flow of traffic entering intersections through signal controllers.

[0005] While such smart RSU-based systems offer improved performance compared to existing systems in terms of real-time capabilities and responsiveness, they still have limitations in precisely predicting complex traffic situations or presenting response strategies based on various variables. Technical constraints are cited, such as the difficulty of relying solely on physics-based models to adequately reflect various unexpected situations, interactions with pedestrians, and behavioral patterns in multi-lane environments, as well as the challenge of deriving proactive avoidance strategies beyond simple warning delivery. Prior art literature

[0007] Republic of Korea Patent Registration No. 10-2226067 (Registered on March 4, 2021) Republic of Korea Patent Registration No. 10-2739940 (Registered on December 3, 2024) The problem to be solved

[0008] In addressing the aforementioned problems, the objective of the present invention is to overcome the limitations of existing traffic safety systems lacking real-time capability and prediction precision, and to provide a method and system for predicting vehicle collisions based on a digital twin in a smart RSU that can precisely predict the probability of collision and respond actively based on AI and digital twins. means of solving the problem

[0010] The method for predicting a vehicle collision based on a digital twin in a smart RSU according to the present invention may include the steps of: acquiring traffic data in real time from a smart RSU installed on a road; simulating the acquired traffic data on a digital twin to recreate a virtual road; inputting the results of the digital twin simulation into an artificial intelligence model to predict the probability of a vehicle collision on the virtual road; generating collision avoidance information through a reinforcement learning algorithm (PPO) based on the probability of a vehicle collision; and transmitting the collision avoidance information to a vehicle and other devices using V2X communication.

[0011] According to one embodiment, the step of reproducing a virtual road may include: generating a virtual scenario for the surrounding conditions of a road and an intersection using pre-collected traffic data; training a collision prediction model based on the generated virtual scenario; dynamically updating the environment of the virtual road on a digital twin using traffic data obtained from a smart RSU; and updating the collision prediction model by comparing the probability of a vehicle collision obtained from an artificial intelligence model with the actual collision result.

[0012] According to one embodiment, the step of predicting the possibility of a vehicle collision may include: a step of recognizing a vehicle by performing CNN-based object recognition on image data at multiple points in time; a step of predicting multiple movement paths of the vehicle by inputting the image data in which the vehicle is recognized in chronological order into an LSTM model; and a step of calculating the possibility of a vehicle collision by integrating the predicted multiple movement paths through a fully connected layer.

[0013] According to one embodiment, the step of generating collision avoidance information may include: acquiring vehicle state information including the vehicle's current position, movement speed, and dynamic information of objects around the vehicle; inputting the vehicle state information into a reinforcement learning algorithm to generate next action information of the vehicle capable of avoiding a collision; calculating a reward score based on the application of the next action information; applying a clipping loss function to a plurality of avoidance policies generated based on the vehicle state information, next action information, and reward score; and determining the avoidance policy with the highest expected reward among the plurality of avoidance policies as the collision avoidance information.

[0014] According to one embodiment, the step of applying a clipping method loss function may include a step of calculating the probability ratio of a new avoidance policy relative to an existing avoidance policy based on vehicle state information, next action information, and a reward score collected for each of a plurality of avoidance policies, and a step of preventing a decrease in stability that occurs during policy update by applying a loss function so that the probability ratio is maintained within a preset range.

[0015] According to one embodiment, policy updates are performed repeatedly in response to a predetermined period or a change in vehicle density on a virtual road, and for each repetition, the parameters of the policy update may be dynamically updated only when the prediction stability score is below a standard, using the real-time number of objects on the virtual road, average vehicle speed, and rate of change of TTC (Time to Collision) as evaluation indicators.

[0016] According to one embodiment, the collision avoidance information is characterized by including first information transmitted to a vehicle, second information transmitted to a signal control device around the vehicle, and third information transmitted to other objects around the vehicle, wherein the first information includes at least one of a warning message notifying the driver of the vehicle or the autonomous driving control system of the vehicle of a collision risk or control information controlling the direction and speed of movement of the vehicle, the second information includes a control command controlling the signal of a signal control device located on the vehicle's movement path according to the possibility of a collision of the vehicle, and the third information includes a collision warning message and recommended avoidance path information transmitted to other vehicles and pedestrians located within a critical radius from the vehicle.

[0017] A system for predicting vehicle collisions based on a digital twin in a smart RSU according to one embodiment of the present invention may include a communication unit that communicates with vehicles, terminals, and external servers around the smart RSU using V2X communication, a sensing unit that senses traffic data around the smart RSU, and an edge computing module that simulates real-time traffic data from the sensing unit on a digital twin to recreate a virtual road, inputs the results of the digital twin simulation into an artificial intelligence model to predict the probability of a vehicle collision on the virtual road, generates collision avoidance information through a reinforcement learning algorithm (PPO) based on the probability of a vehicle collision, and transmits the collision avoidance information to vehicles and other devices using V2X communication.

[0018] According to one embodiment, the communication unit communicates with the vehicle using a V2X communication service when the vehicle is located in a V2X communication service area, and communicates with the vehicle using a mobile communication service when the vehicle is not located in a V2X communication service area, and is characterized by continuously attempting communication using the V2X communication service.

[0019] According to one embodiment, the step of reproducing the virtual road may include: extracting a first vehicle and state information of the first vehicle from the traffic data—the state information includes the position, speed, and direction of travel of the vehicle—; using the state information to create a first vehicle object and a second vehicle object that can be simulated along the movement path of the first vehicle and the second vehicle; creating a pedestrian object with a movement trajectory from the traffic data; creating an intersection object by reflecting the road shape, number of lanes, and signal cycle from the traffic data; and simulating the relative positional relationship and state change at each point in time between the first vehicle object, the second vehicle object, the pedestrian object, and the intersection object, and displaying them on a digital twin.

[0020] According to one embodiment, when the probability of a collision is greater than or equal to a threshold probability at the time when the first vehicle object and the second vehicle object enter the intersection object, the method may include the steps of generating a plurality of avoidance scenarios capable of avoiding a collision between the first vehicle object and the second vehicle object, evaluating the collision avoidance success rate, vehicle operation stability, and degree of interference with the pedestrian object for each of the plurality of avoidance scenarios, and determining the priority of the plurality of avoidance scenarios according to the evaluation results.

[0021] According to one embodiment, the method may include the steps of: calculating the estimated time to reach the intersection (TTI) of each of the first vehicle object and the second vehicle object on the digital twin; calculating the estimated time remaining until collision (TTC) between the first vehicle object and the second vehicle object; and, if the difference (△TTI) of the TTI and the TTC are less than or equal to a preset threshold time, calculating a collision probability value using the following mathematical formula 1.

[0022] [Mathematical Formula 1]

[0023]

[0024] Here, P collision is a collision probability value, TTI1 is the estimated time of arrival of the first vehicle object at the intersection, TTI2 is the estimated time of arrival of the second vehicle object at the intersection, and θ is a threshold time value for determining collision risk. Here, as TTI approaches 0, the value approaches 1, and the collision probability is calculated to be very high, and if △TTI is greater than or equal to θ, it is 0 or ignored.

[0025] According to one embodiment, if the collision probability value is 0.75 or higher, the system determines that a collision is imminent and immediately generates a forced avoidance scenario and transmits it to the vehicle and other servers; if the value is 0.50 or higher but less than 0.75, it determines that a collision is possible and generates a preliminary scenario or sends a warning message to the driver; and if the value is less than 0.25, it determines that the collision probability is low and skips collision response or performs only minimal monitoring. Effects of the invention

[0027] According to one embodiment, by integrating the AI-based collision prevention structure of a smart RSU with digital twin technology, it provides the effect of reducing the high risk of accidents occurring in complex road environments, such as intersections, and overcoming the limitations of existing traffic safety systems. In particular, it can enhance the accuracy, speed, and efficiency of a real-time collision prevention system by compensating for the delay issues of cloud-based data processing and the limitations in precision of physical model-centric collision warning methods.

[0028] According to one embodiment, traffic data collected in real time from a smart RSU installed on a road is reflected in a virtual environment through a digital twin simulation, and the possibility of a vehicle collision is predicted in advance using an artificial intelligence model, thereby enabling early response to collision risks. In particular, active intervention by vehicles or infrastructure becomes possible through the generation of a reinforcement learning-based avoidance policy, which can effectively reduce the accident rate.

[0029] According to one embodiment, the movement path of a vehicle can be precisely tracked through image-based object recognition and path prediction at multiple viewpoints, and dynamic and highly accurate prediction performance can be achieved based on judgment criteria utilizing real-time traffic indicators such as Time to Collision (TTC). This offers greater effectiveness compared to existing static simulation-based prediction techniques. Furthermore, since collision avoidance information is transmitted in a multi-layered manner, categorized into vehicles, signal control devices, and surrounding objects, it enables not only vehicle-only avoidance but also cooperative avoidance control based on road infrastructure. This can be effectively applied to Intelligent Transportation Systems (ITS) based on smart cities. Brief explanation of the drawing

[0031] FIG. 1 is a diagram schematically illustrating the technical features of predicting vehicle collisions based on a digital twin in a smart RSU according to one embodiment. FIG. 2 is a flowchart illustrating a method for predicting vehicle collisions based on a digital twin in a smart RSU according to one embodiment. FIG. 3 is a diagram showing the configuration of a system for predicting vehicle collisions based on a digital twin in a smart RSU according to one embodiment. FIG. 4 is a diagram specifically illustrating the architecture of a system for predicting vehicle collisions based on a digital twin in a smart RSU according to one embodiment. FIG. 5 is a diagram showing a series of flows for generating collision avoidance information according to one embodiment. FIG. 6 is a diagram illustrating the detailed configuration of an artificial intelligence model learning unit included in an edge computing module according to one embodiment. Specific details for implementing the invention

[0032] In the present invention, the attached drawings may be illustrated with exaggerated expressions to distinguish it from the prior art, ensure clarity, and facilitate the understanding of the technology. Furthermore, the terms described below are defined considering their functions in the present invention; since these terms may vary depending on the intentions or conventions of the user or operator, their definitions should be based on the technical content throughout this specification. Meanwhile, the embodiments are merely exemplary details of the components presented in the claims of the present invention and do not limit the scope of the rights of the present invention; the scope of rights should be interpreted based on the technical concept throughout the specification of the present invention.

[0033] Throughout the specification, when a configuration is described as "including" a configuration, this means that, unless specifically stated otherwise, it does not exclude other configurations but may include additional configurations.

[0034] Furthermore, when it is said that one configuration is "connected," "connected," or "combined" with another configuration, this means that it is not only "directly connected," "directly connected," or "directly combined," but also that there may be cases where it is "connected with another configuration interposed," "connected with another configuration interposed," or "combined with another configuration interposed." On the other hand, when it is said that one configuration is "directly connected," "directly connected," or "directly combined" with another configuration, it should be understood that there is no other configuration in between.

[0035] In addition, when directional terms such as "front," "back," "up," "down," "left," "right," "first end," "other end," and "both ends" are used, they are used exemplarily in relation to the orientation of the disclosed drawings and should not be interpreted restrictively, and when terms such as "first" and "second" are used, they are terms used to distinguish each configuration and should not be interpreted restrictively.

[0036] In order to more clearly explain the features of the embodiments of the present invention, detailed descriptions of matters widely known to those skilled in the art to which the following embodiments pertain are omitted. Additionally, detailed descriptions of parts in the drawings that are unrelated to the description of the embodiments are omitted.

[0037] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0039] In the following, the system (10) for predicting vehicle collisions based on a digital twin in a smart RSU will be described as a vehicle collision prediction system (10).

[0040] FIG. 1 is a diagram schematically illustrating the technical features of predicting vehicle collisions based on a digital twin in a smart RSU according to one embodiment.

[0041] According to one embodiment, a vehicle collision prediction system (10) can provide a system that can prevent accidents by simulating a plurality of vehicle objects attempting to enter an intersection on a digital twin, determining the possibility of collision between vehicles, generating collision avoidance information, and transmitting it to vehicles and infrastructure.

[0042] Referring to FIG. 1, the vehicle collision prediction system (10) can calculate the probability of a collision by reproducing the situation in which a first vehicle object and a second vehicle object enter an intersection on a virtual road as a digital twin, and reflecting state information such as the relative position, speed, and direction of travel between them in real time.

[0043] According to one embodiment, the probability of collision can be calculated based on the estimated time to reach the intersection (TTI, Time-To-Intersection) of each of the first vehicle object and the second vehicle object, and the time remaining until collision between the two vehicles (TTC, Time-To-Collision), and if △TTI and TTC are below a predetermined threshold, the probability of collision (P collision ) is calculated to a value close to 1. This calculation can be performed in real time on edge computing devices.

[0044] According to one embodiment, a vehicle collision prediction system (10) follows a four-stage flow (pre-training - virtualization - real-time application - feedback) of a digital twin-based collision prediction structure, object recognition is performed through a CNN model, and trajectory prediction can be performed by an LSTM-based time series model. When the predicted collision probability is greater than or equal to a threshold, multiple avoidance scenarios are generated through a PPO-based reinforcement learning model, and each scenario is evaluated and assigned a priority based on indicators such as avoidance success rate, pedestrian interference, driving stability, and expected reward score.

[0045] According to one embodiment, collision avoidance information and warning messages are transmitted to the vehicle via V2X-based I2V communication, and if necessary, indirect avoidance intervention is also performed through a signal controller (TSC) installed at the intersection.

[0046] According to one embodiment, if the probability of a collision is determined to be greater than a threshold probability, the system may generate a warning message and generate a plurality of avoidance scenarios by reflecting the movement paths, speeds, and surrounding object conditions of the first vehicle object and the second vehicle object. Each avoidance scenario is evaluated based on the collision avoidance success rate, driving stability, and the degree of interference with pedestrian objects, and the optimal avoidance scenario may be determined according to priority based on the evaluation results.

[0047] According to one embodiment, such warning messages and collision avoidance information are transmitted via V2X communication to the vehicle (first vehicle object, second vehicle object), a signal control device located at an intersection, and other objects such as nearby pedestrians or surrounding vehicles, thereby inducing them to perform actions necessary to prevent collision in advance.

[0048] FIG. 2 is a flowchart illustrating a method for predicting vehicle collisions based on a digital twin in a smart RSU according to one embodiment.

[0049] The method for predicting a vehicle collision based on a digital twin in a smart RSU according to the present invention may include the steps of: acquiring traffic data in real time from a smart RSU installed on a road (S10); simulating the acquired traffic data on a digital twin to recreate a virtual road (S20); inputting the results of the digital twin simulation into an artificial intelligence model to predict the probability of a vehicle collision on the virtual road (S30); generating collision avoidance information through a reinforcement learning algorithm (PPO) based on the probability of a vehicle collision (S40); and transmitting the collision avoidance information to a vehicle and other devices using V2X communication (S50).

[0050] According to one embodiment, in step S10, the vehicle collision prediction system (10) can obtain real-time traffic data such as the location, speed, direction of travel, and signal status of a vehicle from a smart RSU installed in the road infrastructure through V2X communication or a sensing device, and this data can be used as an initial value for the simulation.

[0051] According to one embodiment, in step S20, the vehicle collision prediction system (10) creates virtual roads, intersections, vehicle objects (first vehicle object, second vehicle object), pedestrian objects, etc. on a digital twin based on the acquired traffic data, performs a simulation, and dynamically reproduces real-time road conditions. At this time, the state information of the objects (position, speed, etc.) is updated along the time axis.

[0052] According to one embodiment, the step of reproducing a virtual road may include: generating a virtual scenario for the surrounding conditions of a road and an intersection using pre-collected traffic data; training a collision prediction model based on the generated virtual scenario; dynamically updating the environment of the virtual road on a digital twin using traffic data obtained from a smart RSU; and updating the collision prediction model by comparing the probability of a vehicle collision obtained from an artificial intelligence model with the actual collision result.

[0053] According to one embodiment, the vehicle collision prediction system according to the present invention is configured to improve road safety by predicting the probability of a vehicle collision in real time in a digital twin environment based on traffic data collected from a smart RSU (Road Side Unit) installed in road infrastructure, and deriving an optimal scenario for collision avoidance.

[0054] More specifically, taking an urban intersection environment as an example, assume a scenario where the first vehicle is proceeding straight and the second vehicle is attempting a left or right turn from the crossing direction, and both vehicles enter the intersection simultaneously. In this case, the RSU can collect status information including the position, speed, and direction of travel of each of the first and second vehicles.

[0055] According to one embodiment, the step of reproducing the virtual road may include: extracting a first vehicle and state information of the first vehicle from the traffic data—the state information includes the position, speed, and direction of travel of the vehicle—; using the state information to create a first vehicle object and a second vehicle object that can be simulated along the movement path of the first vehicle and the second vehicle; creating a pedestrian object with a movement trajectory from the traffic data; creating an intersection object by reflecting the road shape, number of lanes, and signal cycle from the traffic data; and simulating the relative positional relationship and state change at each point in time between the first vehicle object, the second vehicle object, the pedestrian object, and the intersection object, and displaying them on a digital twin.

[0056] According to one embodiment, the first vehicle object and the second vehicle object can be modeled as simulation objects having trajectories according to their respective positions, speeds, and directions, the pedestrian object is a moving object located on a crosswalk and includes a movement path that may enter an intersection, and the intersection object can be created as an environment object including static information such as the number of lanes, road direction, and signal cycle. Subsequently, on the digital twin, the state information of the above objects is dynamically updated at each point in time along the time axis, and a simulation environment for determining the possibility of collision can be maintained in real time.

[0057] According to one embodiment, when the probability of a collision is greater than or equal to a threshold probability at the time when the first vehicle object and the second vehicle object enter the intersection object, the method may include the steps of generating a plurality of avoidance scenarios capable of avoiding a collision between the first vehicle object and the second vehicle object, evaluating the collision avoidance success rate, vehicle operation stability, and degree of interference with the pedestrian object for each of the plurality of avoidance scenarios, and determining the priority of the plurality of avoidance scenarios according to the evaluation results.

[0058] According to one embodiment, the method may include the steps of: calculating the estimated time to reach the intersection (TTI) of each of the first vehicle object and the second vehicle object on the digital twin; calculating the estimated time remaining until collision (TTC) between the first vehicle object and the second vehicle object; and, if the difference (△TTI) of the TTI and the TTC are less than or equal to a preset threshold time, calculating a collision probability value using the following mathematical formula 1.

[0059]

[0060] Here, P collision is a collision probability value, TTI1 is the estimated time of arrival of the first vehicle object at the intersection, TTI2 is the estimated time of arrival of the second vehicle object at the intersection, and θ is a threshold time value for determining collision risk. Here, as TTI approaches 0, the value approaches 1, and the collision probability is calculated to be very high, and if △TTI is greater than or equal to θ, it can be 0 or ignored.

[0061] According to one embodiment, the classification and avoidance scenarios based on the possibility of collision are derived as shown in Table 1 below.

[0062] Collision probability value Judgment criteria System operation ≥0.75 Imminent collision Immediate generation of warning messages and forced evasion scenarios 0.5≤P<0.75 Possibility of collision exists Create preliminary avoidance scenarios or send warnings P<0.25 Low risk of collision Skip collision response or perform monitoring only

[0063] For example, if the TTI of the first vehicle is 3.0 seconds, the TTI of the second vehicle is 2.0 seconds, and θ=2.0 seconds, P collision When the value is 0.5, it is determined that there is a possibility of collision and a preliminary avoidance scenario can be generated. According to one embodiment, if the collision possibility value is 0.75 or higher, the system determines that a collision is imminent and immediately generates a forced avoidance scenario and transmits it to the vehicle and other servers; if the value is 0.50 or higher but less than 0.75, it determines that there is a possibility of collision and generates a preliminary scenario or sends a warning message to the driver; and if the value is less than 0.25, it determines that there is a low possibility of collision and skips collision response or performs only minimal monitoring.

[0064] According to one embodiment, when the probability of a collision is determined to be greater than a threshold, the system generates the following scenario through a PPO (Proximal Policy Optimization) based reinforcement learning model.

[0065] Scenario A: Deceleration of the 1st Vehicle

[0066] Scenario B: Temporarily stop the second vehicle

[0067] Scenario C: Signal Controller Priority Control (TSC Intervention)

[0068] According to one embodiment, the following evaluation indicators can be calculated for each scenario.

[0069] Collision avoidance success rate (based on prediction simulation)

[0070] Vehicle driving stability (sudden deceleration / sudden braking)

[0071] Whether pedestrian interference occurs (overlap with the movement path of pedestrian objects)

[0072] Reinforcement learning expected reward (based on reward function)

[0073] Based on these evaluation indicators, a priority score is assigned to each scenario, and the avoidance scenario with the highest score can be selected and generated as collision avoidance information.

[0074] According to one embodiment, the finally selected collision avoidance information may be transmitted to the next target via V2X communication. For example, the first information may be a warning message or control command sent to a vehicle driver or an autonomous driving control system, the second information may include a signal change request sent to an intersection signal control device, and the third information may include a collision warning and recommended avoidance path sent to nearby vehicle and pedestrian objects.

[0075] Through such a series of operations, the present invention can contribute to the improvement of safety based on smart mobility infrastructure by determining the possibility of collision in real time in a complex urban intersection environment and performing a preemptive response accordingly.

[0076] According to one embodiment, in step S30, the vehicle collision prediction system (10) inputs the results of a digital twin simulation into an artificial intelligence model (CNN-LSTM, etc.) to predict the probability of a collision between vehicles. Time-To-Intersection (TTI) and Time-To-Collision (TTC) are utilized for the prediction, and if the △TTI of the two vehicles is below a threshold and the TTC is below a certain level, it is determined that there is a high probability of a collision.

[0077] According to one embodiment, in step S40, the vehicle collision prediction system (10) generates a plurality of avoidance scenarios through a reinforcement learning-based model, such as a Proximal Policy Optimization (PPO) algorithm, based on the predicted collision probability, and determines optimal avoidance information by evaluating the reward score, driving stability, degree of pedestrian interference, etc., for each scenario.

[0078] According to one embodiment, in step S50, the vehicle collision prediction system (10) transmits the determined collision avoidance information to the vehicle and surrounding devices (signal controller, pedestrian notification device, etc.) via V2X communication. At this time, control information or a warning message is transmitted to the vehicle, and a priority control command may be transmitted to the signal controller.

[0079] According to one embodiment, the step of predicting the possibility of a vehicle collision may include: a step of recognizing a vehicle by performing CNN-based object recognition on image data at multiple points in time; a step of predicting multiple movement paths of the vehicle by inputting the image data in which the vehicle is recognized in chronological order into an LSTM model; and a step of calculating the possibility of a vehicle collision by integrating the predicted multiple movement paths through a fully connected layer.

[0080] More specifically, to track vehicles approaching an intersection, the RSU can acquire 5 frames of video data at 1-second intervals and perform object recognition using a CNN model based on these video frames.

[0081] According to one embodiment, the step of predicting the possibility of a vehicle collision may include the step of determining the risk of collision between vehicles through an artificial intelligence-based object recognition and movement trajectory prediction model using image data from multiple viewpoints acquired through an image sensor included in a smart RSU installed in road infrastructure.

[0082] According to one embodiment, a vehicle collision prediction system (10) collects five frames of video data at one-second intervals for a single vehicle through a video sensor mounted on a smart RSU, and applies a CNN (Convolutional Neural Network)-based object recognition model to each collected video frame to extract state information such as the position, speed, and direction of the vehicle at each point in time.

[0083] At this time, the extracted vehicle state information is converted into a time-series input arranged in chronological order and input into a Long Short-Term Memory (LSTM) based deep learning model. Based on the input time sequence, the LSTM model generates multiple candidate vehicle movement paths for multiple future time points. For example, in the case of the first vehicle, five candidate movement paths up to 3 seconds later can be predicted.

[0084] According to one embodiment, when the movement trajectories of the first vehicle and the second vehicle are predicted to intersect at the center of the same intersection after 2 seconds and the minimum distance is determined to be 5m or less, the collision probability value calculated by the above mathematical formula is evaluated as 0.81, and since this value exceeds the threshold probability of 0.75, the vehicle collision prediction system determines that a collision is imminent and can then proceed to a step of generating collision avoidance information.

[0085] This series of processes is performed in real time under a digital twin environment, and the predicted collision risk can be used as an input value for generating a subsequent reinforcement learning-based avoidance policy.

[0086] According to one embodiment, the step of generating collision avoidance information may include: acquiring vehicle state information including the vehicle's current position, movement speed, and dynamic information of objects around the vehicle; inputting the vehicle state information into a reinforcement learning algorithm to generate next action information of the vehicle capable of avoiding a collision; calculating a reward score based on the application of the next action information; applying a clipping loss function to a plurality of avoidance policies generated based on the vehicle state information, next action information, and reward score; and determining the avoidance policy with the highest expected reward among the plurality of avoidance policies as the collision avoidance information.

[0087] According to one embodiment, the collision avoidance information is characterized by including first information transmitted to a vehicle, second information transmitted to a signal control device around the vehicle, and third information transmitted to other objects around the vehicle, wherein the first information includes at least one of a warning message notifying the driver of the vehicle or the autonomous driving control system of the vehicle of a collision risk or control information controlling the direction and speed of movement of the vehicle, the second information includes a control command controlling the signal of a signal control device located on the vehicle's movement path according to the possibility of a collision of the vehicle, and the third information includes a collision warning message and recommended avoidance path information transmitted to other vehicles and pedestrians located within a critical radius from the vehicle.

[0088] According to one embodiment, the step of applying a clipping method loss function may include a step of calculating the probability ratio of a new avoidance policy relative to an existing avoidance policy based on vehicle state information, next action information, and a reward score collected for each of a plurality of avoidance policies, and a step of preventing a decrease in stability that occurs during policy update by applying a loss function so that the probability ratio is maintained within a preset range.

[0089] According to one embodiment, policy updates are performed repeatedly in response to a predetermined period or a change in vehicle density on a virtual road, and for each repetition, the parameters of the policy update may be dynamically updated only when the prediction stability score is below a standard, using the real-time number of objects on the virtual road, average vehicle speed, and rate of change of TTC (Time to Collision) as evaluation indicators.

[0090] According to one embodiment, when the probability of a collision is determined to be greater than a threshold probability, a vehicle collision prediction system applies a reinforcement learning-based algorithm to generate optimal action information for avoiding the collision in advance. At this time, vehicle state information, including the vehicle's current state and information about surrounding objects, is used as the state of the reinforcement learning model, and subsequently, an avoidance policy is determined through a series of reward-based policy evaluation and selection processes.

[0091] Specifically, the vehicle collision prediction system according to the present embodiment collects various traffic data, such as, for example, the current position of the first vehicle (x = 120.5m, y = 245.2m), a moving speed of 6.8 m / s, a direction of travel of 82 degrees, a distance to the vehicle ahead and a distance to a pedestrian of 8.3m and 5.6m, respectively, and a state where the traffic light is red. This data is a single state vector S t =[x,y,v,θ,d front ,d ped It consists of ,signal] and is input into the reinforcement learning model.

[0092] According to one embodiment, a reinforcement learning model that receives such state information outputs multiple actions that the vehicle can select based on a Proximal Policy Optimization (PPO) algorithm. For example, deceleration (-3 m / s) as the vehicle's next action. 2 ), maintain lane and then maintain speed, or sudden braking (-6 m / s 2 Candidates for ) are derived, and the selection probabilities for each action are calculated as 0.55, 0.30, and 0.15, respectively.

[0093] Afterwards, a reward score is assigned based on the simulation results when each action is applied. For deceleration, a reward of 0.9 is assigned for successfully avoiding a collision, -1.0 is assigned for maintaining speed when a collision occurs, and for sudden braking, a reward of 0.2 is assigned for avoiding a collision but causing a decrease in ride comfort.

[0094] These multiple action candidates have a probability ratio r that reflects the difference from existing policies. t (θ) is calculated and input into the clipped loss function. For example, if the probability of the existing policy is 0.40 and the new policy is 0.55, the probability ratio is 1.375, and if the clipping coefficient ε = 0.2, the loss function can be defined by Equation 2 below.

[0095]

[0096] Here, is an advantage function, and clipping can prevent a decrease in stability caused by excessive updates when updating policies. After calculating the expected rewards for multiple avoidance policies, the policy with the highest expected reward, such as a deceleration policy, is selected as the final avoidance information. The selected avoidance information can be transmitted to the vehicle control system for immediate application or delivered to the driver as a warning message.

[0097] In addition, the system performs policy updates for the reinforcement learning model at a predetermined interval (e.g., 5 seconds) or according to conditions such as the rate of change in vehicle density, average vehicle speed, and the rate of change in TTC (Time To Collision), and dynamically adjusts parameters only when the prediction stability score for a virtual road environment falls below a threshold value. For example, it can be designed to trigger an update of the reinforcement learning model when the number of vehicle objects is 10 or more, the average speed is 8 m / s or less, and the rate of change in TTC is 40% or more.

[0098] This specific flow is utilized for real-time collision avoidance judgment and control policy optimization, and can dramatically improve the safety and responsiveness of autonomous vehicles in intersection environments with a high probability of collision.

[0099] FIG. 3 is a diagram showing the configuration of a system for predicting vehicle collisions based on a digital twin in a smart RSU according to one embodiment.

[0100] A system (10) for predicting vehicle collisions based on a digital twin in a smart RSU according to one embodiment of the present invention may include a communication unit (100) that communicates with vehicles, terminals, and external servers around the smart RSU using V2X communication, a sensor unit (200) that senses traffic data around the smart RSU, and an edge computing device (400) that simulates real-time traffic data from the sensor unit (200) on a digital twin to recreate a virtual road, inputs the digital twin simulation results into an artificial intelligence model to predict the probability of a vehicle collision on the virtual road, generates collision avoidance information through a reinforcement learning algorithm (PPO) based on the probability of a vehicle collision, and transmits the collision avoidance information to vehicles and other devices using V2X communication.

[0101] According to one embodiment, the communication unit (100) communicates with the vehicle using a V2X communication service when the vehicle is located in a V2X communication service area, and communicates with the vehicle using a mobile communication service when the vehicle is not located in a V2X communication service area, and is characterized by continuously attempting communication using the V2X communication service.

[0102] According to one embodiment, the communication unit (100) is a device that performs V2X communication between a vehicle and an RSU, and transmits and receives data with surrounding vehicles and pedestrians using various communication methods including V2V (Vehicle-to-Vehicle), V2I (Vehicle-to-Infrastructure), and V2P (Vehicle-to-Pedestrian). In addition, mobile communication-based I2V (Infrastructure-to-Vehicle) communication is also possible when necessary, and continuous connectivity with the vehicle is maintained through continuous communication attempts even when out of the V2X communication area.

[0103] According to one embodiment, the sensor unit (200) is composed of various sensors such as LIDAR, RADAR, CCTV, weather sensors, and GNSS (Global Navigation Satellite System), and detects road conditions, vehicle position and speed, pedestrian movement, and weather conditions in real time. In particular, LIDAR precisely detects the relative positions of vehicles and pedestrians through 3D-based spatial recognition, RADAR detects distance and speed, and CCTV performs traffic situation video and object recognition.

[0104] According to one embodiment, the memory (300) temporarily stores large-scale traffic data collected in real time or stores historical information so that the edge computing module can perform prediction and learning based on it. For example, the past movement trajectory of a vehicle, the time of entry into an intersection, collision history, etc., may be stored.

[0105] According to one embodiment, an edge computing module (400) is a core component of the present invention and processes traffic data in real time within an RSU local device and performs artificial intelligence-based collision prediction and avoidance judgment. This module may include a prediction model of a CNN-LSTM structure and a PPO-based reinforcement learning algorithm, and has the ability to determine the probability of a collision of a vehicle entering an intersection, calculate an avoidance policy, and transmit it to the vehicle or surrounding vehicles.

[0106] In particular, the edge computing module (400) enables ultra-low latency responses by analyzing data directly locally without transmitting data to a central cloud. Unlike conventional cloud-based ITS, this structure is designed to prevent collision accidents in high-risk areas, such as intersections, in real time through immediate judgment and warning transmission in milliseconds.

[0107] According to one embodiment, the configuration of such a smart RSU distributes functions by layer, and information collected from sensors is immediately processed by an edge computing module and transmitted to vehicles and infrastructure via a communication unit, enabling traffic flow optimization and accident prevention. The analysis and prediction results of real-time data can be stored in memory and utilized for post-learning and policy optimization.

[0108] Therefore, it can be seen that the structure of Fig. 3 is a core device for implementing the technical essence of the present invention and constitutes the basic hardware architecture of the edge-based real-time collision prediction and avoidance information generation system of the smart RSU.

[0109] FIG. 4 is a diagram specifically illustrating the architecture of a system for predicting vehicle collisions based on a digital twin in a smart RSU according to one embodiment.

[0110] According to one embodiment, the vehicle collision prediction system (10) is composed of a hardware layer, an edge computing-based middleware layer, and a V2X-linked application layer, and predicts the possibility of a collision through a deep learning model based on vehicle driving data at multiple points in time, and can provide an avoidance path to an autonomous vehicle or traffic control system.

[0111] According to one embodiment, the hardware layer is configured around a smart RSU installed in the road infrastructure and includes various sensors such as LIDAR, LADAR, CCTV, weather sensor, GNSS, and V2X communication device (V2X RSU). Traffic information, the state of road objects, environmental conditions, etc. collected from these sensors are input in real time to an edge computing device.

[0112] According to one embodiment, input data is processed in real time through an Edge Computing structure in the middleware layer, specifically, frame-by-frame time-series image information is analyzed by a CNN-LSTM deep learning structure. N input frames (t₁, t₂, t₃, etc.) are each converted into feature vectors of vehicle objects and surrounding objects through a CNN model, and then input into an LSTM cell in chronological order to predict the movement trajectory and dynamic changes of each object in a time-series manner.

[0113] According to one embodiment, state vectors for each time step generated through CNN and LSTM are integrated through a Fully Connected Layer, and finally, the probability of a collision between vehicles is numerically calculated in a Collision Prediction module. At this time, the predicted collision probability value can be utilized in the previously described TTI / TTC-based model or a reinforcement learning-based policy decision algorithm.

[0114] According to one embodiment, the structure of FIG. 4 is also linked with an upper Application layer and can be utilized in various service applications such as autonomous driving, real-time traffic flow optimization, and emergency response systems. This application is connected via V2X communication in the form of V2V (Vehicle-to-Vehicle), V2I (Vehicle-to-Infrastructure), and I2V (Infrastructure-to-Vehicle), enabling the sharing of collision avoidance information between traffic participants.

[0115] Accordingly, FIG. 4 is a diagram that intuitively shows the overall system architecture configured so that the collision avoidance device (10) according to the present invention predicts and analyzes real-time traffic conditions and enables autonomous avoidance control based on this.

[0116] FIG. 5 is a diagram showing a series of flows for generating collision avoidance information according to one embodiment.

[0117] According to one embodiment, a vehicle collision prediction system (10) can perform collision prediction and avoidance judgment based on data flow and division of roles among various elements such as a cloud, V2X RSU, edge computing device, road sensor, weather sensor, traffic signal controller, V2X OBU.

[0118] According to one embodiment, the vehicle collision prediction system (10) collects road condition data and weather data from road sensors and weather sensors to an edge computing device. The collected information is interpreted as status information of objects on the road, such as vehicles, pedestrians, and obstacles, during the object recognition (Object analysis) process. At the same time, real-time traffic light information is also collected from a traffic signal controller.

[0119] According to one embodiment, the vehicle collision prediction system (10) performs V2I reception data analysis on the edge computing device based on collected object information and V2I reception data. This includes state information such as the position, speed, and direction of travel of each vehicle, and performs a collision simulation based on this. This simulation predicts the trajectory of the vehicle on a digital twin and calculates whether there is mutual intersection or the Time-To-Collision (TTC) to derive a collision prediction result.

[0120] According to one embodiment, when the collision prediction result of the vehicle collision prediction system (10) is above a certain threshold, the edge computing device executes signal control judgment logic, and if the condition "when the probability of a collision at an intersection is above a certain value" is met, signal change information is generated. This information is transmitted to a traffic signal controller, and an actual signal change is performed. In addition, traffic information received from surrounding RSUs is integrated (RSU information merged) to enable signal control optimization based on cooperation among multiple RSUs.

[0121] According to one embodiment, the vehicle collision prediction system (10) generates collision prevention safety information in an Infrastructure-to-Vehicle (I2V) manner based on changed signal information and integrated RSU traffic information, and transmits a collision warning message to nearby vehicle and pedestrian terminals (V2X OBU). This message is delivered to the autonomous driving control system or driver of a vehicle within the danger radius to enable immediate avoidance action, and can also be linked to an Emergency Response System (ERS).

[0122] According to one embodiment, the vehicle collision prediction system (10) performs collision simulation and judgment logic within an edge computing device based on various traffic data collected over time, and clearly demonstrates that it implements a field-oriented active safety control system by transmitting the results to vehicles, pedestrians, and traffic control systems in various ways.

[0123] FIG. 6 is a diagram illustrating the detailed configuration of an artificial intelligence model learning unit included in an edge computing module according to one embodiment.

[0124] According to one embodiment, the vehicle collision prediction system (10) can acquire noise data, vibration data, and combustion pressure measurement data obtained using sensors included in the actual vehicle in addition to the smart RSU.

[0125] According to one embodiment, the vehicle collision prediction system (10) can diagnose the driving state of the vehicle by learning information about noise data, vibration data, combustion pressure measurement data and the purpose of vehicle operation using a first artificial intelligence learning model.

[0126] According to one embodiment, the vehicle collision prediction system (10) can determine the driving mode of the vehicle in real time based on the driving state of the vehicle. In addition, based on the determined driving mode of the vehicle, vehicle control information can be generated and transmitted to the vehicle.

[0127] According to one embodiment, the second artificial intelligence learning model refers to a deep learning model trained to analyze preprocessed sensor data to generate control data and a driving path for autonomous driving, or a processor that executes the same.

[0128] According to one embodiment, the data learning unit (401) can learn noise data, vibration data, combustion pressure measurement data, and information regarding the purpose of vehicle operation. Specifically, the data learning unit (401) can learn criteria for diagnosing the driving state of a vehicle from the noise data, vibration data, combustion pressure measurement data, and information regarding the purpose of vehicle operation. In addition, the data learning unit (401) can learn control information according to the driving state of a vehicle based on the aspects of the driving state of the vehicle. That is, the data learning unit (401) can generate, learn, or update new vehicle control information based on the accumulated driving state of the vehicle.

[0129] According to one embodiment, the vehicle control unit (800) can determine the driving mode of the vehicle by analyzing the driving state of the vehicle according to learned criteria. The vehicle control unit (800) can also extract data necessary for learning from the vehicle's driving information, state information, and surrounding road environment information. From this, the data learning unit (401) can generate and update a vehicle control information generation model.

[0130] According to one embodiment, the control information generation unit (402) can recognize a situation from predetermined data by using a learned vehicle control generation model. The control information generation unit (402) can acquire predetermined data according to a preset standard based on learning and use a data recognition model with the acquired data as an input value. For example, by using a learned context model, etc., it can analyze vehicle state information, driving history, destination information, vehicle operation purpose information, traffic information, previously stored map information, vehicle surrounding environment information, etc., and extract data for generating a driving path. In addition, by using the learned model, the control information generation unit (402) can acquire information about the current surrounding situation from an external server and generate the most suitable driving mode for the vehicle. Here, the driving mode may include a driving time reduction mode, a stable driving mode, an obstacle detection mode, a driving mode due to vehicle defects, etc., and the number of modes is not limited.

[0131] According to one embodiment, at least a portion of the data learning unit (401) and at least a portion of the control information generation unit (402) may be implemented as a software module or manufactured in the form of at least one hardware chip and mounted on an electronic device. For example, at least one of the data learning unit (401) and the control information generation unit (402) may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as part of an existing general-purpose processor (e.g., CPU or application processor) or a graphics-dedicated processor (e.g., GPU) and mounted on the aforementioned various electronic devices or content data playback devices. In this case, the dedicated hardware chip for artificial intelligence is a dedicated processor specialized in probabilistic computation, and has higher parallel processing performance than an existing general-purpose processor, so it can quickly process computational tasks in the field of artificial intelligence, such as machine learning.

[0132] According to one embodiment, when the data learning unit (401) and the control information generation unit (402) are implemented as a software module (or a program module including instructions), the software module may be stored on a non-transitory computer-readable media. In this case, the software module may be provided by an operating system (OS) or by a specific application. Alternatively, some of the software modules may be provided by an operating system (OS), and the remaining parts may be provided by a specific application.

[0133] In addition, the computer program may provide a computer program stored on a recording medium that performs all steps of the method provided in the present invention.

[0134] The steps of the method or algorithm described in connection with embodiments of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, a hard disk, a removable disk, a CD-ROM, or any form of computer-readable recording medium well known in the art to which the present invention belongs.

[0135] The components of the present invention may be implemented as a program (or application) and stored on a medium to be executed in combination with a computer, which is hardware. The components of the present invention may be implemented as software programming or software elements, and similarly, embodiments may be implemented in programming or scripting languages ​​such as C, C++, Java, assembler, etc., including various algorithms implemented as combinations of data structures, processes, routines, or other programming configurations. Functional aspects may be implemented as algorithms executed on one or more processors.

[0136] Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. Industrial applicability

[0138] The present invention relates to a method and system for predicting vehicle collisions based on a digital twin in a smart RSU, and is applicable to fields related to smart RSUs or vehicle collision avoidance systems.

[0139] No content

Claims

Claim 1 A step of acquiring traffic data in real time from a smart RSU installed on the road; a step of reproducing a virtual road by performing a digital twin simulation on the acquired traffic data; a step of predicting the probability of a vehicle collision on the virtual road by inputting the results of the digital twin simulation into an artificial intelligence model; and a step of generating collision avoidance information through a reinforcement learning algorithm (PPO) based on the probability of a vehicle collision. The method includes the step of transmitting collision avoidance information to the vehicle and other devices using V2X communication; and the step of reproducing the virtual road comprises: the step of extracting a first vehicle and state information of the first vehicle from the traffic data; the step of creating a first vehicle object and a second vehicle object that can be simulated along the movement path of the first vehicle and the second vehicle using the state information, and then creating a pedestrian object and an intersection object; the step of simulating the relative positional relationship and state change at each point in time between the first vehicle object, the second vehicle object, the pedestrian object, and the intersection object and displaying them on a digital twin; and the step of generating a plurality of avoidance scenarios capable of avoiding a collision between the first vehicle object and the second vehicle object when the probability of a collision is greater than or equal to a critical probability at the time when the first vehicle object and the second vehicle object enter the intersection object. A method for predicting vehicle collisions based on a digital twin in a smart RSU, comprising: a step of evaluating the collision avoidance success rate, vehicle operation stability, and the degree of interference with the pedestrian object for each of the plurality of avoidance scenarios; and a step of determining the priority of the plurality of avoidance scenarios according to the evaluation results. Claim 2 A method for predicting vehicle collisions based on a digital twin in a smart RSU, characterized in that, in claim 1, the step of reproducing the virtual road comprises: a step of generating a virtual scenario regarding the surrounding conditions of the road and intersection using prior collected traffic data; a step of training a collision prediction model based on the generated virtual scenario; a step of dynamically updating the environment of the virtual road on the digital twin using traffic data obtained from the smart RSU; and a step of updating the collision prediction model by comparing the probability of a vehicle collision obtained from the artificial intelligence model with the actual collision result. Claim 3 A method for predicting a vehicle collision based on a digital twin in a smart RSU, characterized in that, in claim 1, the step of predicting the probability of a vehicle collision comprises: a step of recognizing the vehicle by performing CNN-based object recognition on image data at multiple time points; a step of predicting multiple movement paths of the vehicle by inputting the image data in which the vehicle is recognized in chronological order into an LSTM model; and a step of calculating the probability of a collision of the vehicle by integrating the predicted multiple movement paths through a fully connected layer. Claim 4 A method for predicting vehicle collisions based on a digital twin in a smart RSU, characterized in that, in claim 1, the step of generating collision avoidance information comprises: a step of obtaining vehicle state information including the current position, movement speed, and dynamic information of objects surrounding the vehicle; a step of inputting the vehicle state information into a reinforcement learning algorithm to generate next action information of the vehicle capable of avoiding a collision; a step of calculating a reward score based on the application of the next action information; a step of applying a clipping-type loss function to a plurality of avoidance policies generated based on the vehicle state information, the next action information, and the reward score; and a step of determining the avoidance policy with the highest expected reward among the plurality of avoidance policies as the collision avoidance information. Claim 5 A V2X communication unit that communicates with vehicles, terminals, and external servers around the smart RSU using V2X communication; a sensing unit that senses traffic data around the smart RSU; The system includes an edge computing module that reproduces a virtual road by simulating real-time traffic data from the sensing unit on a digital twin, inputs the results of the digital twin simulation into an artificial intelligence model to predict the probability of a vehicle collision on the virtual road, generates collision avoidance information through a reinforcement learning algorithm (PPO) based on the probability of a vehicle collision, and transmits the collision avoidance information to the vehicle and other devices using V2X communication; wherein the computing module extracts a first vehicle and state information of the first vehicle from the traffic data, creates a first vehicle object and a second vehicle object capable of simulating along the movement path of the first vehicle and the second vehicle using the state information, creates a pedestrian object and an intersection object, simulates the relative positional relationship and state change at each point in time between the first vehicle object, the second vehicle object, the pedestrian object, and the intersection object and displays them on the digital twin, and if the probability of a collision is greater than or equal to a threshold probability at the time when the first vehicle object and the second vehicle object enter the intersection object, a plurality of methods capable of avoiding a collision between the first vehicle object and the second vehicle object A system for predicting vehicle collisions based on a digital twin in a smart RSU, comprising the steps of generating avoidance scenarios and evaluating the collision avoidance success rate, vehicle driving stability, and degree of interference with the pedestrian object for each of the plurality of avoidance scenarios, and determining the priority of the plurality of avoidance scenarios according to the evaluation results.

Images