A highway risk simulation and avoidance system and method based on multimodal IoT perception

Abstract

The present invention discloses a highway risk simulation and avoidance system and method based on multimodal IoT perception, belonging to the field of intelligent transportation technology. The method includes the following steps: real-time collection of road condition information and road section characteristic parameters of the current road section; screening multiple monitoring vehicles; extracting historical driving behavior data and road condition characteristics of multiple monitoring vehicles; constructing a historical driving behavior model and a scenario model; extracting historical traffic road condition information and extracting static feature data through the scenario model; constructing a spatial feature matrix and a spatiotemporal feature matrix; performing feature weighted fusion on the spatial feature matrix and the spatiotemporal feature matrix to obtain a multi-model fusion evaluation result and calculate the vehicle collision probability; prompting the driver to adjust the steering to avoid risks, and activating the auxiliary steering avoidance function through the intelligent risk avoidance module. The present invention solves the problems that traditional monitoring functions cannot achieve risk modeling warning and risk avoidance, and lack the real-time prediction function of multiple vehicle collisions.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent transportation technology, and specifically relates to a highway risk simulation and avoidance system and method based on multimodal Internet of Things perception. Background Art

[0002] Traffic warning systems offer more efficient and intelligent solutions for urban traffic management. By providing traffic warnings and route optimization services, they facilitate travel, help avoid traffic congestion, and improve urban traffic efficiency. They also provide road monitoring capabilities, monitoring traffic conditions in real time to identify potential hazards, such as pedestrians crossing zebra crossings, and alert drivers, thereby reducing the risk of traffic accidents.

[0003] However, traditional monitoring functions cannot achieve risk modeling, early warning and risk avoidance, and lack the function of real-time prediction of multi-vehicle collisions. Therefore, a real-time monitoring system is needed to monitor multi-vehicle traffic data, environmental information and user behavior patterns, model, predict and warn of potential traffic accident risks in areas with obstructed vision or complex road conditions, and provide avoidance solutions to further reduce the occurrence of traffic accidents. Summary of the Invention

[0004] The present invention provides a highway risk simulation and avoidance system and method based on multimodal IoT perception, which solves the problems that traditional monitoring functions cannot achieve risk modeling warning and risk avoidance, and lack of real-time prediction function for multiple vehicle collisions.

[0005] To solve the above technical problems, the technical solution of the present invention is: a highway risk simulation and avoidance system based on multimodal IoT perception, comprising an IoT cloud platform and a smart transportation subsystem; the IoT cloud platform comprises IoT devices and a cloud platform server; the IoT cloud platform connects IoT device information via the Internet; and the smart transportation subsystem is connected to the IoT cloud platform via the Internet.

[0006] The cloud platform server is used to monitor and store road condition information and predict the probability of vehicle collision based on the road condition information; the IoT device is used to view the road condition information monitored by the cloud platform server;

[0007] The intelligent traffic subsystem is used for emergency command and activates the assisted steering avoidance function based on the vehicle collision probability predicted by the Internet of Things cloud platform.

[0008] Furthermore, the cloud platform server includes a data monitoring and acquisition module, a database, a data analysis and processing module, and a monitoring and early warning module;

[0009] The IoT devices include mobile phones, computers, and tablets, which are used to display road condition information and vehicle collision probability, and remind users to avoid risks;

[0010] The intelligent transportation subsystem includes an emergency command module and an intelligent risk avoidance module;

[0011] The data monitoring and collection module is used to collect road condition information;

[0012] The database is used to store the road condition information collected by the data monitoring and collection module;

[0013] The data analysis and processing module is used to perform data analysis and processing on the road condition information in the database;

[0014] The monitoring and warning module is used to predict the probability of vehicle collision by analyzing and processing the data;

[0015] The emergency command module is used for decision-making assistance and safety emergency response;

[0016] The intelligent risk avoidance module is used to activate the auxiliary steering avoidance function according to the vehicle collision probability.

[0017] The beneficial effects of the present invention are: (1) the driving path of high-risk areas is predicted in real time through the monitoring and early warning module, and modeling is performed based on building information and vehicle information to predict the collision probability based on the driving paths of multiple vehicles. The prediction probability is more accurate and has real-time performance;

[0018] (2) Process information data through the IoT cloud platform and use IoT devices, such as smart terminal apps, to push and display traffic information;

[0019] (3) Through the emergency command module, the traffic environment is monitored and emergency response is provided for the smooth operation of vehicles through big data and traffic development analysis. The monitoring can also be used to command the vehicle. At the same time, the intelligent risk avoidance module can prompt users to avoid risks. It has a certain practical guidance effect. When driving in a high-risk traffic environment and encountering danger, there is still no reaction or no time to think about how to avoid it, which can greatly reduce the probability of accidents.

[0020] The present invention also provides a road risk simulation and avoidance method based on multimodal IoT perception, comprising the following steps:

[0021] S1, collect the road condition information and road characteristic parameters of the current road section in real time through the data monitoring and acquisition module, and store them in the database;

[0022] S2. Selecting a vehicle adaptive area based on the road condition information in the database through the data analysis and processing module to select multiple monitoring vehicles;

[0023] S3, extracting historical driving behavior data and road condition characteristics of multiple monitored vehicles;

[0024] S4. Based on historical driving behavior data and road condition characteristics, a historical driving behavior model is constructed through the monitoring and warning module, and a scenario model is constructed according to the road section characteristic parameters;

[0025] S5. Extract historical traffic condition information through historical driving behavior data, and extract static feature data through the scenario model;

[0026] S6. Constructing a spatial feature matrix based on the static feature data, and constructing a spatiotemporal feature matrix based on historical traffic condition information;

[0027] S7. Perform feature weighted fusion on the spatial feature matrix and the spatiotemporal feature matrix to obtain a multi-model fusion evaluation result, and obtain the vehicle collision probability based on the multi-model fusion evaluation result;

[0028] S8. Send the vehicle collision probability to the IoT device and the smart transportation subsystem, prompting the driver to adjust the steering to avoid danger, and activate the assisted steering avoidance function through the intelligent risk avoidance module.

[0029] The beneficial effects of the present invention are: the present invention can predict the driving paths in high-risk areas in real time. Traditional early warning methods can only give the probability of risk occurrence in combination with historical data. At the same time, modeling is carried out based on building information and vehicle information to predict the collision probability of multiple vehicle driving paths. The predicted probability is more accurate and real-time.

[0030] Furthermore, the specific steps of S2 are:

[0031] S21, analyzing the road condition information in the database through the data analysis and processing module to extract multiple vehicle information;

[0032] S22, grouping the multiple vehicle information into pairs, taking the front and rear vehicles of the pairwise group as observation samples, and determining whether the front and rear vehicles are in the same lane and traveling in the same direction. If so, proceed to S24; otherwise, proceed to S23;

[0033] S23: Determine whether the two vehicles are in different lanes and whether the difference in their driving direction angles is within 180°. If so, proceed to S24; otherwise, remove the two vehicles from observation samples.

[0034] S24. Determine whether the two vehicles collide within 5 seconds. If so, mark the two vehicles as monitoring vehicles. Otherwise, remove the two vehicles as observation samples.

[0035] The beneficial effect of the above further solution is: screening the vehicles at risk at the same time on the current road section. The real-time computing amount will have a significant performance impact on the server. Therefore, by adaptively selecting areas and screening the monitored vehicles, the pressure on the existing hardware can be alleviated.

[0036] Furthermore, the formula for determining whether the front and rear vehicles collide within 5 seconds in S24 is:

[0037]

[0038]

[0039]

[0040] in, Indicates time Displacement of the rear vehicle, represents the initial position of the following vehicle, represents the initial speed of the following vehicle, represents the acceleration of the following vehicle, Indicates time The displacement of the front vehicle, represents the initial position of the preceding vehicle, represents the initial speed of the preceding vehicle, represents the acceleration of the preceding vehicle, Indicates time, take 5s.

[0041] Furthermore, the road section characteristic parameters include at least one of the road section length, road section width, number of road section lanes and road section type of each road section in the road section set; the road section characteristic parameters also include environmental characteristic data of the road section.

[0042] Furthermore, the elements of the spatial feature matrix in S6 are ,in, , Indicates the total number of target road segments and neighboring road segments, represents a real number, Represents static feature data, Indicates the road section number. Represents the static characteristic data of the road section;

[0043] The x-axis of the spatial feature matrix represents the target road segment and the neighboring road segments, and the z-axis of the spatial feature matrix represents static feature data.

[0044] Furthermore, the spatiotemporal feature matrix in S6 is constructed by the historical traffic condition information after dividing the window;

[0045] The elements in the spatiotemporal feature matrix are ,in, , Indicates the number of historical time windows obtained by dividing the historical time period according to the preset time interval. Indicates historical traffic information. Represents the partitioned time matrix;

[0046] The x-axis of the spatiotemporal feature matrix represents the target road segment and the neighboring road segments, the y-axis of the spatiotemporal feature matrix represents the historical time window, and the z-axis of the spatiotemporal feature matrix represents the historical road condition information.

[0047] The beneficial effect of the above further solution is that by obtaining the spatiotemporal characteristics of historical road conditions, the influence of multiple factors on the real-time prediction of multi-vehicle collisions is taken into account, making the prediction results more accurate.

[0048] Furthermore, the calculation formula for the weighted feature fusion in S7 is:

[0049]

[0050]

[0051]

[0052] in, represents the multi-model fusion evaluation result, i.e., the vehicle collision prediction probability including road construction information, represents the matrix tensor product, represents the matrix XOR operation, represents the activation function, represents the predicted probability of multi-vehicle two-dimensional route intersection, represents the relative bias probability of the preceding vehicle, represents the relative bias probability of the following car, represents the bias probability of the multi-vehicle ordinate matrix, represents the multi-vehicle longitudinal displacement matrix, represents the bias probability of the multi-vehicle horizontal coordinate matrix, Represents the multi-vehicle horizontal coordinate displacement matrix.

[0053] The beneficial effect of the above further solution is that the road section characteristic parameters and the spatiotemporal characteristic matrix are fused to obtain a multimodal evaluation model, which can further calibrate the road construction information and make the multi-model fusion evaluation results more accurate.

[0054] Furthermore, the calculation formula for the collision probability of two-vehicle two-dimensional displacement collision in the multi-vehicle two-dimensional route intersection prediction probability is:

[0055]

[0056]

[0057]

[0058] in, , They represent the horizontal and vertical coordinates of vehicle 1 and vehicle 2 before displacement, and the positive direction of the y-axis is the forward direction to the left, and the negative direction of the y-axis is the right direction. represents the horizontal coordinate deviation of the two vehicles, represents the deviation of the longitudinal coordinates of the two vehicles, represents the two-dimensional horizontal coordinate of the predicted collision point of vehicle 1, represents the two-dimensional ordinate of the predicted collision point of vehicle 1, represents the turning radius of vehicle 1 before the collision, represents the turning radius of vehicle 1 when it is predicted to collide, represents the turning radius of vehicle 2 before collision, represents the turning radius of vehicle 2 when it is predicted to collide, represents the moving deflection angle of vehicle 1 before the collision, represents the moving deflection angle of vehicle 2 before the collision, represents the moving deflection angle of vehicle 1 when it is predicted to collide, represents the moving deflection angle of vehicle 2 when it is predicted to collide, represents the two-dimensional horizontal coordinate of the predicted collision point of vehicle 2, The two-dimensional vertical coordinate of the predicted collision point of vehicle 2.

[0059] in The derivation process is Move to The prediction is as follows: Depend on The displacement is obtained, and so on; the schematic diagram is as follows Figure 4 As shown:

[0060] 1. Modeling based on the ground coordinate system: The calculation of the vehicle's motion trajectory is based on the ground coordinate system OXY;

[0061] Define vehicle The location is , Indicates the vehicle's moving deflection angle before the collision. The vehicle position at the moment is , Indicates the vehicle's predicted moving deviation angle when it collides;

[0062] The turning radius of the vehicle is defined as R, with the positive direction of the y-axis as the forward direction to the left and the negative direction of the y-axis as the rightward direction; the center point of rotation of the vehicle is defined as If the vehicle is moving to the left, the center of rotation falls on the left side of the vehicle body; if the vehicle is moving to the right, the center of rotation falls on the right side of the vehicle body; define The projections on the X and Y axes are and , The projections on the X and Y axes are and Assume that the vehicle is The velocity inside is V, a positive value represents forward movement, and a negative value represents backward movement. The vehicle can be regarded as moving at a constant speed in a short period of time.

[0063] 2. Angle relationship: Assuming the vehicle is moving right forward, V is a positive value and R is a negative value

[0064]

[0065]

[0066] 3. Calculate the projection: Considering the above assumption that the vehicle is traveling to the right and R is a negative value, a “-” sign must be added to represent the projection distance value. Projection on the X and Y axes:

[0067]

[0068] calculate Projection on the X and Y axes:

[0069]

[0070] 4 Calculate vehicle position: Substitute the above process into the following formula:

[0071]

[0072]

[0073] 5. Consider the case where the turning radius at two adjacent moments is different: Assume that the turning radius at two adjacent moments is and ,calculate Projection on the X and Y axes:

[0074]

[0075] calculate Projection on the X and Y axes:

[0076]

[0077] According to the geometric relationship, we get the following formula:

[0078]

[0079]

[0080] but The deduction can be obtained by the same logic.

[0081] Furthermore, the turning radius derivation process of two vehicles in the multi-vehicle two-dimensional route intersection prediction probability is as follows:

[0082]

[0083]

[0084]

[0085]

[0086] in, represents the turning radius of vehicle 1 before the collision, represents the turning radius of vehicle 1 when it is predicted to collide, represents the turning radius of vehicle 2 before collision, represents the turning radius of vehicle 2 when it is predicted to collide, represents the moving deflection angle of vehicle 1 before the collision, represents the moving deflection angle of vehicle 2 before the collision, represents the moving deflection angle of vehicle 1 when it is predicted to collide, Indicates the predicted moving deviation angle of vehicle 2 when it collides. The turning radius refers to the distance from the steering center to the point where the front outer steering wheel contacts the ground during the vehicle's driving process. It is calculated by dividing the distance from the center of the vehicle's front wheel axle to the center of the rear wheel axle by the turning angle (in radians).

[0087] The derivation process of R=L / sin(θ) is as follows: Figure 5 As shown:

[0088] 1. Set the coordinate system definition parameters: Set a coordinate system so that the initial driving direction of the car coincides with the X-axis; L represents the wheelbase of the car, that is, the distance from the center of the front axle to the center of the rear axle; θ represents the steering angle of the front wheels, which usually refers to the maximum turning angle of the front wheels relative to the vehicle's forward direction; R represents the turning radius, that is, the radius of the circle formed by the vehicle when turning.

[0089] 2. Define the steering angle: When the car starts to turn, the front wheels will rotate around a center of a circle; let the steering angle of the front wheels be θ, which is the angle between the front wheels and the straight direction of the vehicle.

[0090] 3. Construct a triangle: In the geometric model of car steering, the wheelbase L of the car forms one right-angled side of a right triangle, and the turning radius R is the hypotenuse of this triangle.

[0091] 4. Use trigonometric functions: In a right triangle, the tangent function sin is defined as the opposite side / hypotenuse; in this case, the opposite side is the car's wheelbase L, and the adjacent side is the perpendicular component of the radius R (that is, the distance from the center of the circle to the car's rear wheels).

[0092] 5. Derivation of the formula: According to the definition of the tangent function, we can obtain sin(θ)=opposite side / hypotenuse=L / R.

[0093] 6. Solve for the radius: Solve the above equation to find R, R = L / sin (θ); where the turning radius R is defined as the vehicle's minimum turning radius, the steering angle θ is the vehicle's maximum steering angle, and L is defined as the distance from the center of the vehicle's front axle to the center of the rear axle.

[0094] The beneficial effect of the above further scheme is: by calculating the displacement deviation of the two vehicles in the two-dimensional plane, the probability of collision between the two vehicles can be inferred, thereby obtaining the predicted probability of intersection of the horizontal and vertical coordinates of the two-dimensional routes of multiple vehicles, and realizing real-time prediction of multi-vehicle collision. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] Figure 1 This is a flowchart of the highway risk simulation and avoidance system based on multimodal IoT perception of the present invention.

[0096] Figure 2 This is a flow chart of the highway risk simulation and avoidance method based on multimodal IoT perception of the present invention.

[0097] Figure 3 This is a schematic diagram of vehicle collision prediction according to the present invention.

[0098] Figure 4 This is a schematic diagram of vehicle displacement position prediction according to the present invention.

[0099] Figure 5 Schematic diagram of vehicle turning radius calculation according to the present invention. DETAILED DESCRIPTION

[0100] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the present invention.

[0101] Example 1

[0102] like Figure 1 As shown, the present invention provides a highway risk simulation and avoidance system based on multimodal IoT perception, including an IoT cloud platform and a smart transportation subsystem; the IoT cloud platform includes IoT devices and a cloud platform server; the IoT cloud platform connects IoT device information via the Internet; the smart transportation subsystem is connected to the IoT cloud platform via the Internet;

[0103] The cloud platform server is used to monitor and store road condition information and predict the probability of vehicle collision based on the road condition information; the IoT device is used to view the road condition information monitored by the cloud platform server;

[0104] The intelligent traffic subsystem is used for emergency command and activates the assisted steering avoidance function based on the vehicle collision probability predicted by the Internet of Things cloud platform.

[0105] The cloud platform server includes a data monitoring and acquisition module, a database, a data analysis and processing module, and a monitoring and early warning module;

[0106] The IoT devices include mobile phones, computers, and tablets, which are used to display road condition information and vehicle collision probability, and remind users to avoid risks;

[0107] The intelligent transportation subsystem includes an emergency command module and an intelligent risk avoidance module;

[0108] The data monitoring and collection module is used to collect road condition information;

[0109] The database is used to store the road condition information collected by the data monitoring and collection module;

[0110] The data analysis and processing module is used to perform data analysis and processing on the road condition information in the database;

[0111] The monitoring and warning module is used to predict the probability of vehicle collision by analyzing and processing the data;

[0112] The emergency command module is used for decision-making assistance and safety emergency response;

[0113] The intelligent risk avoidance module is used to activate the auxiliary steering avoidance function according to the vehicle collision probability.

[0114] In this first embodiment, the data monitoring and acquisition module collects road condition information, including speed, acceleration, direction, lane markings, traffic signs, and the location of surrounding objects. This information is then stored in a database. The data analysis and processing module pre-processes the information and screens monitored vehicles. The monitoring and early warning module then constructs multiple models to calculate the probability of vehicle collision. The emergency command module provides decision support and safety emergency response. IoT devices can access road condition information at any time. When the collision probability reaches a set threshold, the driver is notified via a mobile phone or other device, and the intelligent risk avoidance module performs risk avoidance. The intelligent risk avoidance module comprises an adaptive cruise control (ACC), an advanced driver assistance system that not only maintains a set speed but also adjusts its own speed based on the dynamics of the preceding vehicle to maintain a safe distance from it.

[0115] Example 2

[0116] like Figure 2 As shown, the present invention also provides a road risk simulation avoidance method based on multimodal IoT perception, comprising the following steps:

[0117] S1, collect the road condition information and road characteristic parameters of the current road section in real time through the data monitoring and acquisition module, and store them in the database;

[0118] S2. Selecting a vehicle adaptive area based on the road condition information in the database through the data analysis and processing module to select multiple monitoring vehicles;

[0119] S3, extracting historical driving behavior data and road condition characteristics of multiple monitored vehicles;

[0120] S4. Based on historical driving behavior data and road condition characteristics, a historical driving behavior model is constructed through the monitoring and warning module, and a scenario model is constructed according to the road section characteristic parameters;

[0121] S5. Extract historical traffic condition information through historical driving behavior data, and extract static feature data through the scenario model;

[0122] S6. Constructing a spatial feature matrix based on the static feature data, and constructing a spatiotemporal feature matrix based on historical traffic condition information;

[0123] S7. Perform feature weighted fusion on the spatial feature matrix and the spatiotemporal feature matrix to obtain a multi-model fusion evaluation result, and obtain the vehicle collision probability based on the multi-model fusion evaluation result;

[0124] S8. Send the vehicle collision probability to the IoT device and the smart transportation subsystem, prompting the driver to adjust the steering to avoid danger, and activate the assisted steering avoidance function through the intelligent risk avoidance module.

[0125] The specific steps of S2 are:

[0126] S21, analyzing the road condition information in the database through the data analysis and processing module to extract multiple vehicle information;

[0127] S22, grouping the multiple vehicle information into pairs, taking the front and rear vehicles of the pairwise group as observation samples, and determining whether the front and rear vehicles are in the same lane and traveling in the same direction. If so, proceed to S24; otherwise, proceed to S23;

[0128] S23: Determine whether the two vehicles are in different lanes and whether the difference in their driving direction angles is within 180°. If so, proceed to S24; otherwise, remove the two vehicles from observation samples.

[0129] S24. Determine whether the two vehicles collide within 5 seconds. If so, mark the two vehicles as monitoring vehicles. Otherwise, remove the two vehicles as observation samples.

[0130] The formula for determining whether the front and rear vehicles collide within 5 seconds in S24 is:

[0131]

[0132]

[0133]

[0134] in, Indicates time Displacement of the rear vehicle, represents the initial position of the following vehicle, represents the initial speed of the following vehicle, represents the acceleration of the following vehicle, Indicates time The displacement of the front vehicle, represents the initial position of the preceding vehicle, represents the initial speed of the preceding vehicle, represents the acceleration of the preceding vehicle, Indicates time, take 5s.

[0135] In this embodiment 2, the monitoring and warning module predicts the current vehicle's driving path and predicts whether the vehicle's path in the next 5 seconds will collide. Therefore, it is necessary to monitor multiple vehicles on the same road section at the same time.

[0136] Because the real-time computation required to screen out at-risk vehicles simultaneously on the current road section would significantly impact server performance, adaptive region selection is implemented. Feature extraction, including but not limited to location, speed, and acceleration, is performed based on the vehicles on the current road section. These features are then selected and entered into the dataset, significantly reducing computational effort and alleviating hardware pressure.

[0137] The road section characteristic parameters include at least one of the road section length, road section width, number of road section lanes and road section type of each road section in the road section set; the road section characteristic parameters also include environmental characteristic data of the road section.

[0138] The elements of the spatial feature matrix in S6 are ,in, , Indicates the total number of target road segments and neighboring road segments, represents a real number, Represents static feature data, Indicates the road section number. Represents the static characteristic data of the road section;

[0139] The x-axis of the spatial feature matrix represents the target road segment and the neighboring road segments, and the z-axis of the spatial feature matrix represents static feature data.

[0140] The spatiotemporal feature matrix in S6 is constructed by dividing the historical traffic condition information into windows;

[0141] The elements in the spatiotemporal feature matrix are ,in, , Indicates the number of historical time windows obtained by dividing the historical time period according to the preset time interval. Indicates historical traffic information. Represents the partitioned time matrix;

[0142] The x-axis of the spatiotemporal feature matrix represents the target road segment and the neighboring road segments, the y-axis of the spatiotemporal feature matrix represents the historical time window, and the z-axis of the spatiotemporal feature matrix represents the historical road condition information.

[0143] In this embodiment 2, since there are often multiple vehicles and the actual road conditions are changeable in actual application scenarios, a historical driving behavior model and a scenario model are constructed for collision detection of the driving paths of multiple vehicles, and vehicle information and road information are comprehensively described through multiple models.

[0144] The calculation formula for the feature weighted fusion in S7 is:

[0145]

[0146]

[0147]

[0148] in, represents the multi-model fusion evaluation result, i.e., the vehicle collision prediction probability including road construction information, represents the matrix tensor product, represents the matrix XOR operation, represents the activation function, represents the predicted probability of multi-vehicle two-dimensional route intersection, represents the relative bias probability of the preceding vehicle, represents the relative bias probability of the following car, represents the bias probability of the multi-vehicle ordinate matrix, represents the multi-vehicle longitudinal displacement matrix, represents the bias probability of the multi-vehicle horizontal coordinate matrix, Represents the multi-vehicle horizontal coordinate displacement matrix.

[0149] In this embodiment 2, the probability calculation process of performing feature weighted fusion under multi-segment road conditions is as follows:

[0150]

[0151]

[0152] in, Indicates the total number of roads included in the road network. Indicates the multimodal traffic information of the first road segment, Indicates the multimodal traffic information of the second road segment, Indicates the Multimodal traffic information for each road section, represents the spatial feature matrix of the first road section in time period 1, represents the spatial feature matrix of the first road section in time period 1, Indicates the The spatial feature matrix of the road section in time period 1, Indicates the road sections in the time period The spatial feature matrix within, represents the spatiotemporal feature matrix of the first road section in time period 1, represents the spatiotemporal feature matrix of the first road section in time period 1, Indicates the The spatiotemporal feature matrix of the road section in time period 1, Indicates the road sections in the time period The spatiotemporal feature matrix within.

[0153] In this embodiment 2, Represents the predicted probability of a two-dimensional route intersection of multiple vehicles. Taking two vehicles as an example, when the two vehicles collide, the horizontal coordinate deviation of the two vehicles is calculated. and the vertical coordinate deviation of the two vehicles .

[0154] The calculation formula for the two-dimensional displacement collision prediction probability of two vehicles in the multi-vehicle two-dimensional route intersection prediction probability is:

[0155]

[0156]

[0157]

[0158] in, , They represent the horizontal and vertical coordinates of vehicle 1 and vehicle 2 before displacement, and the positive direction of the y-axis is the forward direction to the left, and the negative direction of the y-axis is the right direction. represents the horizontal coordinate deviation of the two vehicles, represents the deviation of the longitudinal coordinates of the two vehicles, represents the two-dimensional horizontal coordinate of the predicted collision point of vehicle 1, represents the two-dimensional ordinate of the predicted collision point of vehicle 1, represents the turning radius of vehicle 1 before the collision, represents the turning radius of vehicle 1 when it is predicted to collide, represents the turning radius of vehicle 2 before collision, represents the turning radius of vehicle 2 when it is predicted to collide, represents the moving deflection angle of vehicle 1 before the collision, represents the moving deflection angle of vehicle 2 before the collision, represents the moving deflection angle of vehicle 1 when it is predicted to collide, represents the moving deflection angle of vehicle 2 when it is predicted to collide, represents the two-dimensional horizontal coordinate of the predicted collision point of vehicle 2, The two-dimensional vertical coordinate of the predicted collision point of vehicle 2.

[0159] The turning radius of two vehicles in the multi-vehicle two-dimensional route intersection prediction probability is as follows:

[0160]

[0161]

[0162]

[0163]

[0164] in, represents the turning radius of vehicle 1 before the collision, represents the turning radius of vehicle 1 when it is predicted to collide, represents the turning radius of vehicle 2 before collision, represents the turning radius of vehicle 2 when it is predicted to collide, represents the moving deflection angle of vehicle 1 before the collision, represents the moving deflection angle of vehicle 2 before the collision, represents the moving deflection angle of vehicle 1 when it is predicted to collide, Represents the predicted heading angle of vehicle 2 at the time of collision. The turning radius is the distance from the steering center to the point where the front outer steering wheel makes contact with the ground during vehicle travel. It is calculated by dividing the distance from the center of the vehicle's front axle to the center of the rear axle by the turning angle (in radians).

[0165] like Figure 3 As shown, calculate the displacement deviation of the two vehicles. and When the two vehicles' horizontal and vertical coordinates deviate slightly, the probability of a collision is inferred to be high. Therefore, by calculating the two-dimensional displacement collision probability of two vehicles, we can derive the two-dimensional intersection probability of multiple vehicles, enabling real-time multi-vehicle collision prediction. Finally, if the predicted collision probability exceeds 70% through multi-model fusion evaluation, the mobile app will prompt the user to adjust the steering to avoid danger and activate the assisted steering function.

[0166] In summary, the present invention can predict driving paths in high-risk areas in real time. Traditional early warning methods can only give the probability of risk occurrence based on historical data. The present invention simultaneously models road conditions and vehicle information to predict collision probabilities for driving paths of multiple vehicles. The predicted probability is more accurate and real-time.

Claims

1. A road risk simulation and avoidance method based on multimodal IoT perception, characterized in that: The following steps are involved: S1, collect the road condition information and road characteristic parameters of the current road section in real time through the data monitoring and acquisition module, and store them in the database; S2. Selecting a vehicle adaptive area based on the road condition information in the database through the data analysis and processing module to select multiple monitoring vehicles; S3, extracting historical driving behavior data and road condition characteristics of multiple monitored vehicles; S4. Based on historical driving behavior data and road condition characteristics, a historical driving behavior model is constructed through the monitoring and warning module, and a scenario model is constructed according to the road section characteristic parameters; S5. Extract historical traffic condition information through historical driving behavior data, and extract static feature data through the scenario model; S6. Constructing a spatial feature matrix based on the static feature data, and constructing a spatiotemporal feature matrix based on the historical traffic condition information; S7. Perform feature weighted fusion on the spatial feature matrix and the spatiotemporal feature matrix to obtain a multi-model fusion evaluation result, and obtain the vehicle collision probability based on the multi-model fusion evaluation result; S8. Sending the vehicle collision probability to the IoT device and the smart transportation subsystem to prompt the driver to adjust the steering to avoid danger, and activating the assisted steering avoidance function through the intelligent risk avoidance module; It is characterized in that the calculation formula for feature weighted fusion in S7 is: in, represents the multi-model fusion evaluation result, i.e., the vehicle collision prediction probability including road construction information, represents the matrix tensor product, represents the matrix XOR operation, represents the activation function, represents the predicted probability of multi-vehicle two-dimensional route intersection, represents the relative deviation probability of the preceding vehicle, represents the relative bias probability of the following car, represents the bias probability of the multi-vehicle ordinate matrix, represents the multi-vehicle longitudinal displacement matrix, represents the bias probability of the multi-vehicle horizontal coordinate matrix, represents the multi-vehicle horizontal coordinate displacement matrix; The calculation formula for the two-dimensional displacement collision prediction probability of two vehicles in the multi-vehicle two-dimensional route intersection prediction probability is: in, , They represent the horizontal and vertical coordinates of vehicle 1 and vehicle 2 before displacement, and the positive direction of the y-axis is the forward direction to the left, and the negative direction of the y-axis is the right direction. represents the horizontal coordinate deviation of the two vehicles, represents the deviation of the longitudinal coordinates of the two vehicles, represents the two-dimensional horizontal coordinate of the predicted collision point of vehicle 1, represents the two-dimensional ordinate of the predicted collision point of vehicle 1, represents the turning radius of vehicle 1 before the collision, represents the turning radius of vehicle 1 when it is predicted to collide, represents the turning radius of vehicle 2 before collision, represents the turning radius of vehicle 2 when it is predicted to collide, represents the moving deflection angle of vehicle 1 before the collision, represents the moving deflection angle of vehicle 2 before the collision, represents the moving deflection angle of vehicle 1 when it is predicted to collide, represents the moving deflection angle of vehicle 2 when it is predicted to collide, represents the two-dimensional horizontal coordinate of the predicted collision point of vehicle 2, The two-dimensional ordinate representing the predicted collision point of vehicle 2; The formula for the turning radius of two vehicles in the multi-vehicle two-dimensional route intersection prediction probability is as follows: in, represents the turning radius of vehicle 1 before the collision, represents the turning radius of vehicle 1 when it is predicted to collide, represents the turning radius of vehicle 2 before collision, represents the turning radius of vehicle 2 when it is predicted to collide, represents the moving deflection angle of vehicle 1 before the collision, represents the moving deflection angle of vehicle 2 before the collision, represents the moving deflection angle of vehicle 1 when it is predicted to collide, Indicates the predicted moving deviation angle of vehicle 2 when it collides. The turning radius refers to the distance from the steering center to the contact point of the front outer steering wheel with the ground during the vehicle's driving process. Its principle is to divide the distance from the center of the vehicle's front wheel axle to the center of the rear wheel axle by the turning angle.

2. The highway risk simulation and avoidance method based on multimodal IoT perception according to claim 1 is characterized in that: The specific steps of S2 are: S21, analyzing the road condition information in the database through the data analysis and processing module to extract multiple vehicle information; S22, grouping the multiple vehicle information into pairs, taking the front and rear vehicles of the pairwise group as observation samples, and determining whether the front and rear vehicles are in the same lane and traveling in the same direction. If so, proceed to S24; otherwise, proceed to S23; S23: Determine whether the two vehicles are in different lanes and whether the difference in their driving direction angles is within 180°. If so, proceed to S24; otherwise, remove the two vehicles from observation samples. S24. Determine whether the two vehicles collide within 5 seconds. If so, mark the two vehicles as monitoring vehicles. Otherwise, remove the two vehicles as observation samples.

3. The highway risk simulation and avoidance method based on multimodal IoT perception according to claim 2 is characterized in that: The formula for determining whether the front and rear vehicles collide within 5 seconds in S24 is: in, Indicates time Displacement of the rear vehicle, represents the initial position of the following vehicle, represents the initial speed of the following vehicle, represents the acceleration of the following vehicle, Indicates time The displacement of the front vehicle, represents the initial position of the preceding vehicle, represents the initial speed of the preceding vehicle, represents the acceleration of the preceding vehicle, Indicates time, take 5s.

4. The highway risk simulation and avoidance method based on multimodal IoT perception according to claim 1 is characterized in that: The road section characteristic parameters include at least one of the road section length, road section width, number of road section lanes and road section type of each road section in the road section set; the road section characteristic parameters also include environmental characteristic data of the road section.

5. The highway risk simulation and avoidance method based on multimodal IoT perception according to claim 1 is characterized in that: The elements of the spatial feature matrix in S6 are ,in, , Indicates the total number of target road segments and neighboring road segments, represents a real number, Represents static feature data, Indicates the road section number. Represents the static characteristic data of the road section; The x-axis of the spatial feature matrix represents the target road segment and the neighboring road segments, and the z-axis of the spatial feature matrix represents static feature data.

6. The highway risk simulation and avoidance method based on multimodal IoT perception according to claim 5 is characterized in that: The spatiotemporal feature matrix in S6 is constructed by dividing the historical traffic condition information into windows; The elements in the spatiotemporal feature matrix are ,in, , Indicates the number of historical time windows obtained by dividing the historical time period according to the preset time interval. Indicates historical traffic information. Represents the partitioned time matrix; The x-axis of the spatiotemporal feature matrix represents the target road segment and the neighboring road segments, the y-axis of the spatiotemporal feature matrix represents the historical time window, and the z-axis of the spatiotemporal feature matrix represents the historical road condition information.

7. A highway risk simulation and avoidance system based on multimodal IoT perception, used to implement the highway risk simulation and avoidance method based on multimodal IoT perception according to any one of claims 1 to 6, characterized in that: It includes an Internet of Things cloud platform and a smart transportation subsystem; the Internet of Things cloud platform includes Internet of Things devices and a cloud platform server; the Internet of Things cloud platform connects the Internet of Things device information via the Internet; the smart transportation subsystem is connected to the Internet of Things cloud platform via the Internet; The cloud platform server is used to monitor and store road condition information and predict the probability of vehicle collision based on the road condition information; the IoT device is used to view the road condition information monitored by the cloud platform server; The intelligent traffic subsystem is used for emergency command and activates the assisted steering avoidance function based on the vehicle collision probability predicted by the Internet of Things cloud platform.

8. The highway risk simulation and avoidance system based on multimodal IoT perception according to claim 7 is characterized in that: The cloud platform server includes a data monitoring and acquisition module, a database, a data analysis and processing module, and a monitoring and early warning module; The IoT devices include mobile phones, computers, and tablets, which are used to display road condition information and vehicle collision probability, and remind users to avoid risks; The intelligent transportation subsystem includes an emergency command module and an intelligent risk avoidance module; The data monitoring and collection module is used to collect road condition information; The database is used to store the road condition information collected by the data monitoring and collection module; The data analysis and processing module is used to perform data analysis and processing on the road condition information in the database; The monitoring and warning module is used to predict the probability of vehicle collision by analyzing and processing the data; The emergency command module is used for decision-making assistance and safety emergency response; The intelligent risk avoidance module is used to activate the auxiliary steering avoidance function according to the vehicle collision probability.

Images