Highway risk simulation avoidance system and method based on multi-mode internet-of-things perception

Abstract

The invention discloses a road risk simulation avoidance system and method based on multi-mode Internet of Things perception, and belongs to the technical field of intelligent traffic, and the method comprises the following steps: collecting the road condition information and road section characteristic parameters of a current road section in real time; screening out a plurality of monitoring vehicles; extracting historical driving behavior data and road condition characteristics of the plurality of monitored vehicles; constructing a historical driving behavior model, and constructing a scene model; extracting historical traffic road condition information, and extracting static feature data through the scene model; constructing a spatial feature matrix, and constructing a spatial-temporal feature matrix; performing feature weighted fusion on the spatial feature matrix and the spatial-temporal feature matrix to obtain a multi-model fusion evaluation result, and calculating a vehicle collision probability; and a vehicle owner is prompted to avoid danger and adjust steering, and an auxiliary steering avoiding function is started through the intelligent danger avoiding module. The problems that a traditional monitoring function cannot achieve risk modeling early warning and risk avoidance and lacks a multi-vehicle collision real-time prediction function are solved.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent transportation technology, and specifically relates to a highway risk simulation avoidance system and method based on multi-modal Internet of Things perception. Background Art

[0002] Traffic warning systems provide more efficient and intelligent solutions for urban traffic management. Traffic warning systems facilitate people's travel by providing traffic warnings and route optimization services, while also helping to avoid traffic congestion and improve the efficiency of urban traffic. At the same time, they also have the function of road monitoring. By real-time monitoring of road traffic conditions, potential dangers can be discovered in a timely manner, such as pedestrians crossing zebra crossings, and drivers can be reminded to pay attention, thereby reducing the occurrence of traffic accidents.

[0003] However, traditional monitoring functions cannot achieve risk modeling warning and risk avoidance, and lack the real-time prediction function of multiple vehicle collisions. Therefore, a real-time monitoring of multiple vehicle traffic data, environmental information and user behavior patterns is needed to model, predict and warn of potential traffic accident risks in areas with obstructed vision or complex road conditions, and at the same time provide avoidance solutions to further reduce the occurrence of traffic accidents. Summary of the invention

[0004] The present invention provides a highway risk simulation and avoidance system and method based on multi-modal Internet of Things perception, which solves the problem that traditional monitoring functions cannot achieve risk modeling warning and risk avoidance, and lacks real-time prediction function for multi-vehicle collision.

[0005] In order to solve the above technical problems, the technical solution of the present invention is: a highway risk simulation avoidance system based on multimodal IoT perception, including an IoT cloud platform and a smart traffic subsystem; the IoT cloud platform includes IoT devices and a cloud platform server; the IoT cloud platform connects IoT device information via the Internet; the smart traffic subsystem is connected to the IoT cloud platform via the Internet; The cloud platform server is used to monitor and store road condition information and predict the probability of vehicle collision based on the road condition information; the IoT device is used to view the road condition information monitored by the cloud platform server; The intelligent traffic subsystem is used for emergency command and activates the auxiliary steering avoidance function according to the vehicle collision probability predicted by the Internet of Things cloud platform.

[0006] Furthermore, the cloud platform server includes a data monitoring and acquisition module, a database, a data analysis and processing module, and a monitoring and early warning module; The IoT devices include mobile phones, computers and tablet computers, which are used to display road condition information and vehicle collision probability, and remind users to avoid danger; The smart transportation subsystem includes an emergency command module and an intelligent risk avoidance module; The data monitoring and collection module is used to collect road condition information; The database is used to store the road condition information collected by the data monitoring and collection module; The data analysis and processing module is used to perform data analysis and processing on the road condition information in the database; The monitoring and early warning module is used to predict the probability of vehicle collision by analyzing and processing the data; The emergency command module is used for decision-making assistance and safety emergency response; The intelligent risk avoidance module is used to activate the auxiliary steering avoidance function according to the vehicle collision probability.

[0007] The beneficial effects of the present invention are: (1) the driving path of high-risk areas is predicted in real time through the monitoring and early warning module, and modeling is performed based on building information and vehicle information to predict the collision probability based on the driving paths of multiple vehicles. The prediction probability is more accurate and has real-time performance; (2) Process information data through the IoT cloud platform and use IoT devices, such as smart terminal apps, to push and display traffic information; (3) Through the emergency command module, the traffic environment is monitored and emergency response is provided for the smooth operation of vehicles through big data and traffic development analysis. The monitoring can also be used to command the vehicle. At the same time, the intelligent risk avoidance module can be used to remind users to avoid risks. It has a certain practical guidance effect. When driving in a high-risk traffic environment and encountering danger but still does not react or has no time to think about how to avoid it, the probability of accidents can be reduced to a greater extent.

[0008] The present invention also provides a highway risk simulation avoidance method based on multi-modal IoT perception, comprising the following steps: S1. The data monitoring and acquisition module collects the road condition information and road characteristic parameters of the current road section in real time and stores them in the database; S2, selecting a vehicle adaptive area for the road condition information in the database through the data analysis and processing module, and screening out multiple monitoring vehicles; S3, extracting historical driving behavior data and road condition characteristics of multiple monitored vehicles; S4. Based on historical driving behavior data and road condition characteristics, a historical driving behavior model is constructed through a monitoring and early warning module, and a scenario model is constructed according to road section characteristic parameters; S5, extracting historical traffic condition information through historical driving behavior data, and extracting static feature data through the scene model; S6. construct a spatial feature matrix based on the static feature data, and construct a spatiotemporal feature matrix based on the historical traffic condition information; S7, performing feature weighted fusion on the spatial feature matrix and the spatiotemporal feature matrix to obtain a multi-model fusion evaluation result, and obtaining the vehicle collision probability according to the multi-model fusion evaluation result; S8. Send the vehicle collision probability to the IoT device and the smart traffic subsystem to prompt the vehicle owner to adjust the steering to avoid danger, and start the auxiliary steering avoidance function through the intelligent risk avoidance module.

[0009] The beneficial effects of the present invention are as follows: the present invention can predict driving paths in high-risk areas in real time. Traditional early warning methods can only give the probability of risk occurrence in combination with historical data. At the same time, modeling is carried out based on building information and vehicle information to predict the collision probability for driving paths of multiple vehicles. The predicted probability is more accurate and real-time.

[0010] Furthermore, the specific steps of S2 are: S21, analyzing the road condition information in the database through the data analysis and processing module to extract multiple vehicle information; S22, grouping the multiple vehicle information into pairs, taking the front and rear vehicles of the pairwise grouping as observation samples, and determining whether the front and rear vehicles are in the same lane and have the same driving direction. If so, proceed to S24, otherwise, proceed to S23; S23, determining whether the front and rear vehicles are located in different lanes and whether the difference in the driving direction angle is within 180°, if so, proceeding to S24, otherwise, canceling the two vehicles as observation samples; S24. Determine whether the two vehicles in front and behind collide within 5 seconds. If so, mark the two vehicles as monitoring vehicles. Otherwise, remove the two vehicles as observation samples.

[0011] The beneficial effect of the above further scheme is: to screen out the vehicles involved in danger at the same time on the current road section, the real-time calculation amount will have a significant performance impact on the server. Therefore, by selecting adaptive areas and screening the monitored vehicles, the pressure on existing hardware can be alleviated.

[0012] Furthermore, the formula for determining whether the front and rear vehicles collide within 5 seconds in S24 is:

[0013]

[0014]

[0015] in, Indicates time Displacement of the rear vehicle, represents the initial position of the following vehicle, represents the initial speed of the following vehicle, represents the acceleration of the following vehicle, Indicates time The displacement of the front vehicle, represents the initial position of the front vehicle, represents the initial speed of the front vehicle, represents the acceleration of the preceding vehicle, Indicates time, take 5s.

[0016] Furthermore, the road section characteristic parameters include at least one of the section length, section width, number of section lanes and section type of each section in the road section set; the road section characteristic parameters also include environmental characteristic data of the section.

[0017] Furthermore, the elements of the spatial feature matrix in S6 are ,in, , Represents the total number of target segments and neighboring segments, represents a real number, Represents static feature data, Indicates the road section number. Represents the static characteristic data of the road section; The x-axis of the spatial feature matrix represents the target road segment and the neighboring road segment, and the z-axis of the spatial feature matrix represents the static feature data.

[0018] Furthermore, the spatiotemporal feature matrix in S6 is constructed by the historical traffic condition information after dividing the window; The elements in the spatiotemporal feature matrix are ,in, , Indicates the number of historical time windows obtained by dividing the historical time period according to the preset time interval. Indicates historical traffic information. represents the time matrix of the partition; The x-axis of the spatiotemporal feature matrix represents the target road section and the neighboring road sections, the y-axis of the spatiotemporal feature matrix represents the historical time window, and the z-axis of the spatiotemporal feature matrix represents the historical road condition information.

[0019] The beneficial effect of the above further scheme is: by obtaining the spatiotemporal characteristics of historical road conditions, the influence of multiple factors on the real-time prediction of multi-vehicle collision is taken into account, so that the prediction result is more accurate.

[0020] Furthermore, the calculation formula for the feature weighted fusion in S7 is:

[0021]

[0022]

[0023] in, represents the multi-model fusion evaluation result, that is, the vehicle collision prediction probability including road construction information, represents the matrix tensor product, represents the matrix XOR operation, represents the activation function, represents the predicted probability of multi-vehicle two-dimensional route intersection, represents the relative bias probability of the front vehicle, represents the relative bias probability of the following vehicle, represents the bias probability of the multi-vehicle ordinate matrix, represents the multi-vehicle longitudinal displacement matrix, represents the probability of the multi-vehicle horizontal coordinate matrix bias, Represents the multi-vehicle horizontal coordinate displacement matrix.

[0024] The beneficial effect of the above further scheme is: the road section characteristic parameters and the spatiotemporal characteristic matrix are fused to obtain a multimodal evaluation model, which can further calibrate the road construction information and make the multi-model fusion evaluation results more accurate.

[0025] Furthermore, the calculation formula of the collision probability of two-vehicle two-dimensional displacement collision in the multi-vehicle two-dimensional route intersection prediction probability is:

[0026]

[0027]

[0028] in, , They represent the horizontal and vertical coordinates of vehicle 1 and vehicle 2 before displacement, and the positive direction of the y-axis is the forward direction to the left, and the negative direction of the y-axis is the rightward direction. represents the horizontal coordinate deviation of the two vehicles, represents the deviation of the longitudinal coordinates of the two vehicles, represents the two-dimensional horizontal coordinate of the predicted collision point of vehicle 1, represents the two-dimensional ordinate of the predicted collision point of vehicle 1, represents the turning radius of vehicle 1 before the collision, represents the turning radius of vehicle 1 when it is predicted to collide, represents the turning radius of vehicle 2 before collision, represents the turning radius of vehicle 2 when it is predicted to collide, represents the moving deflection angle of vehicle 1 before the collision, represents the moving deflection angle of vehicle 2 before the collision, represents the moving deflection angle of vehicle 1 when it is predicted to collide, represents the moving deflection angle of vehicle 2 when it is predicted to collide, represents the two-dimensional horizontal coordinate of the predicted collision point of vehicle 2, The two-dimensional vertical coordinate of the predicted collision point of vehicle 2.

[0029] in The derivation process is Move to The prediction example is as follows: Depend on The displacement is obtained, and so on; the schematic diagram is as follows Figure 4 As shown: 1. Modeling based on the ground coordinate system: The calculation of the vehicle's motion trajectory is based on the ground coordinate system OXY; Define Vehicle The location is , It indicates the moving deflection angle of the vehicle before the collision. The vehicle position at the moment is , Indicates the moving deflection angle of the vehicle when it is predicted to collide; The turning radius of the vehicle is defined as R, with the positive direction of the y-axis as the forward direction to the left and the negative direction of the y-axis as the rightward direction; the center point of rotation of the vehicle is defined as , if driving to the left, the center of rotation falls on the left side of the vehicle body, if driving to the right, the center of rotation falls on the right side of the vehicle body; define The projections on the X and Y axes are and , The projections on the X and Y axes are and Assume that the vehicle is The speed inside is V, a positive value represents forward travel, and a negative value represents backward travel. The vehicle can be regarded as moving at a constant speed in a short period of time.

[0030] 2. Angle relationship: Assuming the vehicle is moving right forward, V is a positive value and R is a negative value.

[0031]

[0032] 3. Calculate the projection: Considering the above assumption that the vehicle is traveling to the right and R is a negative value, a “-” sign is added to represent the projection distance value. Calculate Projection on the X and Y axes:

[0033] calculate Projection on the X and Y axes:

[0034] 4 Calculate the vehicle position: Substitute the above process into the following formula:

[0035]

[0036] 5. Consider the case where the turning radius at two adjacent moments is different: Assume that the turning radius at two adjacent moments is and ,calculate Projection on the X and Y axes:

[0037] calculate Projection on the X and Y axes:

[0038] According to the geometric relationship, we get the following formula:

[0039]

[0040] but The deduction can be obtained by the same logic.

[0041] Furthermore, the turning radius derivation process of two vehicles in the multi-vehicle two-dimensional route intersection prediction probability is as follows:

[0042]

[0043]

[0044]

[0045] in, represents the turning radius of vehicle 1 before the collision, represents the turning radius of vehicle 1 when it is predicted to collide, represents the turning radius of vehicle 2 before collision, represents the turning radius of vehicle 2 when it is predicted to collide, represents the moving deflection angle of vehicle 1 before the collision, represents the moving deflection angle of vehicle 2 before the collision, represents the moving deflection angle of vehicle 1 when it is predicted to collide, It indicates the moving deflection angle of vehicle 2 when it is predicted to collide. The turning radius refers to the distance from the steering center to the contact point between the front outer steering wheel and the ground during the driving process of the car. The principle is the distance from the center of the front wheel axle to the center of the rear wheel axle of the vehicle divided by the turning angle (in radians).

[0046] The derivation process of R=L / sin(θ) is as follows: Figure 5 As shown: 1. Set the coordinate system definition parameters: Set a coordinate system so that the initial driving direction of the car coincides with the X-axis; L represents the wheelbase of the car, that is, the distance from the center of the front wheel axle to the center of the rear wheel axle; θ represents the steering angle of the front wheel, usually refers to the maximum turning angle of the front wheel relative to the forward direction of the vehicle; R represents the turning radius, that is, the radius of the circle formed by the vehicle when turning.

[0047] 2. Define the steering angle: When the car starts to turn, the front wheels will rotate around a center of a circle; let the steering angle of the front wheels be θ, which is the angle between the front wheels and the straight direction of the vehicle.

[0048] 3. Construct a triangle: In the geometric model of car steering, the wheelbase L of the car forms a right-angled side of a right-angled triangle, and the turning radius R is the hypotenuse of this triangle.

[0049] 4. Use trigonometric functions: In a right triangle, the tangent function sin is defined as the opposite side / hypotenuse; in this case, the opposite side is the wheelbase L of the car, and the adjacent side is the perpendicular component of the radius R (that is, the distance from the center of the circle to the rear wheels of the car).

[0050] 5. Derived formula: According to the definition of tangent function, we can get sin(θ)=opposite side / hypotenuse=L / R.

[0051] 6. Solve for the radius: Solve the above equation to obtain R, R = L / sin(θ); where the turning radius R is defined as the minimum turning radius of the vehicle, the steering angle θ is the maximum steering angle of the vehicle, and L is defined as the distance from the center of the front axle to the center of the rear axle of the vehicle.

[0052] The beneficial effect of the above further scheme is: by calculating the displacement deviation of the two-dimensional plane of the two vehicles, the probability of the collision between the two vehicles can be inferred, thereby obtaining the predicted probability of the intersection of the horizontal and vertical coordinates of the two-dimensional routes of multiple vehicles, and realizing real-time prediction of multiple vehicle collisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flowchart of the highway risk simulation and avoidance system based on multi-modal IoT perception of the present invention.

[0054] Figure 2 The present invention is a flow chart of a highway risk simulation avoidance method based on multimodal IoT perception.

[0055] Figure 3 This is a schematic diagram of vehicle collision prediction according to the present invention.

[0056] Figure 4 It is a schematic diagram of vehicle displacement position prediction according to the present invention.

[0057] Figure 5 This is a schematic diagram of calculating the turning radius of a vehicle according to the present invention. DETAILED DESCRIPTION

[0058] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

[0059] Example 1 like Figure 1 As shown, the present invention provides a highway risk simulation avoidance system based on multi-modal IoT perception, including an IoT cloud platform and a smart traffic subsystem; the IoT cloud platform includes IoT devices and a cloud platform server; the IoT cloud platform connects IoT device information via the Internet; the smart traffic subsystem is connected to the IoT cloud platform via the Internet; The cloud platform server is used to monitor and store road condition information and predict the probability of vehicle collision based on the road condition information; the IoT device is used to view the road condition information monitored by the cloud platform server; The intelligent traffic subsystem is used for emergency command and activates the auxiliary steering avoidance function according to the vehicle collision probability predicted by the Internet of Things cloud platform.

[0060] The cloud platform server includes a data monitoring and acquisition module, a database, a data analysis and processing module, and a monitoring and early warning module; The IoT devices include mobile phones, computers and tablet computers, which are used to display road condition information and vehicle collision probability, and remind users to avoid danger; The smart transportation subsystem includes an emergency command module and an intelligent risk avoidance module; The data monitoring and collection module is used to collect road condition information; The database is used to store the road condition information collected by the data monitoring and collection module; The data analysis and processing module is used to perform data analysis and processing on the road condition information in the database; The monitoring and early warning module is used to predict the probability of vehicle collision by analyzing and processing the data; The emergency command module is used for decision-making assistance and safety emergency response; The intelligent risk avoidance module is used to activate the auxiliary steering avoidance function according to the vehicle collision probability.

[0061] In this embodiment 1, the data monitoring and acquisition module collects road condition information, including speed, acceleration, direction, lane lines, traffic signs, the location of surrounding objects and other information, and then stores the collected road condition information in the database, and pre-processes the road condition information and screens the monitored vehicles through the data analysis and processing module, and then constructs multiple models through the monitoring and early warning module to calculate the probability of vehicle collision; at the same time, the emergency command module is used for decision-making assistance and safety emergency response, and the Internet of Things device can check the road condition information at any time, and when the collision probability reaches the set threshold, the owner is prompted through the mobile phone, etc., and the intelligent risk avoidance module is used to avoid risks. Among them, the intelligent risk avoidance module is composed of an adaptive cruise control system (ACC), which is an advanced driving assistance system that can not only maintain the set vehicle speed, but also adjust its own vehicle speed according to the dynamics of the vehicle in front to maintain a safe distance from the vehicle in front.

[0062] Example 2 like Figure 2 As shown, the present invention also provides a highway risk simulation avoidance method based on multi-modal IoT perception, comprising the following steps: S1. The data monitoring and acquisition module collects the road condition information and road characteristic parameters of the current road section in real time and stores them in the database; S2, selecting a vehicle adaptive area for the road condition information in the database through the data analysis and processing module, and screening out multiple monitoring vehicles; S3, extracting historical driving behavior data and road condition characteristics of multiple monitored vehicles; S4. Based on historical driving behavior data and road condition characteristics, a historical driving behavior model is constructed through a monitoring and early warning module, and a scenario model is constructed according to road section characteristic parameters; S5, extracting historical traffic condition information through historical driving behavior data, and extracting static feature data through the scene model; S6. construct a spatial feature matrix based on the static feature data, and construct a spatiotemporal feature matrix based on the historical traffic condition information; S7, performing feature weighted fusion on the spatial feature matrix and the spatiotemporal feature matrix to obtain a multi-model fusion evaluation result, and obtaining the vehicle collision probability according to the multi-model fusion evaluation result; S8. Send the vehicle collision probability to the IoT device and the smart traffic subsystem to prompt the vehicle owner to adjust the steering to avoid danger, and start the auxiliary steering avoidance function through the intelligent risk avoidance module.

[0063] The specific steps of S2 are: S21, analyzing the road condition information in the database through the data analysis and processing module to extract multiple vehicle information; S22, grouping the multiple vehicle information into pairs, taking the front and rear vehicles of the pairwise grouping as observation samples, and determining whether the front and rear vehicles are in the same lane and have the same driving direction. If so, proceed to S24, otherwise, proceed to S23; S23, determining whether the front and rear vehicles are located in different lanes and whether the difference in the driving direction angle is within 180°, if so, proceeding to S24, otherwise, canceling the two vehicles as observation samples; S24. Determine whether the two vehicles in front and behind collide within 5 seconds. If so, mark the two vehicles as monitoring vehicles. Otherwise, remove the two vehicles as observation samples.

[0064] The formula for determining whether the front and rear vehicles collide within 5 seconds in S24 is:

[0065]

[0066]

[0067] in, Indicates time Displacement of the rear vehicle, represents the initial position of the following vehicle, represents the initial speed of the following vehicle, represents the acceleration of the following vehicle, Indicates time The displacement of the front vehicle, represents the initial position of the front vehicle, represents the initial speed of the front vehicle, represents the acceleration of the preceding vehicle, Indicates time, take 5s.

[0068] In the second embodiment, the current driving path of the vehicle is predicted by the monitoring and early warning module, and whether the vehicle path in the next 5 seconds will collide or not is predicted. Therefore, it is necessary to monitor multiple vehicles on the same road section at the same time.

[0069] Since the real-time calculation of screening vehicles at risk at the same time on the current road section will have a significant impact on the performance of the server, adaptive area selection is performed. Feature extraction is performed based on the vehicles in the current road section, including but not limited to location, speed and acceleration, and the monitoring objects are screened and entered into the data set, which can greatly reduce the amount of calculation and relieve hardware pressure.

[0070] The road section characteristic parameters include at least one of the road section length, road section width, number of road section lanes and road section type of each road section in the road section set; the road section characteristic parameters also include environmental characteristic data of the road section.

[0071] The elements of the spatial feature matrix in S6 are ,in, , Represents the total number of target road segments and neighboring road segments, represents a real number, Represents static feature data, Indicates the road section number. Represents the static characteristic data of the road section; The x-axis of the spatial feature matrix represents the target road segment and the neighboring road segment, and the z-axis of the spatial feature matrix represents the static feature data.

[0072] The spatiotemporal feature matrix in S6 is constructed by the historical traffic condition information after dividing the window; The elements in the spatiotemporal feature matrix are ,in, , Indicates the number of historical time windows obtained by dividing the historical time period according to the preset time interval. Indicates historical traffic information. represents the time matrix of the partition; The x-axis of the spatiotemporal feature matrix represents the target road section and the neighboring road sections, the y-axis of the spatiotemporal feature matrix represents the historical time window, and the z-axis of the spatiotemporal feature matrix represents the historical road condition information.

[0073] In this embodiment 2, since there are often multiple vehicles and actual road conditions are changeable in actual application scenarios, a historical driving behavior model and a scenario model are constructed for collision detection of the driving paths of multiple vehicles, and multiple models are used to comprehensively describe vehicle information and road information.

[0074] The calculation formula for weighted feature fusion in S7 is:

[0075]

[0076]

[0077] in, represents the multi-model fusion evaluation result, that is, the vehicle collision prediction probability including road construction information, represents the matrix tensor product, represents the matrix XOR operation, represents the activation function, represents the predicted probability of multi-vehicle two-dimensional route intersection, represents the relative bias probability of the front vehicle, represents the relative bias probability of the following vehicle, represents the bias probability of the multi-vehicle ordinate matrix, represents the multi-vehicle longitudinal displacement matrix, represents the probability of the multi-vehicle horizontal coordinate matrix bias, Represents the multi-vehicle horizontal coordinate displacement matrix.

[0078] In this embodiment 2, the probability calculation process of performing feature weighted fusion under multi-section road conditions is as follows:

[0079]

[0080] in, Indicates the total number of roads contained in the road network. Indicates the multimodal traffic information of the first road segment. Indicates the multimodal traffic information of the second road segment. Indicates Multimodal traffic information for each road segment, represents the spatial feature matrix of the first road section in time period 1, represents the spatial feature matrix of the first road section in time period 1, Indicates The spatial feature matrix of the road section in time period 1, Indicates Road sections in the time period The spatial feature matrix within represents the spatiotemporal feature matrix of the first road section in time period 1, represents the spatiotemporal feature matrix of the first road section in time period 1, Indicates The spatiotemporal feature matrix of road sections in time period 1, Indicates Road sections in the time period The spatiotemporal feature matrix within.

[0081] In this embodiment 2, Represents the predicted probability of a multi-vehicle two-dimensional route intersection. Taking two vehicles as an example, when the two vehicles collide, the horizontal coordinate deviation of the two vehicles is calculated and the vertical coordinate deviation of the two vehicles .

[0082] The calculation formula of the two-dimensional displacement collision prediction probability of two vehicles in the multi-vehicle two-dimensional route intersection prediction probability is:

[0083]

[0084]

[0085] in, , They represent the horizontal and vertical coordinates of vehicle 1 and vehicle 2 before displacement, and the positive direction of the y-axis is the forward direction to the left, and the negative direction of the y-axis is the rightward direction. represents the horizontal coordinate deviation of the two vehicles, represents the vertical coordinate deviation of the two vehicles, represents the two-dimensional horizontal coordinate of the predicted collision point of vehicle 1, represents the two-dimensional ordinate of the predicted collision point of vehicle 1, represents the turning radius of vehicle 1 before the collision, represents the turning radius of vehicle 1 when it is predicted to collide, represents the turning radius of vehicle 2 before the collision, represents the turning radius of vehicle 2 when it is predicted to collide, represents the moving deflection angle of vehicle 1 before the collision, represents the moving deflection angle of vehicle 2 before the collision, represents the moving deflection angle of vehicle 1 when it is predicted to collide, represents the moving deflection angle of vehicle 2 when it is predicted to collide, represents the two-dimensional horizontal coordinate of the predicted collision point of vehicle 2, The two-dimensional vertical coordinate of the predicted collision point of vehicle 2.

[0086] The turning radius of two vehicles in the multi-vehicle two-dimensional route intersection prediction probability is as follows:

[0087]

[0088]

[0089]

[0090] in, represents the turning radius of vehicle 1 before the collision, represents the turning radius of vehicle 1 when it is predicted to collide, represents the turning radius of vehicle 2 before the collision, represents the turning radius of vehicle 2 when it is predicted to collide, represents the moving deflection angle of vehicle 1 before the collision, represents the moving deflection angle of vehicle 2 before the collision, represents the moving deflection angle of vehicle 1 when it is predicted to collide, Indicates the moving deflection angle of vehicle 2 when it is predicted to collide. Turning radius refers to the distance from the steering center to the point where the front outer steering wheel contacts the ground during the driving process of the vehicle. Its principle is the distance from the center of the front wheel axle to the center of the rear wheel axle divided by the turning angle (in radians).

[0091] like Figure 3 As shown, the displacement deviation of the two vehicles is calculated. and When , it means that the deviation between the horizontal and vertical coordinates of the two vehicles is small, and it can be inferred that the probability of collision between the two vehicles is extremely high. Therefore, by calculating the two-dimensional displacement collision prediction probability of the two vehicles, the two-dimensional route intersection prediction probability of multiple vehicles is obtained, and the real-time prediction function of multi-vehicle collision is realized. Finally, through the multi-model fusion evaluation results, if the predicted vehicle collision probability is >70%, the mobile app will be triggered to prompt the user to adjust the steering to avoid risks and start the assisted steering function.

[0092] In summary, the present invention can predict driving paths in high-risk areas in real time. Traditional early warning methods can only give the probability of risk occurrence in combination with historical data, while the present invention simultaneously models road conditions and vehicle information to predict collision probabilities for driving paths of multiple vehicles. The predicted probability is more accurate and real-time.

Claims

1. A highway risk simulation and avoidance system based on multi-modal IoT perception, characterized in that: It includes an Internet of Things cloud platform and a smart transportation subsystem; the Internet of Things cloud platform includes Internet of Things devices and a cloud platform server; the Internet of Things cloud platform connects the Internet of Things device information through the Internet; the smart transportation subsystem is connected to the Internet of Things cloud platform through the Internet; The cloud platform server is used to monitor and store road condition information and predict the probability of vehicle collision based on the road condition information; the IoT device is used to view the road condition information monitored by the cloud platform server; The intelligent traffic subsystem is used for emergency command and activates the auxiliary steering avoidance function according to the vehicle collision probability predicted by the Internet of Things cloud platform.

2. The highway risk simulation and avoidance system based on multimodal IoT perception according to claim 1 is characterized in that: The cloud platform server includes a data monitoring and acquisition module, a database, a data analysis and processing module, and a monitoring and early warning module; The IoT devices include mobile phones, computers and tablet computers, which are used to display road condition information and vehicle collision probability, and remind users to avoid danger; The smart transportation subsystem includes an emergency command module and an intelligent risk avoidance module; The data monitoring and collection module is used to collect road condition information; The database is used to store the road condition information collected by the data monitoring and collection module; The data analysis and processing module is used to perform data analysis and processing on the road condition information in the database; The monitoring and early warning module is used to predict the probability of vehicle collision by analyzing and processing the data; The emergency command module is used for decision-making assistance and safety emergency response; The intelligent risk avoidance module is used to activate the auxiliary steering avoidance function according to the vehicle collision probability.

3. A highway risk simulation avoidance method based on multimodal IoT perception, characterized in that: The following steps are involved: S1. The data monitoring and acquisition module collects the road condition information and road characteristic parameters of the current road section in real time and stores them in the database; S2, selecting a vehicle adaptive area for the road condition information in the database through the data analysis and processing module, and screening out multiple monitoring vehicles; S3, extracting historical driving behavior data and road condition characteristics of multiple monitored vehicles; S4. Based on historical driving behavior data and road condition characteristics, a historical driving behavior model is constructed through a monitoring and early warning module, and a scenario model is constructed according to road section characteristic parameters; S5, extracting historical traffic condition information through historical driving behavior data, and extracting static feature data through the scene model; S6. construct a spatial feature matrix based on the static feature data, and construct a spatiotemporal feature matrix based on the historical traffic condition information; S7, performing feature weighted fusion on the spatial feature matrix and the spatiotemporal feature matrix to obtain a multi-model fusion evaluation result, and obtaining the vehicle collision probability according to the multi-model fusion evaluation result; S8. Send the vehicle collision probability to the IoT device and the smart traffic subsystem to prompt the vehicle owner to adjust the steering to avoid danger, and start the auxiliary steering avoidance function through the intelligent risk avoidance module.

4. The highway risk simulation avoidance method based on multi-modal IoT perception according to claim 3 is characterized in that: The specific steps of S2 are: S21, analyzing the road condition information in the database through the data analysis and processing module to extract multiple vehicle information; S22, grouping the multiple vehicle information into pairs, taking the front and rear vehicles of the pairwise grouping as observation samples, and determining whether the front and rear vehicles are in the same lane and have the same driving direction. If so, proceed to S24, otherwise, proceed to S23; S23, determining whether the front and rear vehicles are located in different lanes and whether the difference in the driving direction angle is within 180°, if so, proceeding to S24, otherwise, canceling the two vehicles as observation samples; S24. Determine whether the two vehicles in front and behind collide within 5 seconds. If so, mark the two vehicles as monitoring vehicles. Otherwise, remove the two vehicles as observation samples.

5. The highway risk simulation avoidance method based on multi-modal IoT perception according to claim 4 is characterized in that: The formula for determining whether the front and rear vehicles collide within 5 seconds in S24 is: in, Indicates time Displacement of the rear vehicle, represents the initial position of the following vehicle, represents the initial speed of the following vehicle, represents the acceleration of the following vehicle, Indicates time The displacement of the front vehicle, represents the initial position of the front vehicle, represents the initial speed of the front vehicle, represents the acceleration of the preceding vehicle, Indicates time, take 5s.

6. The highway risk simulation avoidance method based on multi-modal IoT perception according to claim 3 is characterized in that: The road section characteristic parameters include at least one of the road section length, road section width, number of road section lanes and road section type of each road section in the road section set; the road section characteristic parameters also include environmental characteristic data of the road section.

7. The highway risk simulation avoidance method based on multi-modal IoT perception according to claim 3 is characterized in that: The elements of the spatial feature matrix in S6 are ,in, , Represents the total number of target segments and neighboring segments, represents a real number, Represents static feature data, Indicates the road section number. Represents the static characteristic data of the road section; The x-axis of the spatial feature matrix represents the target road segment and the neighboring road segment, and the z-axis of the spatial feature matrix represents the static feature data.

8. The highway risk simulation avoidance method based on multi-modal IoT perception according to claim 7 is characterized in that: The spatiotemporal feature matrix in S6 is constructed by the historical traffic condition information after dividing the window; The elements in the spatiotemporal feature matrix are ,in, , Indicates the number of historical time windows obtained by dividing the historical time period according to the preset time interval. Indicates historical traffic information. represents the time matrix of the partition; The x-axis of the spatiotemporal feature matrix represents the target road section and the neighboring road sections, the y-axis of the spatiotemporal feature matrix represents the historical time window, and the z-axis of the spatiotemporal feature matrix represents the historical road condition information.

9. The highway risk simulation avoidance method based on multi-modal IoT perception according to claim 5 is characterized in that: The calculation formula for weighted feature fusion in S7 is: in, represents the multi-model fusion evaluation result, that is, the vehicle collision prediction probability including road construction information, represents the matrix tensor product, represents the matrix XOR operation, represents the activation function, represents the predicted probability of multi-vehicle two-dimensional route intersection, represents the relative bias probability of the front vehicle, represents the relative bias probability of the following vehicle, represents the bias probability of the multi-vehicle ordinate matrix, represents the multi-vehicle longitudinal displacement matrix, represents the probability of the multi-vehicle horizontal coordinate matrix bias, Represents the multi-vehicle horizontal coordinate displacement matrix.

10. The highway risk simulation avoidance method based on multi-modal IoT perception according to claim 9 is characterized in that: The calculation formula of the two-dimensional displacement collision prediction probability of two vehicles in the multi-vehicle two-dimensional route intersection prediction probability is: in, , They represent the horizontal and vertical coordinates of vehicle 1 and vehicle 2 before displacement, respectively, and the positive direction of the y-axis is the forward direction to the left, and the negative direction of the y-axis is the rightward direction. represents the horizontal coordinate deviation of the two vehicles, represents the deviation of the longitudinal coordinates of the two vehicles, represents the two-dimensional horizontal coordinate of the predicted collision point of vehicle 1, represents the two-dimensional ordinate of the predicted collision point of vehicle 1, represents the turning radius of vehicle 1 before the collision, represents the turning radius of vehicle 1 when it is predicted to collide, represents the turning radius of vehicle 2 before collision, represents the turning radius of vehicle 2 when it is predicted to collide, represents the moving deflection angle of vehicle 1 before the collision, represents the moving deflection angle of vehicle 2 before the collision, represents the moving deflection angle of vehicle 1 when it is predicted to collide, represents the moving deflection angle of vehicle 2 when it is predicted to collide, represents the two-dimensional horizontal coordinate of the predicted collision point of vehicle 2, The two-dimensional vertical coordinate of the predicted collision point of vehicle 2.

11. The highway risk simulation avoidance method based on multi-modal IoT perception according to claim 9 is characterized in that: The formula for the turning radius of two vehicles in the multi-vehicle two-dimensional route intersection prediction probability is as follows: in, represents the turning radius of vehicle 1 before the collision, represents the turning radius of vehicle 1 when it is predicted to collide, represents the turning radius of vehicle 2 before collision, represents the turning radius of vehicle 2 when it is predicted to collide, represents the moving deflection angle of vehicle 1 before the collision, represents the moving deflection angle of vehicle 2 before the collision, represents the moving deflection angle of vehicle 1 when it is predicted to collide, Indicates the moving deflection angle of vehicle 2 when it is predicted to collide. Turning radius refers to the distance from the steering center to the point where the front outer steering wheel contacts the ground during the driving process of the vehicle. Its principle is the distance from the center of the front wheel axle to the center of the rear wheel axle divided by the turning angle (in radians).

Images