Body contact type safety warning system based on potential risk of automatic driving

Abstract

The invention discloses a body contact type safety warning system based on potential risks of automatic driving, which relates to the technical field of automatic driving and comprises a data receiving module, a data processing module, a safety warning module and a terminal interaction module. The data receiving module is used for receiving data to obtain an image data set, a vehicle motion data set and communication transmission data; the data processing module is used for performing calculation to obtain a visual risk early warning degree, a predicted vehicle body stability degree and a communication data packet distortion degree; the safety warning module is used for carrying out classification judgment to obtain an early warning result and sending out a vibration warning; and the terminal interaction module is used for providing a visual interaction page for a user, receiving an instruction and storing data. According to automatic driving service degradation and potential risks, a driver is reminded to keep alert and take over in a body contact type early warning mode, meanwhile, a safety belt is tightened in advance under the high-risk condition, and a scientific basis is provided for defining automatic driving accident liability affiliation while driving safety is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving, and in particular to a body contact safety warning system based on potential risks of autonomous driving. Background Art

[0002] In recent years, with the implementation of innovative achievements in the fields of information technology, integrated circuit technology and artificial intelligence technology, the assisted autonomous driving technology of intelligent networked vehicles has made rapid progress. In closed scenarios represented by highways and open scenarios represented by urban roads, L2+ level intelligent assisted driving has been put into practical application, and higher levels of autonomous driving and unmanned driving are the development trend of automobiles. At present, there are still many problems to be solved before autonomous driving technology can develop to a fully mature level, and the autonomous driving system still has shortcomings. In the foreseeable future, for a considerable period of time, in actual operation, vehicles equipped with intelligent assisted driving or autonomous driving may have problems such as complex external traffic environment, vehicle sensor failure, network signal disconnection and artificial intelligence judgment error, and there is an objective need for temporary manual takeover of driving. Therefore, it is of great significance to judge the risks of the autonomous driving system during operation and prompt the driver to take over the vehicle manually in time. Since autonomous driving has the characteristic of not requiring driver intervention, drivers are often distracted to a high degree and focus on other matters such as watching movies, chatting with passengers or using mobile phones. The intensity of traditional sound and light reminders is difficult to control. Too strong sound and light reminders are prone to cause additional interference, and too weak sound and light reminders are prone to low warning efficiency. In addition, when operating low-level assisted driving such as L2+, the driver is required to hold the steering wheel, but in reality, there are cases where the driver's hands are off the steering wheel, and the warning method based on the vibration of the steering wheel is easily ignored. Therefore, in view of the potential risks of the current autonomous driving system, the development of a body-contact safety warning system based on the vibration of the seat belt and the seat is conducive to sufficient warning and provides a basis for scientifically defining the responsibility for autonomous driving accidents. It is a technical direction with application prospects.

[0003] At present, the Chinese invention patent application with application number CN202411327794.2 discloses a safety warning method and system for vehicle automatic driving. The application includes: obtaining the surrounding environment information of the automatic driving vehicle, and determining the urgency of the vehicle according to the analysis of the surrounding environment information, wherein the surrounding environment information includes the type, position, speed, acceleration and moving direction of the obstacle; obtaining the driver's state information in the current preset time period, and determining the driver's current driving attention according to the analysis of the driver's state information; generating a safety degree according to the urgency of the vehicle and the current driving attention, and executing the corresponding safety prompt instruction according to the safety degree. The application determines whether to generate a safety prompt instruction based on the actual driving state of the driver and the urgency of the vehicle, provides a flexible prompt mode, reduces unnecessary prompts on the basis of fitting the actual driving state, and ensures driving safety. However, when the application warns the driver, it is necessary to calculate the trajectory according to the vehicle environment, the data processing complexity is high, and there is duplication with the vehicle's own automatic driving system function. At the same time, it does not fully consider the situation that the sound and light warning is easy to be ignored when the driver's attention is distracted during automatic driving, and lacks linkage with the safety protection measures for the driver. Summary of the invention

[0004] The technical problem solved by the present invention is that when the traditional method warns vehicles in autonomous driving state, it performs calculations and analyses on specific road condition images, lidar data, millimeter wave radar data, etc., which has high calculation complexity and duplicates the functions of the autonomous driving system. It does not fully consider the situation where the received complex sensor data such as cameras and radars have problems themselves. The intensity of traditional sound and light reminders is difficult to control and is easily ignored by the driver. At the same time, it lacks linkage with safety protection measures for drivers.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] A body contact safety warning system based on potential risks of autonomous driving, comprising: a data receiving module, a data processing module, a safety warning module and a terminal interaction module;

[0007] The data receiving module includes an image data receiving unit, a motion data receiving unit and a transmission data receiving unit, which are used to receive real-time images collected by the vehicle camera to obtain an image data set, receive a vehicle motion data set collected by the vehicle motion sensor, and receive communication transmission data transmitted from the communication transmission interface by the vehicle automatic driving system;

[0008] The data processing module includes an image data analysis unit, a motion data analysis unit and a transmission data analysis unit, which are used to calculate the image data set to obtain the visual risk warning degree, calculate the vehicle motion data set to obtain the predicted vehicle body stability, and analyze and calculate the communication transmission data to obtain the communication data packet distortion and special instruction data;

[0009] The safety warning module includes a classification warning unit and a physical vibration unit, which is used to classify and judge the warning result according to the visual risk warning degree, the predicted vehicle body stability, the communication data packet distortion degree and the special instruction data, and issue a safety warning to the driver according to the warning result;

[0010] The terminal interaction module includes an interaction unit and a storage unit, which are used to provide users with visual interaction pages and receive user operation instructions, and store the received data of the data receiving module, the output data of the data processing module, the output data of the safety warning module and the output data of the interaction unit data.

[0011] As a preferred solution of the body contact safety warning system based on potential risks of autonomous driving described in the present invention, wherein: the image data receiving unit is used to receive real-time images collected by a vehicle-mounted camera to obtain an image data set, and the vehicle-mounted camera includes a front-view camera, a rear-view camera, a surround-view camera and a side-view camera;

[0012] The motion data receiving unit is used to receive a vehicle motion data set acquired by a vehicle motion sensor, wherein the vehicle motion sensor includes a velocity sensor, an acceleration sensor, a gyroscope and a pressure sensor, and the vehicle motion data set includes vehicle velocity data, horizontal acceleration data, vertical acceleration data, rotational angular velocity data and vertical load force data;

[0013] The transmission data receiving unit is used to obtain the communication transmission data transmitted by the vehicle's automatic driving system from the communication transmission interface, the communication transmission interface includes a vehicle-mounted sensor data API, a vehicle condition information API, a map platform API and a software application support API, and the communication transmission data includes vehicle-mounted sensor transmission data, vehicle condition information transmission data, map platform transmission data and software application support transmission data.

[0014] As a preferred solution of the body contact safety warning system based on potential risks of autonomous driving described in the present invention, the image data analysis unit is used to calculate the image data set to obtain a visual risk warning degree, sample the image data set through a first sliding window with a preset K frame length to obtain a sampled image set, add corresponding timestamps to each frame image in the sampled image set to obtain an image sequence to be analyzed, perform grayscale mapping on the image sequence to be analyzed through a weighted average method to obtain a grayscale image to be analyzed, and calculate the visual risk warning degree through the grayscale image to be analyzed.

[0015] As a preferred solution of the body contact safety warning system based on potential risks of autonomous driving described in the present invention, the logic of calculating the visual risk warning degree by the grayscale image to be analyzed includes:

[0016] The grayscale image to be analyzed is subjected to Gaussian filtering to obtain a denoised image, the grayscale gradient amplitude of each pixel point is calculated by the Sobel operator, and each pixel point in the denoised image is assigned a value by the grayscale gradient amplitude to obtain a grayscale gradient feature image;

[0017] The grayscale image to be analyzed is calculated by a second sliding window with a preset size of 5*5 to obtain the grayscale mean of the neighborhood pixels, and the grayscale standard deviation of the neighborhood pixels is calculated based on the grayscale mean of the neighborhood pixels. The grayscale standard deviation of the neighborhood pixels is used to assign values ​​to each pixel in the grayscale image to be analyzed to obtain a grayscale discreteness feature image;

[0018] The grayscale discreteness feature image is used to perform weighted linear mapping on the corresponding pixels in the grayscale gradient feature image to obtain an abnormal feature map, and the pixel values ​​in the abnormal feature map are sorted from high to low. The top P% of pixel values ​​are selected based on the preset proportional coefficient P% and the average value is calculated to obtain the visual risk warning degree;

[0019] Its calculation expression is:

[0020]

[0021] Among them, SNV(i,j) represents the grayscale discreteness value of the pixel with coordinates (i,j), (m,n) represents the coordinates of the pixel in a 5*5 neighborhood centered on (i,j), I(m,n) represents the grayscale value of the pixel with coordinates (m,n), μ(i,j) represents the mean grayscale value of the neighborhood pixels centered on (i,j), DOA(m,n) represents the pixel value with coordinates (i,j) in the abnormal feature map, β represents, GFM(i,j) represents the gradient amplitude of the pixel with coordinates (i,j), x represents the total number of columns of pixels in the grayscale image to be analyzed, and y represents the total number of rows of pixels in the grayscale image to be analyzed.

[0022] As a preferred solution of the body contact safety warning system based on potential risks of autonomous driving described in the present invention, the motion data analysis unit is used to calculate the vehicle motion data set to obtain the predicted body stability, calculate the load transfer rate based on the horizontal acceleration data, vertical acceleration data, rotational angular velocity data and vertical load force data in the vehicle motion data set, calculate the body inclination angle by the load transfer rate, vehicle wheelbase and vehicle center of gravity height, derive the body inclination angle to obtain the first-order derivative and the second-order derivative of the body inclination angle, calculate the predicted inclination angle based on the preset critical time, the real-time body inclination angle, the first-order derivative and the second-order derivative of the body inclination angle, and calculate the predicted body stability through the stability model.

[0023] As a preferred solution of the body contact safety warning system based on potential risks of autonomous driving described in the present invention, the logic of predicting vehicle body stability calculated by the stability model includes:

[0024] When the second derivative of the vehicle body tilt angle is not equal to 0, the predicted vehicle body stability is obtained by calculating the predicted tilt angle, the vehicle body tilt angle, the first derivative of the vehicle body tilt angle, the second derivative of the vehicle body tilt angle and the preset critical time through the first piecewise sub-formula in the stability model;

[0025] When the second derivative of the vehicle body tilt angle is equal to 0 and the first derivative of the tilt angle is not equal to 0, the predicted vehicle body stability is obtained by calculating the predicted tilt angle, the vehicle body tilt angle, the first derivative of the vehicle body tilt angle, the second derivative of the vehicle body tilt angle and the preset critical time through the first piecewise sub-formula in the stability model;

[0026] When the second derivative of the vehicle body tilt angle is equal to 0 and the first derivative of the tilt angle is equal to 0, the vehicle body stability is predicted by predicting the tilt angle through the third piecewise sub-formula in the stability model;

[0027] Its calculation expression is:

[0028]

[0029]

[0030] Wherein, PSL represents predicted vehicle body stability, PTA represents predicted tilt angle, t represents time, A(t) represents vehicle body tilt angle, α1 represents the first weight, and represents the preset critical time. represents the first-order derivative of the vehicle body inclination angle, α2 represents the second weight, represents the second-order derivative of the vehicle body inclination angle, lg represents the logarithmic function with base 10, v represents the vehicle speed, and m represents the vehicle mass.

[0031] As a preferred solution of the body contact safety warning system based on potential risks of autonomous driving described in the present invention, the transmission data analysis unit is used to analyze and calculate the communication transmission data to obtain the communication data packet distortion, and decode and identify the communication transmission data to obtain special instruction data;

[0032] Based on the communication transmission interface and the data type identifier, the communication transmission data is read and matched with the instruction code to obtain special instruction data, wherein the special instruction data includes an automatic driving level reduction instruction, a local map failure instruction, an emergency avoidance instruction, a planned path change instruction, a support application update instruction, and an inter-vehicle collaboration instruction, and the communication transmission interface includes an on-board sensor data API, a vehicle condition information API, a map platform API, and a software application support API;

[0033] The processing logic for analyzing and calculating the communication transmission data to obtain the communication data packet distortion includes: calculating the received signal-to-noise ratio by preset communication evaluation time, communication transmission data signal power and noise power, calculating the signal strength fluctuation variance corresponding to the communication transmission data by preset communication evaluation time and received signal strength indication, calculating the average data packet delay rate by preset communication evaluation time, data packet receiving timestamp and data packet sending timestamp of communication transmission data, calculating the data packet integrity by preset communication evaluation time, total number of data packets sent for communication transmission data and total number of data packets received for communication transmission data, and calculating the communication data packet distortion by receiving signal-to-noise ratio, signal strength fluctuation variance, average data packet delay rate and data packet integrity, and the calculation expression is:

[0034] DCE = γ1(RSN-RSN TH )+γ2(VSF-VSF TH )+γ3(APD-APD TH )+γ4(COP

[0035] -COP TH )

[0036] Among them, DCE represents the communication data packet distortion, RSN represents the received signal-to-noise ratio, and RSN TH represents the received signal-to-noise ratio threshold, γ1 represents the received signal-to-noise ratio coefficient, VSF represents the signal strength fluctuation variance, VSF TH represents the signal strength fluctuation variance threshold, γ2 represents the signal strength fluctuation variance coefficient, APD represents the average packet delay rate, APD TH represents the average packet delay rate threshold, γ3 represents the average packet delay rate coefficient, COP represents the packet integrity, COP TH represents the packet integrity threshold, and γ4 represents the packet integrity coefficient.

[0037] As a preferred solution of the body contact safety warning system based on potential risks of autonomous driving described in the present invention, the classification warning unit is used to perform classification judgment according to the visual risk warning degree, the predicted vehicle body stability, the communication data packet distortion degree and the special instruction data to obtain the warning result. The calculation logic for performing classification judgment to obtain the warning result includes:

[0038] When the visual risk warning degree is less than the preset first visual risk warning degree threshold, the warning result is that no warning is required;

[0039] When the visual risk warning degree is greater than or equal to the preset first threshold and less than the preset second visual risk warning degree threshold, the warning result is a first-level warning;

[0040] When the visual risk warning degree is greater than the preset second visual risk warning degree threshold, the warning result is a second-level warning;

[0041] When the predicted vehicle body stability is less than a preset first predicted vehicle body stability threshold, the warning result is obtained as no warning is needed;

[0042] When the predicted vehicle body stability is greater than or equal to a preset first predicted vehicle body stability threshold and less than a preset second predicted vehicle body stability threshold, the warning result is a level 2 warning;

[0043] When the predicted vehicle body stability is greater than a preset second predicted vehicle body stability threshold, the warning result is a level 3 warning;

[0044] When the communication data packet distortion is less than or equal to a preset first communication data packet distortion threshold, the warning result is that no warning is required;

[0045] When the communication data packet distortion is greater than or equal to a preset first communication data packet distortion threshold and less than a preset second communication data packet distortion threshold, the warning result is a first-level warning;

[0046] When the communication data packet distortion is greater than a preset second communication data packet distortion threshold, the warning result is a level 2 warning;

[0047] When there are planned route change instructions, local map invalidation instructions, supported application update instructions and inter-vehicle collaboration instructions in the special instruction data, the warning result is a level 1 warning;

[0048] When there are automatic driving level reduction instructions and emergency avoidance instructions in the special instruction data, the warning result is a level 2 warning.

[0049] As a preferred solution of the body contact safety warning system based on potential risks of autonomous driving described in the present invention, the physical vibration unit is used to issue a safety warning to the driver according to the warning result output by the classification warning unit, and the safety warning adopts a vibration reminder and a tension reminder;

[0050] When the warning result received is that no warning is needed, the physical vibration unit remains on standby;

[0051] When the warning result received is a first-level warning, the physical vibration unit starts the primary vibration warning mode, the main driver's seat belt is tightened based on the first force, the main driver's seat belt and the vibration belt in the seat start to vibrate based on the primary vibration intensity and primary vibration frequency, vibrate for T1 seconds and then stop vibrating for T2 seconds, after repeating N times, the vibration belt on the seat belt and the vibration belt in the seat automatically stops vibrating, and the seat belt stops tightening;

[0052] When the warning result received is a level 2 warning, the physical vibration unit starts the advanced vibration warning mode, all seat belts are tightened based on the second force, the driver's seat belt and the vibration belt in the seat start to vibrate continuously based on the advanced vibration intensity and advanced vibration frequency, and the steering wheel grip is collected through the grip sensor in the steering wheel. When the steering wheel grip is greater than or equal to the preset grip threshold, the seat belt and the vibration belt in the seat stop vibrating and the seat belt stops tightening;

[0053] When the warning result received is a level 3 warning, all seat belts are tightened based on the third force until the driver manually inputs a warning shutdown command or the seat belts stop tightening when the level 3 warning ends.

[0054] As a preferred solution of the physical contact safety warning system based on potential risks of autonomous driving described in the present invention, the interaction unit is used to provide a visual interaction page to the user and receive user operation instructions, and the user operation instructions include a warning closing instruction, a system preset parameter setting instruction, and a historical record data query instruction;

[0055] The storage unit is used to store the received data of the data receiving module, the output data of the data processing module, the output data of the safety warning module and the output data of the interactive unit data.

[0056] Beneficial effects of the present invention: The present invention improves the direct acquisition of data from the camera terminal, the sensor terminal and the vehicle-mounted communication terminal, and obtains image data when the data is transmitted to the vehicle automatic driving system, ensuring that the warning system of the present invention receives the same data set as the vehicle's own automatic driving system, which is conducive to accurate analysis. The present invention does not analyze the road conditions for specific image content, but focuses on judging the image quality received by the automatic driving system. By calculating the visual risk warning degree, the image data error, image blur, low visibility of the vehicle environment and sudden change of light intensity are fully considered, and the calculation complexity is effectively reduced, and the timeliness of the warning is improved. By calculating and predicting the stability of the vehicle body, it is conducive to accurate quantitative evaluation of the stability of the vehicle, and by presetting the critical time, sufficient reaction time is reserved for the driver, which improves the robustness of the automatic driving system in complex environments and abnormal conditions. Transmission data analysis is analyzed from two aspects: the distortion of the communication data packet and the identification of special instruction data, which is conducive to accurately identifying the key instructions of the automatic driving system to change the vehicle condition, and comprehensively evaluating the communication status to timely discover the potential risks related to communication anomalies. Through the graded warning mechanism, the first-level warning, the second-level warning and the third-level warning are carried out for different risk levels, and the corresponding primary vibration warning mode and advanced vibration warning mode are started. At the same time, the vibration of the seat belt can attract the driver's attention in time without interfering with the driver's normal vision and operation. Compared with traditional sound and light reminders, it is more targeted and effective. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A basic flow chart of a body contact safety warning system based on potential risks of autonomous driving is provided as an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0059] Reference Figure 1 , is an embodiment of the present invention, and provides a body contact safety warning system based on potential risks of autonomous driving, including: a data receiving module, a data processing module, a safety warning module and a terminal interaction module;

[0060] The data receiving module includes an image data receiving unit, a motion data receiving unit and a transmission data receiving unit, which are used to receive real-time images collected by the vehicle camera to obtain an image data set, receive a vehicle motion data set collected by the vehicle motion sensor, and receive communication transmission data transmitted from the communication transmission interface by the vehicle automatic driving system;

[0061] The data processing module includes an image data analysis unit, a motion data analysis unit and a transmission data analysis unit, which are used to calculate the image data set to obtain the visual risk warning degree, calculate the vehicle motion data set to obtain the predicted vehicle body stability, and analyze and calculate the communication transmission data to obtain the communication data packet distortion and special instruction data;

[0062] The safety warning module includes a classification warning unit and a physical vibration unit, which is used to classify and judge the warning result according to the visual risk warning degree, the predicted vehicle body stability, the communication data packet distortion degree and the special instruction data, and issue a safety warning to the driver according to the warning result;

[0063] The terminal interaction module includes an interaction unit and a storage unit, which are used to provide users with visual interaction pages and receive user operation instructions, and store the received data of the data receiving module, the output data of the data processing module, the output data of the safety warning module and the output data of the interaction unit data.

[0064] In this embodiment, it is characterized in that: the image data receiving unit is used to receive real-time images collected by vehicle-mounted cameras to obtain image data sets, and the vehicle-mounted cameras include a front-view camera, a rear-view camera, a surround-view camera and a side-view camera;

[0065] The motion data receiving unit is used to receive a vehicle motion data set acquired by a vehicle motion sensor, wherein the vehicle motion sensor includes a velocity sensor, an acceleration sensor, a gyroscope and a pressure sensor, and the vehicle motion data set includes vehicle velocity data, horizontal acceleration data, vertical acceleration data, rotational angular velocity data and vertical load force data;

[0066] The transmission data receiving unit is used to obtain the communication transmission data transmitted by the vehicle automatic driving system from the communication transmission interface, the communication transmission interface includes the vehicle sensor data API, the vehicle condition information API, the map platform API and the software application support API, and the communication transmission data includes the vehicle sensor transmission data, the vehicle condition information transmission data, the map platform transmission data and the software application support transmission data;

[0067] Embodiment: When the vehicle's automatic driving system is running, there is a situation where the camera terminal, sensor terminal and on-board communication terminal are operating normally, but a transmission line failure occurs, resulting in data errors. The vehicle's automatic driving system receives erroneous data, and then performs calculations and processing according to the erroneous data, and there is a need for early warning. Traditional warning systems directly collect data through terminals. In this case, it is easy to obtain results that do not require warnings, which is inconsistent with the actual situation and causes the problem of missed warnings. The present invention changes the method of directly obtaining data from camera terminals, sensor terminals and on-board communication terminals, and instead obtains image data when data is transmitted to the vehicle's automatic driving system, ensuring that the warning system of the present invention and the vehicle's automatic driving system receive the same image data set, vehicle motion data set and transmission data set.

[0068] In this embodiment, the image data analysis unit is used to calculate the image data set to obtain a visual risk warning degree, sample the image data set through a first sliding window with a preset K frame length to obtain a sampled image set, add corresponding timestamps to each frame image in the sampled image set to obtain an image sequence to be analyzed, perform grayscale mapping on the image sequence to be analyzed through a weighted average method to obtain a grayscale image to be analyzed, and calculate the visual risk warning degree through the grayscale image to be analyzed.

[0069] Embodiment: The sampled image set includes images of K consecutive frames from the current frame image to the previous K-1 frame image, which is conducive to analyzing the dynamic changes in the quality of the acquired images.

[0070] In this embodiment, the logic of calculating the visual risk warning degree through the grayscale image to be analyzed includes:

[0071] The grayscale image to be analyzed is subjected to Gaussian filtering to obtain a denoised image, the grayscale gradient amplitude of each pixel point is calculated by the Sobel operator, and each pixel point in the denoised image is assigned a value by the grayscale gradient amplitude to obtain a grayscale gradient feature image;

[0072] The grayscale image to be analyzed is calculated by a second sliding window with a preset size of 5*5 to obtain the grayscale mean of the neighborhood pixels, and the grayscale standard deviation of the neighborhood pixels is calculated based on the grayscale mean of the neighborhood pixels. The grayscale standard deviation of the neighborhood pixels is used to assign values ​​to each pixel in the grayscale image to be analyzed to obtain a grayscale discreteness feature image;

[0073] Among them, the calculation expression for grayscale mapping of the environment image by weighted average method is:

[0074] Gray=0.299R+0.587G+0.114B

[0075] Among them, Gray represents the gray value of the pixel after grayscale mapping, R represents the red component of the pixel before grayscale mapping, G represents the green component of the pixel before grayscale mapping, and B represents the blue component of the pixel before grayscale mapping.

[0076] The grayscale discreteness feature image is used to perform weighted linear mapping on the corresponding pixels in the grayscale gradient feature image to obtain an abnormal feature map, and the pixel values ​​in the abnormal feature map are sorted from high to low. The top P% of pixel values ​​are selected based on the preset proportional coefficient P% and the average value is calculated to obtain the visual risk warning degree;

[0077] Its calculation expression is:

[0078]

[0079] Among them, SNV(i,j) represents the grayscale discreteness value of the pixel with coordinates (i,j), (m,n) represents the coordinates of the pixel in a 5*5 neighborhood centered on (i,j), I(m,n) represents the grayscale value of the pixel with coordinates (m,n), μ(i,j) represents the mean grayscale value of the neighborhood pixels centered on (i,j), DOA(m,n) represents the pixel value with coordinates (i,j) in the abnormal feature map, β represents, GFM(i,j) represents the gradient amplitude of the pixel with coordinates (i,j), x represents the total number of columns of pixels in the grayscale image to be analyzed, and y represents the total number of rows of pixels in the grayscale image to be analyzed.

[0080] Image analysis is a key technology for autonomous driving systems to judge road conditions, and it relies on the support of real-time high-definition images. The present invention does not analyze road conditions based on specific image content, but focuses on judging the quality of images received by the autonomous driving system. By calculating the visual risk warning degree, it repeatedly considers issues such as image data errors, image blur, low visibility of the vehicle environment, and sudden changes in light intensity, and effectively reduces the computational complexity, thereby improving the timeliness of the warning.

[0081] In this embodiment, the motion data analysis unit is used to calculate the vehicle motion data set to obtain a predicted vehicle body stability, calculate the load transfer rate based on the horizontal acceleration data, vertical acceleration data, rotational angular velocity data and vertical load force data in the vehicle motion data set, calculate the vehicle body inclination angle through the load transfer rate, the vehicle wheelbase and the vehicle center of gravity height, derivative the vehicle body inclination angle to obtain the first-order derivative and the second-order derivative of the vehicle body inclination angle, calculate the predicted inclination angle based on the preset critical time, the real-time vehicle body inclination angle, the first-order derivative and the second-order derivative of the vehicle body inclination angle, and calculate the predicted vehicle body stability through the stability model.

[0082] In this embodiment, the logic of calculating the predicted vehicle body stability through the stability model includes:

[0083] When the second derivative of the vehicle body tilt angle is not equal to 0, the predicted vehicle body stability is obtained by calculating the predicted tilt angle, the vehicle body tilt angle, the first derivative of the vehicle body tilt angle, the second derivative of the vehicle body tilt angle and the preset critical time through the first piecewise sub-formula in the stability model;

[0084] When the second derivative of the vehicle body tilt angle is equal to 0 and the first derivative of the tilt angle is not equal to 0, the predicted vehicle body stability is obtained by calculating the predicted tilt angle, the vehicle body tilt angle, the first derivative of the vehicle body tilt angle, the second derivative of the vehicle body tilt angle and the preset critical time through the first piecewise sub-formula in the stability model;

[0085] When the second derivative of the vehicle body tilt angle is equal to 0 and the first derivative of the tilt angle is equal to 0, the vehicle body stability is predicted by predicting the tilt angle through the third piecewise sub-formula in the stability model;

[0086] Its calculation expression is:

[0087]

[0088] Wherein, PSL represents predicted vehicle body stability, PTA represents predicted tilt angle, t represents time, A(t) represents vehicle body tilt angle, α1 represents the first weight, and represents the preset critical time. represents the first-order derivative of the vehicle body inclination angle, α2 represents the second weight, represents the second-order derivative of the vehicle body inclination angle, lg represents the logarithmic function with base 10, v represents the vehicle speed, and m represents the vehicle mass.

[0089] Among them, calculating and predicting the vehicle body stability is conducive to accurate quantitative evaluation of vehicle stability. By presetting the critical time, sufficient reaction time is reserved for the driver, which improves the robustness of the autonomous driving system in complex environments and abnormal situations.

[0090] In this embodiment, the transmission data analysis unit is used to analyze and calculate the communication transmission data to obtain the communication data packet distortion, and decode and identify the communication transmission data to obtain the special instruction data;

[0091] Based on the communication transmission interface and the data type identifier, the communication transmission data is read and matched with the instruction code to obtain special instruction data, wherein the special instruction data includes an automatic driving level reduction instruction, a local map failure instruction, an emergency avoidance instruction, a planned path change instruction, a support application update instruction, and an inter-vehicle collaboration instruction, and the communication transmission interface includes an on-board sensor data API, a vehicle condition information API, a map platform API, and a software application support API;

[0092] The processing logic for analyzing and calculating the communication transmission data to obtain the communication data packet distortion includes: calculating the received signal-to-noise ratio by preset communication evaluation time, communication transmission data signal power and noise power, calculating the signal strength fluctuation variance corresponding to the communication transmission data by preset communication evaluation time and received signal strength indication, calculating the average data packet delay rate by preset communication evaluation time, data packet receiving timestamp and data packet sending timestamp of communication transmission data, calculating the data packet integrity by preset communication evaluation time, total number of data packets sent for communication transmission data and total number of data packets received for communication transmission data, and calculating the communication data packet distortion by receiving signal-to-noise ratio, signal strength fluctuation variance, average data packet delay rate and data packet integrity, and the calculation expression is:

[0093] DCE = γ1(RSN-RSN TH )+γ2(VSF-VSF TH )+γ3(APD-APD TH )+γ4(COP

[0094] -COP TH )

[0095] Among them, DCE represents the communication data packet distortion, RSN represents the received signal-to-noise ratio, and RSN TH represents the received signal-to-noise ratio threshold, γ1 represents the received signal-to-noise ratio coefficient, VSF represents the signal strength fluctuation variance, VSF TH represents the signal strength fluctuation variance threshold, γ2 represents the signal strength fluctuation variance coefficient, APD represents the average packet delay rate, APD TH represents the average packet delay rate threshold, γ3 represents the average packet delay rate coefficient, COP represents the packet integrity, COP TH represents the packet integrity threshold, and γ4 represents the packet integrity coefficient.

[0096] Among them, the transmission data analysis is carried out from two aspects: the distortion of communication data packets and the identification of special command data. This is conducive to accurately identifying the key commands of the autonomous driving system to change the vehicle condition, while comprehensively evaluating the communication status to promptly detect communication anomalies.

[0097] In this embodiment, the classification warning unit is used to perform classification judgment according to the visual risk warning degree, the predicted vehicle body stability, the communication data packet distortion degree and the special instruction data to obtain the warning result. The calculation logic for performing classification judgment to obtain the warning result includes:

[0098] When the visual risk warning degree is less than the preset first visual risk warning degree threshold, the warning result is that no warning is required;

[0099] When the visual risk warning degree is greater than or equal to the preset first threshold and less than the preset second visual risk warning degree threshold, the warning result is a first-level warning;

[0100] When the visual risk warning degree is greater than the preset second visual risk warning degree threshold, the warning result is a second-level warning;

[0101] When the predicted vehicle body stability is less than a preset first predicted vehicle body stability threshold, the warning result is obtained as no warning is needed;

[0102] When the predicted vehicle body stability is greater than or equal to a preset first predicted vehicle body stability threshold and less than a preset second predicted vehicle body stability threshold, the warning result is a level 2 warning;

[0103] When the predicted vehicle body stability is greater than a preset second predicted vehicle body stability threshold, the warning result is a level 3 warning;

[0104] When the communication data packet distortion is less than or equal to a preset first communication data packet distortion threshold, the warning result is that no warning is required;

[0105] When the communication data packet distortion is greater than or equal to a preset first communication data packet distortion threshold and less than a preset second communication data packet distortion threshold, the warning result is a first-level warning;

[0106] When the communication data packet distortion is greater than a preset second communication data packet distortion threshold, the warning result is a level 2 warning;

[0107] When there are planned route change instructions, local map invalidation instructions, supported application update instructions and inter-vehicle collaboration instructions in the special instruction data, the warning result is a level 1 warning;

[0108] When there are automatic driving level reduction instructions and emergency avoidance instructions in the special instruction data, the warning result is a level 2 warning.

[0109] In this embodiment, the physical vibration unit is used to issue a safety warning to the driver according to the warning result output by the classification warning unit, and the safety warning takes the form of a vibration reminder and a tension reminder;

[0110] When the warning result received is that no warning is needed, the physical vibration unit remains on standby;

[0111] When the warning result received is a first-level warning, the physical vibration unit starts the primary vibration warning mode, the main driver's seat belt is tightened based on the first force, the main driver's seat belt and the vibration belt in the seat start to vibrate based on the primary vibration intensity and primary vibration frequency, vibrate for T1 seconds and then stop vibrating for T2 seconds, after repeating N times, the vibration belt on the seat belt and the vibration belt in the seat automatically stops vibrating, and the seat belt stops tightening;

[0112] When the warning result received is a level 2 warning, the physical vibration unit starts the advanced vibration warning mode, all seat belts are tightened based on the second force, the driver's seat belt and the vibration belt in the seat start to vibrate continuously based on the advanced vibration intensity and advanced vibration frequency, and the steering wheel grip is collected through the grip sensor in the steering wheel. When the steering wheel grip is greater than or equal to the preset grip threshold, the seat belt and the vibration belt in the seat stop vibrating and the seat belt stops tightening;

[0113] When the warning result received is a level 3 warning, all seat belts are tightened based on the third force until the driver manually inputs a warning shutdown command or the seat belts stop tightening when the level 3 warning ends.

[0114] Among them, the graded warning mechanism is used to carry out the first-level and second-level warnings for different risk levels, and the corresponding primary vibration warning mode and advanced vibration warning mode are activated. At the same time, the seat belt vibration can attract the driver's attention in time without interfering with the driver's normal vision and operation, which is more targeted and effective than the traditional sound and light reminder method. A three-level warning is added to tighten the seat belts of all seats to prevent vehicle impact, and this state will continue until the end of the third-level warning state, which is conducive to protecting the safety of drivers and passengers in the event of rollover or collision.

[0115] In this embodiment, the interaction unit is used to provide a visual interaction page to the user and receive user operation instructions, wherein the user operation instructions include an instruction to close the warning, an instruction to set the system preset parameters, and an instruction to query the historical record data;

[0116] The storage unit is used to store the received data of the data receiving module, the output data of the data processing module, the output data of the safety warning module and the output data of the interactive unit data.

[0117] It should be understood by those skilled in the art that embodiments of the present invention may provide methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes. Among them, the storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk. These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0118] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A physical contact safety warning system based on potential risks of autonomous driving, characterized in that: include: Data receiving module, data processing module, safety warning module and terminal interaction module; The data receiving module includes an image data receiving unit, a motion data receiving unit and a transmission data receiving unit, which are used to receive real-time images collected by the vehicle camera to obtain an image data set, receive a vehicle motion data set collected by the vehicle motion sensor, and receive communication transmission data transmitted from the communication transmission interface by the vehicle automatic driving system; The data processing module includes an image data analysis unit, a motion data analysis unit and a transmission data analysis unit, which are used to calculate the image data set to obtain the visual risk warning degree, calculate the vehicle motion data set to obtain the predicted vehicle body stability, and analyze and calculate the communication transmission data to obtain the communication data packet distortion and special instruction data; The safety warning module includes a classification warning unit and a physical vibration unit, which is used to classify and judge the warning result according to the visual risk warning degree, the predicted vehicle body stability, the communication data packet distortion degree and the special instruction data, and issue a safety warning to the driver according to the warning result; The terminal interaction module includes an interaction unit and a storage unit, which are used to provide users with visual interaction pages and receive user operation instructions, and store the received data of the data receiving module, the output data of the data processing module, the output data of the safety warning module and the output data of the interaction unit data.

2. A body contact safety warning system based on potential risks of autonomous driving as claimed in claim 1, characterized in that: The image data receiving unit is used to receive real-time images collected by vehicle-mounted cameras to obtain image data sets, wherein the vehicle-mounted cameras include a front-view camera, a rear-view camera, a surround-view camera, and a side-view camera; The motion data receiving unit is used to receive a vehicle motion data set acquired by a vehicle motion sensor, wherein the vehicle motion sensor includes a velocity sensor, an acceleration sensor, a gyroscope and a pressure sensor, and the vehicle motion data set includes vehicle velocity data, horizontal acceleration data, vertical acceleration data, rotational angular velocity data and vertical load force data; The transmission data receiving unit is used to obtain the communication transmission data transmitted by the vehicle's automatic driving system from the communication transmission interface, the communication transmission interface includes a vehicle-mounted sensor data API, a vehicle condition information API, a map platform API and a software application support API, and the communication transmission data includes vehicle-mounted sensor transmission data, vehicle condition information transmission data, map platform transmission data and software application support transmission data.

3. A body contact safety warning system based on potential risks of autonomous driving as claimed in claim 1, characterized in that: The image data analysis unit is used to calculate the visual risk warning degree for the image data set, sample the image data set through a first sliding window with a preset K frame length to obtain a sampled image set, add corresponding timestamps to each frame image in the sampled image set to obtain an image sequence to be analyzed, perform grayscale mapping on the image sequence to be analyzed through a weighted average method to obtain a grayscale image to be analyzed, and calculate the visual risk warning degree through the grayscale image to be analyzed.

4. A body contact safety warning system based on potential risks of autonomous driving as claimed in claim 3, characterized in that: The logic of calculating the visual risk warning degree through the grayscale image to be analyzed includes: The grayscale image to be analyzed is subjected to Gaussian filtering to obtain a denoised image, the grayscale gradient amplitude of each pixel point is calculated by the Sobel operator, and each pixel point in the denoised image is assigned a value by the grayscale gradient amplitude to obtain a grayscale gradient feature image; The grayscale image to be analyzed is calculated by a second sliding window with a preset size of 5*5 to obtain the grayscale mean of the neighborhood pixels, and the grayscale standard deviation of the neighborhood pixels is calculated based on the grayscale mean of the neighborhood pixels. The grayscale standard deviation of the neighborhood pixels is used to assign values ​​to each pixel in the grayscale image to be analyzed to obtain a grayscale discreteness feature image; The grayscale discreteness feature image is used to perform weighted linear mapping on the corresponding pixels in the grayscale gradient feature image to obtain an abnormal feature map, and the pixel values ​​in the abnormal feature map are sorted from high to low. The top P% of pixel values ​​are selected based on the preset proportional coefficient P% and the average value is calculated to obtain the visual risk warning degree; Its calculation expression is: Among them, SNV(i,j) represents the grayscale discreteness value of the pixel with coordinates (i,j), (m,n) represents the coordinates of the pixel in a 5*5 neighborhood centered on (i,j), I(m,n) represents the grayscale value of the pixel with coordinates (m,n), μ(i,j) represents the mean grayscale value of the neighborhood pixels centered on (i,j), DOA(m,n) represents the pixel value with coordinates (i,j) in the abnormal feature map, β represents, GFM(i,j) represents the gradient amplitude of the pixel with coordinates (i,j), x represents the total number of columns of pixels in the grayscale image to be analyzed, and y represents the total number of rows of pixels in the grayscale image to be analyzed.

5. The body contact safety warning system based on potential risks of autonomous driving as claimed in claim 1, characterized in that: The motion data analysis unit is used to calculate the vehicle motion data set to obtain the predicted body stability, calculate the load transfer rate based on the horizontal acceleration data, vertical acceleration data, rotational angular velocity data and vertical load force data in the vehicle motion data set, calculate the body inclination angle through the load transfer rate, vehicle wheelbase and vehicle center of gravity height, derive the body inclination angle to obtain the first-order derivative of the body inclination angle and the second-order derivative of the body inclination angle, calculate the predicted inclination angle based on the preset critical time, the real-time body inclination angle, the first-order derivative of the body inclination angle and the second-order derivative of the body inclination angle, and calculate the predicted body stability through the stability model.

6. A body contact safety warning system based on potential risks of autonomous driving as claimed in claim 5, characterized in that: The logic of predicting vehicle body stability through stability model calculation includes: When the second derivative of the vehicle body tilt angle is not equal to 0, the predicted vehicle body stability is obtained by calculating the predicted tilt angle, the vehicle body tilt angle, the first derivative of the vehicle body tilt angle, the second derivative of the vehicle body tilt angle and the preset critical time through the first piecewise sub-formula in the stability model; When the second derivative of the vehicle body tilt angle is equal to 0 and the first derivative of the tilt angle is not equal to 0, the predicted vehicle body stability is obtained by calculating the predicted tilt angle, the vehicle body tilt angle, the first derivative of the vehicle body tilt angle, the second derivative of the vehicle body tilt angle and the preset critical time through the first piecewise sub-formula in the stability model; When the second derivative of the vehicle body tilt angle is equal to 0 and the first derivative of the tilt angle is equal to 0, the vehicle body stability is predicted by predicting the tilt angle through the third piecewise sub-formula in the stability model; Its calculation expression is: Wherein, PSL represents predicted vehicle body stability, PTA represents predicted tilt angle, t represents time, A(t) represents vehicle body tilt angle, α1 represents the first weight, and represents the preset critical time. represents the first-order derivative of the vehicle body inclination angle, α2 represents the second weight, represents the second-order derivative of the vehicle body inclination angle, lg represents the logarithmic function with base 10, v represents the vehicle speed, and m represents the vehicle mass.

7. The body contact safety warning system based on potential risks of autonomous driving as claimed in claim 1, characterized in that: The transmission data analysis unit is used to analyze and calculate the communication transmission data to obtain the communication data packet distortion, and decode and identify the communication transmission data to obtain special instruction data; Based on the communication transmission interface and the data type identifier, the communication transmission data is read and matched with the instruction code to obtain special instruction data, wherein the special instruction data includes an automatic driving level reduction instruction, a local map failure instruction, an emergency avoidance instruction, a planned path change instruction, a support application update instruction, and an inter-vehicle collaboration instruction; The communication transmission interface includes vehicle sensor data API, vehicle condition information API, map platform API and software application support API; The processing logic for analyzing and calculating the communication transmission data to obtain the communication data packet distortion includes: calculating the received signal-to-noise ratio by preset communication evaluation time, communication transmission data signal power and noise power, calculating the signal strength fluctuation variance corresponding to the communication transmission data by preset communication evaluation time and received signal strength indication, calculating the average data packet delay rate by preset communication evaluation time, data packet receiving timestamp and data packet sending timestamp of communication transmission data, calculating the data packet integrity by preset communication evaluation time, total number of data packets sent for communication transmission data and total number of data packets received for communication transmission data, and calculating the communication data packet distortion by receiving signal-to-noise ratio, signal strength fluctuation variance, average data packet delay rate and data packet integrity, and the calculation expression is: DCE=γ1(RSN-RSN TH )+γ2(VSF-VSF TH )+γ3(APD-APD TH )+γ4(COP -COP TH ) Among them, DCE represents the communication data packet distortion, RSN represents the received signal-to-noise ratio, and RSN TH represents the received signal-to-noise ratio threshold, γ1 represents the received signal-to-noise ratio coefficient, VSF represents the signal strength fluctuation variance, VSF TH represents the signal strength fluctuation variance threshold, γ2 represents the signal strength fluctuation variance coefficient, APD represents the average packet delay rate, APD TH represents the average packet delay rate threshold, γ3 represents the average packet delay rate coefficient, COP represents the packet integrity, COP TH represents the packet integrity threshold, and γ4 represents the packet integrity coefficient.

8. The body contact safety warning system based on potential risks of autonomous driving as claimed in claim 1, characterized in that: The classification warning unit is used to classify and judge the warning results according to the visual risk warning degree, the predicted vehicle body stability, the communication data packet distortion and the special instruction data. The calculation logic for classifying and judging the warning results includes: When the visual risk warning degree is less than the preset first visual risk warning degree threshold, the warning result is that no warning is required; When the visual risk warning degree is greater than or equal to the preset first threshold and less than the preset second visual risk warning degree threshold, the warning result is a first-level warning; When the visual risk warning degree is greater than the preset second visual risk warning degree threshold, the warning result is a second-level warning; When the predicted vehicle body stability is less than a preset first predicted vehicle body stability threshold, the warning result is obtained as no warning is needed; When the predicted vehicle body stability is greater than or equal to a preset first predicted vehicle body stability threshold and less than a preset second predicted vehicle body stability threshold, the warning result is a level 2 warning; When the predicted vehicle body stability is greater than a preset second predicted vehicle body stability threshold, the warning result is a level 3 warning; When the communication data packet distortion is less than or equal to a preset first communication data packet distortion threshold, the warning result is that no warning is required; When the communication data packet distortion is greater than or equal to a preset first communication data packet distortion threshold and less than a preset second communication data packet distortion threshold, the warning result is a first-level warning; When the communication data packet distortion is greater than a preset second communication data packet distortion threshold, the warning result is a level 2 warning; When there are planned route change instructions, local map invalidation instructions, supported application update instructions and inter-vehicle collaboration instructions in the special instruction data, the warning result is a level 1 warning; When there are automatic driving level reduction instructions and emergency avoidance instructions in the special instruction data, the warning result is a level 2 warning.

9. The body contact safety warning system based on potential risks of autonomous driving as claimed in claim 1, characterized in that: The physical vibration unit is used to issue a safety warning to the driver according to the warning result output by the classification warning unit. The safety warning takes the form of vibration reminder and tension reminder; When the warning result received indicates that no warning is needed, the physical vibration unit remains in standby mode; When the warning result received is a first-level warning, the physical vibration unit starts the primary vibration warning mode, the main driver's seat belt is tightened based on the first force, the main driver's seat belt and the vibration belt in the seat start to vibrate based on the primary vibration intensity and primary vibration frequency, vibrate for T1 seconds and then stop vibrating for T2 seconds, after repeating N times, the vibration belt on the seat belt and the vibration belt in the seat automatically stops vibrating, and the seat belt stops tightening; When the warning result received is a level 2 warning, the physical vibration unit starts the advanced vibration warning mode, all seat belts are tightened based on the second force, the driver's seat belt and the vibration belt in the seat start to vibrate continuously based on the advanced vibration intensity and advanced vibration frequency, and the steering wheel grip is collected through the grip sensor in the steering wheel. When the steering wheel grip is greater than or equal to the preset grip threshold, the seat belt and the vibration belt in the seat stop vibrating and the seat belt stops tightening; When the warning result received is a level 3 warning, all seat belts are tightened based on the third force until the driver manually inputs a warning shutdown command or the seat belts stop tightening when the level 3 warning ends.

10. The body contact safety warning system based on potential risks of autonomous driving as claimed in claim 1, characterized in that: The interactive unit is used to provide a visual interactive page to the user and receive user operation instructions, wherein the user operation instructions include instructions for closing warnings, instructions for setting system preset parameters, and instructions for querying historical record data; The storage unit is used to store the received data of the data receiving module, the output data of the data processing module, the output data of the safety warning module and the output data of the interactive unit data.

Images