A vehicle state monitoring and early warning method and system

Abstract

The present application belongs to the technical field of vehicle state detection and early warning, in particular to a vehicle state monitoring and early warning method and system, the monitoring and early warning method is: a sensor acquires real-time data, an abnormality detection algorithm is applied, and an abnormal event is marked; according to the abnormal severity and driving scene, the information priority is calculated and distributed to a high priority queue; a multi-modal early warning is generated, high-frequency sound and vibration are used when driving at high speed; sound prompts are extracted, voice waveform data is generated through a voice synthesis module; driver voice input is recognized, intent is analyzed, and abnormal information feedback is provided; according to the scene and abnormal state, the interactive output is optimized, detailed information is displayed when driving at low speed in congestion; the dashboard interface is dynamically adjusted, the key area is enlarged when the abnormality is serious; image and voice data are integrated, multi-modal signals are generated, and after rendering processing, transmission is performed. The present application can timely and accurately transmit vehicle abnormal information to the driver, effectively improving driving safety.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of vehicle state detection and early warning, and particularly relates to a vehicle state monitoring and early warning method and system. BACKGROUND

[0002] Vehicle state monitoring and early warning technology plays a crucial role in modern automobile industry, which is directly related to driving safety, vehicle performance maintenance, and user experience improvement. With the rapid development of intelligent driving and vehicle networking technology, vehicle state monitoring systems are expected to be more advanced, not only needing to perceive vehicle operating conditions in real time, but also providing intuitive and timely feedback to drivers through human-computer interaction interfaces. However, many current solutions still have obvious deficiencies in practical application, limiting their full play of functions.

[0003] Existing vehicle state monitoring systems often have limitations in information presentation and interaction methods. Many systems only provide single dashboard displays, with complex information and lack of hierarchical processing, making it difficult for drivers to quickly obtain key abnormal information. In addition, some systems rely only on a single prompt method, such as lights or text, which can be easily ignored, especially in complex driving environments, making it difficult to effectively attract the attention of drivers. At the same time, traditional systems lack flexibility in human-computer interaction, requiring manual operation when drivers query state information, increasing the risk of driving distraction.

[0004] These limitations reflect the core technical challenges faced by the field. First, dynamic hierarchical display of information is a difficult problem. How to dynamically adjust the display content according to the importance of the state, both to ensure the simplicity of daily driving and to provide detailed feedback in abnormal situations, has not been fully solved. Second, the integration of multi-modal warning mechanisms is difficult. A single prompt method is difficult to adapt to different driving scenarios, and a variety of methods such as sound and image need to be integrated to improve the warning effect. Finally, the intelligence level of voice interaction is insufficient. Existing systems perform poorly in terms of the accuracy of voice recognition and the real-time response of instruction, making it difficult to meet the needs of drivers for convenient inquiries while driving.

[0005] Therefore, how to design a system that can dynamically display vehicle state information, integrate multi-modal warning mechanisms, and support intelligent voice interaction to achieve efficient information transmission and low distraction operation for drivers has become a key problem in the field of vehicle state monitoring and early warning. SUMMARY

[0006] In view of the deficiencies of the prior art, the present application provides a vehicle state monitoring and early warning method and system, which can timely and accurately transmit vehicle abnormal information to drivers, effectively improving driving safety.

[0007] In order to achieve the above purpose, the technical scheme of the present application is:

[0008] A vehicle state monitoring and early warning method, the method comprising:

[0009] S1, obtaining real-time state data from vehicle sensors, preliminarily screening the data through a preset abnormality detection algorithm, marking as an abnormal event if the parameter exceeds the preset threshold, and obtaining an abnormal trend set;

[0010] S2, for the abnormal trend set, using a dynamic hierarchical display algorithm to calculate the information presentation priority according to the severity of the abnormality and the complexity of the driving scene, if the information presentation priority score is higher than the preset threshold, the abnormal information is assigned to the high priority display queue, and the dynamic hierarchical display content is determined;

[0011] S3, generating a multi-modal early warning signal according to the dynamic hierarchical display content, if the driving scene is high-speed driving, preferentially using high-frequency sound and vibration feedback, and obtaining a multi-modal early warning combination; the multi-modal early warning signal includes sound prompt, instrument panel image and tactile feedback;

[0012] S4, extracting the audio signal of the sound prompt part from the multi-modal early warning combination, and synthesizing the extracted audio signal through a speech synthesis module to generate speech waveform data with natural language description;

[0013] S5, obtaining the driver's voice input, performing semantic analysis on the input instruction through a speech recognition model, if the recognition result contains a state query intent, extracting abnormal information from the abnormal trend set, generating voice interaction response content, and determining interaction feedback information;

[0014] S6, according to the interaction feedback information, combining the current driving scene and the abnormal state, judging the scene type, if the scene is low-speed congestion, displaying detailed feedback content through the instrument panel, otherwise preferentially outputting concise information through voice, obtaining the optimized interaction output;

[0015] S7, extracting the display content from the optimized interaction output, dynamically adjusting the instrument panel interface through an image processing module, if the abnormal feedback detail is higher than the preset threshold, enlarging the key abnormal area, and generating the final display interface;

[0016] S8, obtaining the image and voice data of the final display interface, integrating the image and voice data into a unified data stream, obtaining a multi-modal signal, rendering the multi-modal signal, generating image and voice data stream adapted to the vehicle terminal, obtaining the rendered transmission signal, transmitting the rendered transmission signal to the vehicle terminal, and measuring the transmission delay, if the transmission delay is lower than the preset threshold, it is determined that the signal transmission meets the requirements.

[0017] Preferably, the real-time status data acquired from vehicle sensors is initially filtered using a preset anomaly detection algorithm. If a parameter exceeds a preset threshold, it is marked as an abnormal event, resulting in an abnormal trend set, including:

[0018] Acquire real-time data transmitted by vehicle sensors, including engine speed, brake pressure and tire pressure data, and remove noise through mean filtering to generate a standardized dataset;

[0019] Anomaly detection algorithms are used to initially screen parameters in a standardized dataset. If any parameter exceeds a preset threshold, it is marked as an anomalous event using a fixed threshold judgment method. The K-Means algorithm is then used to classify the anomalous events to obtain an anomalous event set.

[0020] For the set of anomalous events, the timestamps and parameter values ​​of each type of anomalous event are extracted to generate an anomalous feature dataset. The decision tree algorithm is then used to predict the development trend of anomalous events based on the anomalous feature dataset, resulting in an anomalous trend set.

[0021] Preferably, the severity of the anomaly includes the frequency of anomaly occurrence, the scope of impact, and the level of potential risk; the complexity of the driving scenario includes the current number of lanes, the distance to the vehicle ahead, and weather visibility.

[0022] Preferably, the specific method for determining the dynamically hierarchical display content includes:

[0023] S201. Based on the frequency of anomalies, the scope of impact, and the level of potential risk, a weighted scoring method is used to calculate the severity score of the anomaly. When calculating the severity score, the weight of the frequency of anomalies is set at 0.4, the weight of the scope of impact at 0.3, and the weight of the level of potential risk at 0.3. The formula for calculating the severity score is as follows: ,in, The score indicates the severity of the abnormality. Indicates the frequency of occurrence score. Indicates the scope of influence score. Indicates the potential risk level score;

[0024] S202. Calculate the driving scenario complexity score based on factors such as the current number of lanes, distance to vehicles ahead, and weather visibility. The calculation formula is as follows:

[0025]

[0026] Where C represents the driving scenario complexity score, Indicates the number of evaluation factors. Indicates the first The weights of each factor Indicates the first a score value of the factor;

[0027] S203, according to the abnormal severity score and the driving scene complexity score, the information presentation priority score is calculated by using the linear weighting method, when calculating, the weight of the abnormal severity score is 0.6, the weight of the driving scene complexity score is 0.4, and the calculation formula is represented as: , wherein, represents the information presentation priority score, represents the abnormal severity score, and C represents the driving scene complexity score;

[0028] S204, if the information presentation priority score is higher than the preset threshold value, the abnormal information is distributed to the high priority display queue, and the sliding window algorithm is used to update the display of the high priority display queue in real time, so as to ensure that the high priority information can be timely presented.

[0029] Preferably, the way of generating the multi-modal warning signal is:

[0030] The driving scene data is obtained from the vehicle-mounted millimeter wave radar and camera, including three characteristics of vehicle speed, acceleration and lane line curvature, the random forest classifier is used to judge whether the current scene is high-speed driving, and when the classification confidence is greater than the preset threshold value, it is determined that it is a high-speed scene;

[0031] In the high-speed scene, the sound signal frequency is set to 3000-5000Hz, and the vibration intensity is set to 0.5-1.2G as the priority signal parameter, and the initial warning signal set is generated according to the priority configuration, including the sound frequency, the vibration intensity and the duration three parameters;

[0032] The sound frequency and duration parameters are extracted from the initial warning signal set, and the image signal of the flashing red frame is generated by inputting the instrument panel display module;

[0033] The weighted average fusion algorithm is used to integrate the sound signal, the vibration signal and the image signal, when calculating, the sound weight is 0.4, the vibration weight is 0.3, and the image weight is 0.3; the weighted average fusion algorithm is represented as: , wherein represents the fused signal, represents the sound signal, represents the vibration signal, represents the image signal;

[0034] The fused signal is encoded into the CAN bus protocol format, and is output to the loudspeaker, the vibration motor and the display screen through the vehicle control unit, to generate the multi-modal warning signal.

[0035] Preferably, the audio signal of the sound prompt part is extracted from the multi-modal early warning combination, and the extracted audio signal is synthesized by a speech synthesis module to generate voice waveform data of natural language description, including:

[0036] The audio signal of the 20Hz-20kHz frequency band is intercepted from the multi-modal early warning combination to obtain a sound prompt signal, and the mel frequency cepstral coefficient of the sound prompt signal is calculated to obtain an audio feature vector;

[0037] The audio feature vector is input into a pre-trained support vector machine classifier to output an abnormal type label and an abnormal severity score, and if the abnormal type label is an emergency brake failure, the vehicle fault code is read and parsed through the OBD-II interface to obtain a fault location code;

[0038] The abnormal type label, the abnormal severity score and the fault location code are input into a speech synthesis module to generate voice waveform data of natural language description.

[0039] Preferably, according to the interactive feedback information, the current driving scene and the abnormal state are combined to determine the scene type, if the scene is a low-speed congestion, the detailed feedback content is displayed through the instrument panel, otherwise the concise information is preferentially output through the voice to obtain the optimized interactive output, including:

[0040] Real-time data is obtained from vehicle sensors and user interaction interfaces, driving environment and interactive feedback information are analyzed, and the feature description of the current state is determined;

[0041] According to the feature description of the current state, the driving environment and the abnormal state are classified by using a support vector machine algorithm according to a pre-established scene classification model to determine whether it belongs to a low-speed congestion scene;

[0042] If the classification result is a low-speed congestion scene, the detailed feedback content containing the specific description of the current state and the recommended operation is generated through the instrument panel display module to obtain a visual output result;

[0043] The specific description of the current state includes the current vehicle speed and the length of the congestion ahead, and the recommended operation includes the suggestion that the driver remains patient or chooses a detour route;

[0044] If the classification result does not belong to a low-speed congestion scene, the concise information containing the key prompt of the current state is generated through the speech synthesis module to obtain the voice output content;

[0045] For the generated detailed feedback content or concise information, the response data of the user interaction interface is obtained, the acceptance degree of the user to the output mode is analyzed, and the adjustment direction of the optimization scheme is determined;

[0046] According to the analysis result of the user response data, the priority and content form of the output mode are adjusted, an updated interaction scheme is generated, and the optimization processing of the scene adaptation is completed.

[0047] Preferably, the display content is extracted from the optimized interaction output, the dashboard interface is dynamically adjusted through the image processing module, if the abnormal feedback detail is higher than the preset threshold, the key abnormal information area is enlarged, and a final display interface is generated, including:

[0048] The original display content in JSON format is obtained from the interaction output, and the digital scale and pointer angle of the dashboard interface are extracted through the image processing module to generate structured display data containing numerical values and coordinates;

[0049] According to the numerical value range in the structured display data, the color contrast and scale line width of the dashboard are adjusted, a preliminary adjustment interface in bitmap format is output, the abnormal score of each pixel block in the preliminary adjustment interface is calculated, and when the abnormal score of the pixel block exceeds the preset threshold, the abnormal area is marked;

[0050] The area with an area greater than 50 pixels is extracted from the abnormal area, the center coordinates and boundary box size thereof are recorded as key abnormal data, the display proportion of the key abnormal area is enlarged by 2 times, an enlarged abnormal area bitmap is generated, and the enlarged abnormal area bitmap is superimposed on the corresponding coordinates of the preliminary adjustment interface to output the final display interface.

[0051] A vehicle state monitoring and warning system, the system comprises:

[0052] A data acquisition and processing module is used to obtain real-time state data from vehicle sensors, and to preliminarily screen the data, if the parameter exceeds the preset threshold, it is marked as an abnormal event, and an abnormal trend set is generated;

[0053] A priority calculation and distribution module is used to calculate the information presentation priority according to the abnormal severity and driving scene complexity, if the information presentation priority score is higher than the preset threshold, the abnormal information is distributed to the high priority display queue, and the dynamic hierarchical display content is determined;

[0054] A multi-modal warning generation module is used to generate a multi-modal warning signal according to the dynamic hierarchical display content, if the driving scene is high-speed driving, high-frequency sound and vibration feedback are preferentially used, and a multi-modal warning combination is obtained; the multi-modal warning signal includes sound prompt, dashboard image and tactile feedback;

[0055] Speech Synthesis and Interaction Module: This module extracts the audio signal of the sound prompt from the multimodal warning combination, generates speech waveform data with natural language description through the speech synthesis module, acquires the driver's voice input, parses the semantics through the speech recognition model, and if the recognition result contains the intention to query the status, it extracts abnormal information from the abnormal trend set, generates voice interaction response content, and determines the interaction feedback information.

[0056] Interaction optimization and display module: Based on the interaction feedback information, combined with the current driving scenario and abnormal status, it determines the scenario type. If the scenario is low-speed congestion, it displays detailed feedback content through the instrument panel; otherwise, it prioritizes outputting concise information through voice to obtain the optimized interaction output.

[0057] The interface dynamic adjustment module is used to extract the display content from the optimized interactive output, and dynamically adjust the dashboard interface through the image processing module. If the level of detail of the abnormal feedback is higher than the preset threshold, the key abnormal area is enlarged to generate the final display interface.

[0058] Signal transmission and rendering module: used to acquire image and voice data of the final display interface, integrate them into a unified data stream, generate image and voice data streams adapted to the vehicle terminal, measure transmission delay, and determine that the signal transmission meets the requirements if the transmission delay is lower than the preset threshold.

[0059] Preferably, the priority calculation and allocation module includes:

[0060] Anomaly Severity Assessment Unit: This unit calculates the anomaly severity score using a weighted scoring method based on the anomaly occurrence frequency, impact scope, and potential risk level. The weights for anomaly occurrence frequency and impact scope are 0.4, and for potential risk level, 0.3 respectively. The formula for calculating the anomaly severity score is as follows: ,in, The score indicates the severity of the abnormality. Indicates the frequency of occurrence score. Indicates the scope of influence score. Indicates the potential risk level score;

[0061] Driving Scenarios Complexity Assessment Unit: This unit calculates a driving scenario complexity score based on the current lane number, distance to the vehicle ahead, and weather visibility. The calculation formula is as follows:

[0062]

[0063] Where C represents the driving scenario complexity score, Indicates the number of evaluation factors. Indicates the first The weights of each factor Indicates the first a score value of the factor;

[0064] The priority calculation unit is used for calculating the information presentation priority score by using a linear weighting method according to the abnormal severity score and the driving scene complexity score, wherein the weight of the abnormal severity score is 0.6, and the weight of the driving scene complexity score is 0.4; the calculation formula is represented as: wherein, represents the information presentation priority score, represents the abnormal severity score, and C represents the driving scene complexity score;

[0065] The high-priority queue management unit is used for distributing the abnormal information to the high-priority display queue if the information presentation priority score is higher than a preset threshold, and using a sliding window algorithm to update the display queue in real time, so as to ensure that the high-priority information is timely presented.

[0066] The multi-modal early warning generation module comprises:

[0067] The driving scene recognition unit is used for obtaining driving scene data from the vehicle-mounted millimeter wave radar and the camera, and using a random forest classifier to determine whether the current scene is high-speed driving.

[0068] The early warning signal configuration unit is used for setting the sound signal frequency to 3000-5000Hz and the vibration intensity to 0.5-1.2G in the high-speed scene, and generating an initial early warning signal set.

[0069] The signal fusion unit is used for using a weighted average fusion algorithm to integrate the sound signal, the vibration signal and the image signal, and generating a multi-modal early warning signal.

[0070] The interface dynamic adjustment module comprises:

[0071] The abnormal area extraction unit is used for extracting an abnormal area from the instrument panel interface, and recording the center coordinates and the boundary box size thereof.

[0072] The area magnification unit is used for expanding the display proportion of the key abnormal area by 2 times, generating a magnified abnormal area bitmap, and superimposing the magnified abnormal area bitmap to the preliminary adjustment interface to generate a final display interface.

[0073] The technical effects and advantages of the present application are as follows:

[0074] 1. This invention provides a vehicle status monitoring and early warning method. It acquires real-time status data through vehicle sensors and uses a preset anomaly detection algorithm to initially filter the data, enabling rapid and accurate labeling of abnormal events and generating anomaly trend sets. This helps to promptly identify potential vehicle problems and take preventative measures to ensure driving safety. A dynamic hierarchical display algorithm is employed, calculating information presentation priorities based on the severity of anomalies and the complexity of the driving scenario. This ensures that high-priority information is presented promptly, avoiding information overload for the driver in complex driving scenarios and improving the efficiency and accuracy of information transmission. A multimodal warning signal is generated, including sound prompts, dashboard images, and tactile feedback. The warning method is adjusted according to the driving scenario, prioritizing high-frequency sound and vibration feedback during high-speed driving to more effectively attract the driver's attention, enhance perception, and improve driving safety. A speech synthesis module synthesizes the extracted audio signals to generate natural language-described speech waveform data. This natural language interaction method is more intuitive and easier to understand, improving the driver's interactive experience and reducing the possibility of misoperation. A speech recognition model is used to further enhance the driver's understanding of the vehicle's status. The system performs semantic parsing of driver voice input to quickly identify driver intentions and extracts relevant information from anomaly trend sets to generate voice interaction responses. This intelligent voice interaction method improves response speed, allowing drivers to more conveniently obtain the information they need while driving. It determines the scenario type based on the current driving scenario and abnormal state, and adjusts the information presentation accordingly. In low-speed, congested scenarios, detailed feedback is displayed on the dashboard, while in other scenarios, concise information is output via voice. This scenario-adaptive optimization method improves the quality of information presentation, ensuring drivers receive clear and accurate information in different situations. The image processing module dynamically adjusts the dashboard interface, magnifying key abnormal information areas to generate the final display interface. This dynamic interface adjustment enhances readability, enabling drivers to quickly obtain key information and make correct decisions. Finally, the integrated multimodal signals are rendered, and transmission latency is measured to ensure timely and accurate transmission to the vehicle terminal. This low-latency transmission method guarantees real-time information, allowing drivers to obtain the latest vehicle status information promptly. Attached Figure Description

[0075] Figure 1 This is a flowchart of the vehicle status monitoring and early warning process of the present invention;

[0076] Figure 2 This is a structural diagram of the vehicle condition monitoring and early warning system of the present invention;

[0077] Figure 3 This is a schematic diagram of the priority calculation and allocation module of the vehicle condition monitoring and early warning system of the present invention. Detailed Implementation

[0078] The embodiments given below in conjunction with the drawings further illustrate the application.

[0079] Referring to Figure 1 As shown in the figure, a vehicle state monitoring and early warning method, the method comprises:

[0080] S1, obtaining real-time state data from vehicle sensors, and preliminarily screening the data through a preset anomaly detection algorithm, if the parameters exceed the preset threshold, marking it as an abnormal event, and obtaining an abnormal trend set; this step includes the following steps in specific implementation:

[0081] Obtaining real-time data transmitted by vehicle sensors, including engine speed, brake pressure and tire pressure data, and removing noise through mean filtering to generate a standardized data set;

[0082] Preliminarily screening the parameters in the standardized data set through an anomaly detection algorithm, if any of the parameters exceeds the preset threshold, marking it as an abnormal event using a fixed threshold judgment method, and classifying the abnormal events using a K-Means algorithm to obtain an abnormal event set;

[0083] For the abnormal event set, extracting the timestamp and parameter value of each type of abnormal event to generate an abnormal feature data set, and using a decision tree algorithm to predict the development trend of the abnormal event for the abnormal feature data set to obtain an abnormal trend set.

[0084] In one embodiment, the vehicle sensors collect real-time running parameters such as engine speed, brake pressure, tire pressure, etc. The normal range of engine speed is 800 to 3000 rpm, the normal range of brake pressure is 10 to 20 MPa, and the normal range of tire pressure is 2.2 to 2.8 Kpa. Through a preset anomaly detection algorithm, such as the 3σ principle based on statistical process control (SPC), the data is preliminarily screened; if the engine speed exceeds 3000 rpm or is lower than 800 rpm, the brake pressure is lower than 10 MPa or higher than 20 MPa, and the tire pressure is lower than 2.2 Kpa or higher than 2.8 Kpa, then it is marked as an abnormal event.

[0085] For example, when the engine speed is detected to be 3500 rpm, the system automatically marks it as abnormal and records the relevant timestamp and parameter value, and the abnormal event is aggregated into an abnormal information set for subsequent analysis.

[0086] Further, if the brake pressure is detected to be lower than 10 MPa multiple times, the system may predict that there is a risk of brake system leakage and generate a warning message. The entire process realizes the automation of data collection, anomaly detection, and fault prediction, ensuring the safety of vehicle operation.

[0087] S2. For the abnormal trend set, a dynamic hierarchical display algorithm is used to calculate the information presentation priority based on the severity of the abnormality and the complexity of the driving scenario. If the information presentation priority score is higher than the preset threshold, the abnormal information is assigned to the high-priority display queue to determine the dynamic hierarchical display content.

[0088] In practice, the severity of the anomaly includes the frequency of anomaly occurrence, the scope of impact, and the level of potential risk; the complexity of the driving scenario includes the number of current lanes, the distance to the vehicle ahead, and weather visibility.

[0089] In one embodiment, the specific method for determining the dynamically hierarchical display content includes:

[0090] S201. Based on the frequency of anomalies, the scope of impact, and the level of potential risk, a weighted scoring method is used to calculate the severity score of the anomaly. When calculating the severity score, the weight of the frequency of anomalies is set at 0.4, the weight of the scope of impact at 0.3, and the weight of the level of potential risk at 0.3. The formula for calculating the severity score is as follows: ,in, The score indicates the severity of the abnormality. Indicates the frequency of occurrence score. Indicates the scope of influence score. This represents the potential risk level score; through this formula, a comprehensive severity assessment score can be obtained by comprehensively considering the frequency of anomalies, the scope of their impact, and the potential risk level.

[0091] S202. Calculate the driving scenario complexity score based on factors such as the current number of lanes, distance to vehicles ahead, and weather visibility. The calculation formula is as follows:

[0092]

[0093] Where C represents the driving scenario complexity score, Indicates the number of evaluation factors. Indicates the first The weights of each factor Indicates the first The score values ​​for each factor;

[0094] S203. Based on the anomaly severity score and the driving scenario complexity score, a linear weighted method is used to calculate the information presentation priority score. In the calculation, the weight of the anomaly severity score is set to 0.6, and the weight of the driving scenario complexity score is set to 0.4. The calculation formula is as follows: ,in, This indicates the priority score for information presentation. The score indicates the severity of the anomaly, and C indicates the complexity of the driving scenario.

[0095] S204, if the information presentation priority score is higher than the preset threshold, the abnormal information is assigned to a high priority display queue, and a sliding window algorithm is used to update the high priority display queue in real time to ensure that high priority information can be presented in time.

[0096] For example, the preset threshold is set to 80 points, when the information presentation priority score of a certain abnormal information is 85 points, and there are 3 high priority information in the current display queue, the system will automatically remove the information with the lowest score from the queue to ensure that the new information can be displayed in time. Through this dynamic hierarchical display algorithm, the driving safety and information processing efficiency can be effectively improved.

[0097] S3, according to the dynamic hierarchical display content, a multi-modal warning signal is generated, if the driving scene is high-speed driving, high-frequency sound and vibration feedback are preferred, and a multi-modal warning combination is obtained; the multi-modal warning signal includes sound prompt, instrument panel image and tactile feedback.

[0098] In specific implementation, the way of generating a multi-modal warning signal is:

[0099] The driving scene data is obtained from the vehicle-mounted millimeter wave radar and camera, including vehicle speed, acceleration and lane line curvature, a random forest classifier is used to determine whether the current scene is high-speed driving, and when the classification confidence is greater than the preset threshold, it is determined as a high-speed scene;

[0100] For example, the classification confidence is set to , the preset threshold is set to , when > , it is determined as a high-speed scene, in specific implementation, the preset threshold is set to 0.85, when the classification confidence is greater than the preset threshold 0.85, it is determined as a high-speed scene.

[0101] In the high-speed scene, the sound signal frequency is set to 3000-5000Hz, and the vibration intensity is set to 0.5-1.2G as the preferred signal parameter, and an initial warning signal set is generated according to the priority configuration, including sound frequency, vibration intensity and duration;

[0102] The sound frequency and duration parameters are extracted from the initial warning signal set, and the instrument panel display module is input to generate an image signal with a flashing red border;

[0103] The weighted average fusion algorithm is used to integrate the sound signal, vibration signal and image signal, when calculating, the sound weight is set to 0.4, the vibration weight is set to 0.3, and the image weight is set to 0.3; the weighted average fusion algorithm is represented as: , wherein represents the fused signal, representative sound signal, representative vibration signal, representative image signal;

[0104] The fused signal is encoded into a CAN bus protocol format, output to a speaker, a vibration motor and a display screen through a vehicle control unit, and a multi-modal early warning signal is generated.

[0105] S4, extract the audio signal of the sound prompt part from the multi-modal early warning combination, and synthesize the extracted audio signal through a speech synthesis module to generate voice waveform data of natural language description; the specific implementation of this step includes:

[0106] The audio signal of the 20Hz-20kHz frequency band is intercepted from the multi-modal early warning combination to obtain a sound prompt signal, and the mel frequency cepstrum coefficient of the sound prompt signal is calculated to obtain an audio feature vector;

[0107] When the audio signal is intercepted, the mel frequency cepstrum coefficient (MFCC) algorithm is used for feature extraction of the audio signal, the frame length is set to 25ms, the frame shift is set to 10ms, and 13-dimensional MFCC coefficients are extracted as sound features;

[0108] The audio feature vector is input into a pre-trained support vector machine classifier to output an abnormal type label and an abnormal severity score, and if the abnormal type label is an emergency brake fault, the vehicle fault code is read through the OBD-II interface and parsed to obtain a fault location code;

[0109] The abnormal type label, the abnormal severity score and the fault location code are input into a speech synthesis module to generate voice waveform data of natural language description; the speech synthesis module can use a Tacotron2 model.

[0110] For example, the input text is "left rear wheel brake pressure sensor fault, need to be repaired immediately", the acoustic feature dimension is 80-dimensional mel spectrum, the sampling rate is 16kHz, and the voice waveform data of natural language description is generated through a WaveNet vocoder.

[0111] The application can also sort the voice waveform data of natural language description in descending order according to the abnormal severity score to obtain a priority queue, and play the voice waveform data in the priority queue through a vehicle-mounted speaker; a microphone is used to collect the played voice signal, and a dynamic time warping algorithm is used to calculate the similarity between the voice signal and the original voice waveform data, and if the similarity is less than 95%, the voice waveform data is regenerated.

[0112] S5, obtaining the driver's voice input, performing semantic analysis on the input instruction through a voice recognition model, if the recognition result contains a state query intention, extracting abnormal information from the abnormal trend set, generating voice interaction response content, and determining interaction feedback information;

[0113] In specific implementation, first, when obtaining the driver's voice input, a four-channel circular microphone array can be used to capture the driver's voice at a sampling rate of 16 kHz, thereby generating an original audio signal, and a NS module in WebRTC can be used to perform noise reduction processing on the original audio signal, outputting clear voice data in 16-bit PCM format; then, when performing semantic analysis, the clear voice data is input into the Kaldi voice recognition tool to output an instruction text string in UTF-8 encoding, and RasaNLU is used to classify the intention of the instruction text. When a state query intention is recognized, information including vehicle speed, fuel level and tire pressure is extracted. For example, when the driver says "check tire pressure", the voice recognition model can use the BERT pre-training model to classify the intention, and output a query intention probability of 0.87, which exceeds the threshold value of 0.65 to determine a state query intention; finally, according to the vehicle speed, fuel level and tire pressure information, the matching fault code record in the last 24 hours is queried from the abnormal trend set, the fault code record is filled into the preset Jinja2 text template, the response text string is generated, the response text is converted into an MP3 format audio stream with a sampling rate of 48 kHz by GoogleTTS, and the audio stream is played through the car speaker at a sound pressure level of 80 dB. For example, when the left front tire pressure is 210 kPa (normal range 220-250 kPa), an abnormality judgment rule is triggered, and the template filling technology is used to convert the text "left front tire pressure is low, current value 210 kPa, suggest to supplement to 230 kPa" into voice output through Tacotron2 vocoder.

[0114] It should be noted that the entire voice processing process of the present application can be completed within 800 ms, meeting the real-time requirements of vehicle-mounted applications, wherein the voice recognition takes 300 ms, the semantic analysis takes 150 ms, the data query takes 50 ms, and the voice synthesis takes 300 ms.

[0115] S6, according to the interaction feedback information, combining the current driving scene and the abnormal state, judging the scene type, if the scene is low-speed congestion, displaying detailed feedback content through the instrument panel, otherwise, preferentially outputting concise information through voice, obtaining the optimized interaction output; the specific implementation of this step includes:

[0116] By obtaining real-time data from vehicle sensors and user interaction interfaces, analyzing the driving environment and interaction feedback information, and determining the characteristic description of the current state;

[0117] Further, the vehicle sensors include speed sensors, distance sensors, and cameras, etc., for acquiring vehicle speed, vehicle distance, and surrounding environment information. For example, assuming that the current vehicle speed is 5 kilometers per hour, the front vehicle distance is only 2 meters, and the camera captures a dense arrangement of vehicles in front, these data will be transmitted to the system for analysis, and a preliminary judgment can be made that it may be in a congestion state. Such data collection method can ensure that the system's perception of the driving environment has high precision and real-time, providing a reliable basis for subsequent classification.

[0118] According to the characteristic description of the current state, combined with the pre-established scene classification model, the driving environment and abnormal state are classified by using the support vector machine algorithm to determine whether it belongs to the low-speed congestion scene;

[0119] If the classification result is a low-speed congestion scene, detailed feedback content is generated through the instrument panel display module, including specific description of the current state and suggested operation, to obtain visual output results;

[0120] The specific description of the current state includes the current vehicle speed and the length of the congestion in front, and the suggested operation includes suggesting the driver to remain patient or choose a detour route; for example, the current vehicle speed is 5 kilometers per hour, and the length of the congestion in front is about 200 meters, at this time, the driver is suggested to remain patient or choose a detour route, such visual output is intuitive and clear, which helps the driver quickly understand the situation and take reasonable measures, thereby improving driving safety and comfort.

[0121] If the classification result does not belong to the low-speed congestion scene, concise information is generated through the speech synthesis module, including key prompts of the current state, to obtain speech output content; for example, the vehicle speed is 60 kilometers per hour and the vehicle distance is greater than 50 meters, the system will generate a concise prompt such as "the current road is smooth, please pay attention to maintaining a safe distance" through the speech synthesis module. Such speech output method is concise and efficient, avoiding distracting the driver's attention, while ensuring that key information is conveyed in time and enhancing the interactive experience.

[0122] For the generated detailed feedback content or concise information, response data of the user interaction interface is acquired, the acceptance degree of the user to the output method is analyzed, and the adjustment direction of the optimization scheme is determined;

[0123] According to the analysis result of the user response data, the priority and content form of the output method are adjusted, an updated interaction scheme is generated, and the optimization processing of scene adaptation is completed. For example, in the low-speed congestion scene, the detailed feedback displayed on the instrument panel is frequently viewed by the driver, while the voice prompt obtains a high response rate in the non-congestion scene, the system will adjust the output priority accordingly, and preferentially select the method more accepted by the user.

[0124] S7, extract the display content from the optimized interaction output, dynamically adjust the dashboard interface through the image processing module, if the abnormal feedback detail is higher than the preset threshold, enlarge the key abnormal area, and generate the final display interface; the specific implementation of this step includes:

[0125] Obtain the original display content in JSON format from the interaction output, and extract the digital scale and pointer angle of the dashboard interface through the image processing module to generate structured display data containing numerical values and coordinates;

[0126] It should be noted that the image processing module used can be the Canny edge detection algorithm of the OpenCV image analysis module.

[0127] According to the numerical range in the structured display data, adjust the color contrast and scale line width of the dashboard, output a preliminary adjustment interface in bitmap format, calculate the abnormal score of each pixel block in the preliminary adjustment interface, and when the abnormal score of the pixel block exceeds the preset threshold, mark it as an abnormal area;

[0128] It should be noted that when adjusting the color contrast and scale line width of the dashboard, the Pillow library can be called for adjustment, and when calculating the abnormal score of each pixel block in the preliminary adjustment interface, the Isolation Forest algorithm can be used for calculation.

[0129] Extract the area with an area greater than 50 pixels from the abnormal area, record the center coordinates and bounding box size as key abnormal data, and enlarge the display ratio of the key abnormal area by 2 times to generate an enlarged abnormal area bitmap. Superimpose the enlarged abnormal area bitmap onto the corresponding coordinates of the preliminary adjustment interface to output the final display interface.

[0130] It should be noted that when the display ratio of the key abnormal area is enlarged by 2 times, a bilinear interpolation algorithm can be used.

[0131] S8, obtain the image and voice data of the final display interface, integrate the image and voice data into a unified data stream, obtain a multi-modal signal, render the multi-modal signal, generate image and voice data streams adapted to the vehicle terminal, obtain the rendered transmission signal, transmit the rendered transmission signal to the vehicle terminal, and measure the transmission delay. If the transmission delay is lower than the preset threshold, it is determined that the signal transmission meets the requirements.

[0132] It should be noted that when rendering the multi-modal signal, OpenGL rendering engine can be used. When acquiring the image, the image signal is collected at a rate of 30 frames per second, and the H.265 encoding algorithm is used to compress the image, and the compression ratio is 1:50; when acquiring the voice data, the sound signal is captured at a sampling rate of 44.1kHz, and the AAC encoding is used to compress the sound signal, and the compression ratio is 1:10.

[0133] The application first acquires real-time running parameters from vehicle sensors, detects abnormal events using a preset algorithm, and generates an abnormal trend set; secondly, according to the abnormal severity and driving scene complexity, the information presentation priority is calculated, and the dynamic hierarchical display content is determined; thirdly, according to the dynamic hierarchical display content, a multi-modal warning signal is generated, including sound, image and tactile feedback, and is optimized and combined according to the driving scene; finally, voice interaction is also supported, and the corresponding abnormal state feedback can be provided according to the driver's voice instruction. Through the above technical scheme, the application can timely and accurately transmit vehicle abnormal information to the driver, effectively improving the driving safety.

[0134] Referring to Figures 2-3 The application also proposes a vehicle state monitoring and warning system, which comprises:

[0135] The data acquisition and processing module is used to acquire real-time state data from vehicle sensors, and to preliminarily screen the data. If the parameter exceeds the preset threshold, it is marked as an abnormal event, and an abnormal trend set is generated.

[0136] The priority calculation and distribution module is used to calculate the information presentation priority according to the abnormal severity and driving scene complexity. If the information presentation priority score is higher than the preset threshold, the abnormal information is distributed to the high-priority display queue, and the dynamic hierarchical display content is determined. The module also comprises:

[0137] The abnormal severity evaluation unit is used to calculate the abnormal severity score by using the weighted scoring method according to the abnormal occurrence frequency, influence range and potential risk level, wherein the weight of the abnormal occurrence frequency is 0.4, the weight of the influence range is 0.3, and the weight of the potential risk level is 0.3. The calculation formula of the abnormal severity score is: wherein, represents the abnormal severity score, represents the abnormal occurrence frequency score, represents the influence range score, represents the potential risk level score;

[0138] The driving scene complexity evaluation unit is used to calculate the driving scene complexity score according to the current lane number, the distance to the front vehicle and the weather visibility, and the calculation formula is as follows:

[0139]

[0140] wherein C represents a driving scene complexity score, represents the number of evaluation factors, represents the weight of the th factor, represents the score value of the th factor;

[0141] a priority calculation unit: configured to calculate an information presentation priority score according to the abnormal severity score and the driving scene complexity score by using a linear weighting method, wherein the weight of the abnormal severity score is 0.6 and the weight of the driving scene complexity score is 0.4; the calculation formula is represented as: wherein, represents the information presentation priority score, represents the abnormal severity score, and C represents the driving scene complexity score;

[0142] a high-priority queue management unit: configured to assign the abnormal information to a high-priority display queue if the information presentation priority score is higher than a preset threshold, and to update the display queue in real time by using a sliding window algorithm to ensure that the high-priority information is timely presented.

[0143] a multi-modal warning generation module: configured to generate a multi-modal warning signal according to the dynamically graded display content, and to preferentially use high-frequency sound and vibration feedback to obtain a multi-modal warning combination if the driving scene is high-speed driving; the multi-modal warning signal comprises a sound prompt, an instrument panel image and a tactile feedback; the module further comprises:

[0144] a driving scene recognition unit: configured to obtain driving scene data from a vehicle-mounted millimeter wave radar and a camera, and to determine whether the current scene is high-speed driving by using a random forest classifier;

[0145] a warning signal configuration unit: configured to set the sound signal frequency to 3000-5000 Hz and the vibration intensity to 0.5-1.2 G in a high-speed scene to generate an initial warning signal set;

[0146] a signal fusion unit: configured to integrate the sound signal, the vibration signal and the image signal by using a weighted average fusion algorithm to generate a multi-modal warning signal.

[0147] a speech synthesis and interaction module: configured to extract an audio signal of the sound prompt part from the multi-modal warning combination, to generate voice waveform data of a natural language description by using a speech synthesis module, to obtain a driver voice input, to analyze the semantics by using a speech recognition model, to extract abnormal information from an abnormal trend set if the recognition result contains a state query intent, to generate voice interaction response content, and to determine interaction feedback information.

[0148] Interaction optimization and display module: for judging the scene type according to the interaction feedback information, combining the current driving scene and the abnormal state, if the scene is low-speed congestion, then display the detailed feedback content through the instrument panel, otherwise prefer to output concise information through voice, get the optimized interaction output.

[0149] Interface dynamic adjustment module: for extracting display content from the optimized interaction output, dynamically adjusting the instrument panel interface through the image processing module, if the abnormal feedback detail is higher than the preset threshold, then enlarge the key abnormal area, generate the final display interface; the module also includes:

[0150] Abnormal area extraction unit: for extracting the abnormal area from the instrument panel interface, recording the center coordinates and the boundary box size;

[0151] Region enlargement unit: for expanding the display proportion of the key abnormal area by 2 times, generating the enlarged abnormal area bitmap, and superimposing it to the preliminary adjustment interface to generate the final display interface.

[0152] Signal transmission and rendering module: for obtaining the image and voice data of the final display interface, integrating them into a unified data stream, generating the image and voice data stream adapted to the vehicle terminal, measuring the transmission delay, if the transmission delay is lower than the preset threshold, then determine that the signal transmission meets the requirements.

[0153] The above only describes the preferred embodiments of the present application, it should be noted that for those skilled in the art, without departing from the inventive concept, a number of modifications and improvements can be made, which are all within the scope of the present application.

Claims

1. A method for vehicle condition monitoring and early warning, characterized in that, The method includes: S1. Acquire real-time status data from vehicle sensors, perform preliminary data filtering, and mark any parameters exceeding preset thresholds as abnormal events to obtain an abnormal trend set, including: Acquire real-time data transmitted by vehicle sensors, including engine speed, brake pressure and tire pressure data, and remove noise through mean filtering to generate a standardized dataset; Anomaly detection algorithms are used to initially screen parameters in a standardized dataset. If any parameter exceeds a preset threshold, it is marked as an anomalous event using a fixed threshold judgment method. The K-Means algorithm is then used to classify the anomalous events to obtain an anomalous event set. For the set of anomalous events, the timestamps and parameter values ​​of each type of anomalous event are extracted to generate an anomalous feature dataset. The decision tree algorithm is then used to predict the development trend of anomalous events based on the anomalous feature dataset to obtain an anomalous trend set. S2. For the abnormal trend set, calculate the information presentation priority based on the severity of the abnormality and the complexity of the driving scenario. If the information presentation priority score is higher than the preset threshold, the abnormal information is assigned to the high priority display queue to determine the dynamically graded display content. S3. Generate a multimodal warning signal based on the dynamically hierarchical display content. If the driving scenario is high-speed driving, high-frequency sound and vibration feedback are used first to obtain a multimodal warning combination. The multimodal warning signal includes sound prompts, dashboard images and tactile feedback. S4. Extract the audio signal of the sound prompt part from the multimodal warning combination, and synthesize the extracted audio signal through the speech synthesis module to generate speech waveform data of natural language description. S5. Obtain the driver's voice input, perform semantic parsing of the input command through the voice recognition model, and if the recognition result contains the intent of status query, extract abnormal information from the abnormal trend set, generate voice interaction response content, and determine the interaction feedback information. S6. Based on the interactive feedback information, combined with the current driving scenario and abnormal state, determine the scenario type. If the scenario is low-speed congestion, display detailed feedback content through the instrument panel; otherwise, prioritize outputting concise information through voice to obtain optimized interactive output. S7. Extract the display content from the optimized interactive output, dynamically adjust the dashboard interface through the image processing module, and if the level of detail of the abnormal feedback is higher than the preset threshold, enlarge the key abnormal area and generate the final display interface. S8. Obtain the image and voice data of the final display interface, integrate the image and voice data into a unified data stream to obtain a multimodal signal, perform rendering processing on the multimodal signal to generate an image and voice data stream adapted to the vehicle terminal, obtain the rendered transmission signal, transmit the rendered transmission signal to the vehicle terminal, and measure the transmission delay. If the transmission delay is lower than the preset threshold, it is determined that the signal transmission meets the requirements.

2. The vehicle status monitoring and early warning method according to claim 1, characterized in that, The severity of the anomaly includes the frequency of occurrence, the scope of impact, and the level of potential risk. The complexity of the driving scenario includes the current number of lanes, the distance to the vehicle ahead, and weather visibility.

3. The vehicle status monitoring and early warning method according to claim 2, characterized in that, The specific methods for determining the dynamically hierarchical display content include: S201. Based on the frequency of anomalies, the scope of impact, and the level of potential risk, a weighted scoring method is used to calculate the severity score of the anomaly. When calculating the severity score, the weight of the frequency of anomalies is set at 0.4, the weight of the scope of impact at 0.3, and the weight of the level of potential risk at 0.

3. The formula for calculating the severity score is as follows: ,in, The score indicates the severity of the abnormality. Indicates the frequency of occurrence score. Indicates the scope of influence score. Indicates the potential risk level score; S202. Calculate the driving scenario complexity score based on factors such as the current number of lanes, distance to vehicles ahead, and weather visibility. The calculation formula is as follows: Where C represents the driving scenario complexity score, Indicates the number of evaluation factors. Indicates the first The weights of each factor Indicates the first The score values ​​for each factor; S203. Based on the anomaly severity score and the driving scenario complexity score, a linear weighted method is used to calculate the information presentation priority score. In the calculation, the weight of the anomaly severity score is set to 0.6, and the weight of the driving scenario complexity score is set to 0.

4. The calculation formula is as follows: ,in, This indicates the priority score for information presentation. The score indicates the severity of the anomaly, and C indicates the complexity of the driving scenario. S204. If the information presentation priority score is higher than the preset threshold, the abnormal information is assigned to the high priority display queue, and the sliding window algorithm is used to update the high priority display queue in real time to ensure that high priority information can be presented in a timely manner.

4. The vehicle status monitoring and early warning method according to claim 1, characterized in that, The method for generating the multimodal early warning signal is as follows: Driving scene data is obtained from vehicle-mounted millimeter-wave radar and cameras, including three features: vehicle speed, acceleration, and lane curvature. A random forest classifier is used to determine whether the current scene is high-speed driving. When the classification confidence is greater than a preset threshold, it is determined to be a high-speed scene. In high-speed scenarios, the sound signal frequency is set to 3000-5000Hz and the vibration intensity is set to 0.5-1.2G as priority signal parameters. An initial warning signal set is generated according to the priority configuration, which includes three parameters: sound frequency, vibration intensity and duration. The sound frequency and duration parameters are extracted from the initial warning signal set and input into the dashboard display module to generate an image signal with a flashing red border. A weighted average fusion algorithm is used to integrate sound, vibration, and image signals. During calculation, the weights are set as follows: sound weight 0.4, vibration weight 0.3, and image weight 0.

3. The weighted average fusion algorithm is expressed as follows: ,in Represents the fused signal. Represents sound signals. Represents vibration signal, Represents image signals; The fused signal is encoded into the CAN bus protocol format and output to the speaker, vibration motor and display screen through the vehicle control unit to generate a multimodal warning signal.

5. The vehicle status monitoring and early warning method according to claim 4, characterized in that, The process of extracting the audio signal of the sound prompt portion from the multimodal warning combination and synthesizing the extracted audio signal through a speech synthesis module to generate speech waveform data with natural language description includes: Audio signals in the 20Hz-20kHz frequency band are extracted from the multimodal warning combination to obtain the sound prompt signal. Mel frequency cepstral coefficients are calculated on the sound prompt signal to obtain the audio feature vector. The audio feature vector is input into a pre-trained support vector machine classifier, which outputs an anomaly type label and an anomaly severity score. If the anomaly type label is emergency braking fault, the vehicle fault code is read and parsed through the OBD-II interface to obtain the fault location code. Input the anomaly type label, anomaly severity score, and fault location code into the speech synthesis module to generate speech waveform data with natural language description.

6. The vehicle status monitoring and early warning method according to claim 5, characterized in that, Based on the interactive feedback information, combined with the current driving scenario and abnormal state, the scenario type is determined. If the scenario is low-speed congestion, detailed feedback content is displayed on the instrument panel; otherwise, concise information is output via voice first, resulting in optimized interactive output, including: By acquiring real-time data from vehicle sensors and user interaction interfaces, analyzing driving environment and interaction feedback information, and determining the characteristic description of the current state; Based on the feature description of the current state, combined with the pre-established scene classification model, the support vector machine algorithm is used to classify the driving environment and abnormal state to determine whether it belongs to a low-speed congestion scene. If the classification result is a low-speed congestion scenario, detailed feedback content will be generated through the dashboard display module, including a specific description of the current status and suggested actions, to obtain a visual output result; The specific description of the current state includes the current vehicle speed and the length of the congestion ahead, and the suggested actions include advising the driver to be patient or to choose an alternate route. If the classification result does not belong to the low-speed congestion scenario, the speech synthesis module generates concise information, including key prompts of the current status, to obtain the speech output content. For the generated detailed feedback or concise information, obtain the response data of the user interaction interface, analyze the user's acceptance of the output method, and determine the direction of the optimization plan adjustment; Based on the analysis results of user response data, the priority and content format of the output method are adjusted to generate an updated interaction plan and complete the optimization process for scene adaptation.

7. The vehicle status monitoring and early warning method according to claim 6, characterized in that, The process of extracting display content from the optimized interactive output, dynamically adjusting the dashboard interface through the image processing module, and enlarging the key abnormal information area if the level of detail in the abnormal feedback exceeds a preset threshold to generate the final display interface includes: The system retrieves the raw display content in JSON format from the interactive output and extracts the digital scales and pointer angles from the dashboard interface through the image processing module to generate structured display data containing numerical values ​​and coordinates. Based on the numerical range in the structured display data, adjust the color contrast and scale line width of the dashboard, output a preliminary adjustment interface in bitmap format, calculate the abnormal score of each pixel block in the preliminary adjustment interface, and mark the pixel block as an abnormal area when the abnormal score exceeds the preset threshold. Extract areas larger than 50 pixels from the abnormal region, record their center coordinates and bounding box size as key anomaly data, enlarge the display scale of the key anomaly region by 2 times, generate an enlarged anomaly region bitmap, overlay the enlarged anomaly region bitmap onto the corresponding coordinates of the initially adjusted interface, and output the final display interface.

8. A vehicle condition monitoring and early warning system, characterized in that, The system includes: Data acquisition and processing module: used to acquire real-time status data from vehicle sensors, perform preliminary data screening, and mark abnormal events if parameters exceed preset thresholds, generating an abnormal trend set; Priority calculation and allocation module: It is used to calculate the information presentation priority based on the severity of the anomaly and the complexity of the driving scenario. If the information presentation priority score is higher than the preset threshold, the abnormal information is allocated to the high priority display queue to determine the dynamically graded display content. Multimodal warning generation module: used to generate multimodal warning signals based on dynamically hierarchical display content. If the driving scenario is high-speed driving, high-frequency sound and vibration feedback are used first to obtain a multimodal warning combination. The multimodal warning signal includes sound prompts, dashboard images and tactile feedback. Speech Synthesis and Interaction Module: This module extracts the audio signal of the sound prompt from the multimodal warning combination, generates speech waveform data with natural language description through the speech synthesis module, acquires the driver's voice input, parses the semantics through the speech recognition model, and if the recognition result contains the intention to query the status, it extracts abnormal information from the abnormal trend set, generates voice interaction response content, and determines the interaction feedback information. Interaction optimization and display module: Based on the interaction feedback information, combined with the current driving scenario and abnormal status, it determines the scenario type. If the scenario is low-speed congestion, it displays detailed feedback content through the instrument panel; otherwise, it prioritizes outputting concise information through voice to obtain the optimized interaction output. The interface dynamic adjustment module is used to extract the display content from the optimized interactive output, and dynamically adjust the dashboard interface through the image processing module. If the level of detail of the abnormal feedback is higher than the preset threshold, the key abnormal area is enlarged to generate the final display interface. Signal transmission and rendering module: used to acquire image and voice data of the final display interface, integrate them into a unified data stream, generate image and voice data streams adapted to the vehicle terminal, measure transmission delay, and determine that the signal transmission meets the requirements if the transmission delay is lower than the preset threshold.

9. A vehicle status monitoring and early warning system according to claim 8, characterized in that, The priority calculation and allocation module includes: Anomaly Severity Assessment Unit: This unit calculates the anomaly severity score using a weighted scoring method based on the anomaly occurrence frequency, impact scope, and potential risk level. The weights for anomaly occurrence frequency and impact scope are 0.4, and for potential risk level, 0.3 respectively. The formula for calculating the anomaly severity score is as follows: ,in, The score indicates the severity of the abnormality. Indicates the frequency of occurrence score. Indicates the scope of influence score. Indicates the potential risk level score; Driving Scenarios Complexity Assessment Unit: This unit calculates a driving scenario complexity score based on the current lane number, distance to the vehicle ahead, and weather visibility. The calculation formula is as follows: Where C represents the driving scenario complexity score, Indicates the number of evaluation factors. Indicates the first The weights of each factor Indicates the first The score values ​​for each factor; Priority Calculation Unit: This unit calculates a priority score based on the severity of the anomaly and the complexity of the driving scenario, using a linear weighting method. The severity of the anomaly score has a weight of 0.6, and the complexity of the driving scenario score has a weight of 0.

4. The calculation formula is as follows: ,in, This indicates the priority score for information presentation. The score indicates the severity of the anomaly, and C indicates the complexity of the driving scenario. High-priority queue management unit: If the information presentation priority score is higher than a preset threshold, the abnormal information will be assigned to the high-priority display queue, and the display queue will be updated in real time using a sliding window algorithm to ensure that high-priority information is presented in a timely manner. The multimodal early warning generation module includes: Driving Scene Recognition Unit: Used to acquire driving scene data from vehicle-mounted millimeter-wave radar and cameras, and use a random forest classifier to determine whether the current scene is high-speed driving; Warning signal configuration unit: used to set the sound signal frequency to 3000-5000Hz and the vibration intensity to 0.5-1.2G in high-speed scenarios, and generate an initial warning signal set; Signal fusion unit: used to integrate sound signals, vibration signals and image signals using a weighted average fusion algorithm to generate multimodal early warning signals; The interface dynamic adjustment module includes: Anomaly region extraction unit: used to extract anomaly regions from the dashboard interface and record their center coordinates and bounding box dimensions; Region zoom unit: Used to increase the display scale of key abnormal areas by 2 times, generate an enlarged abnormal area bitmap, and overlay it onto the preliminary adjustment interface to generate the final display interface.

Images