Vehicle monitoring and abnormal behavior early warning method based on multi-dimensional information fusion

Abstract

The invention relates to the technical field of vehicle monitoring, in particular to a multi-dimensional information fusion vehicle monitoring and abnormal behavior early warning method, which comprises the following steps: acquiring real-time vehicle operation data, environment perception data and driver behavior data; performing time-space synchronization and feature fusion on the vehicle operation data, the environment perception data and the driver behavior data to generate a multi-dimensional fusion feature vector; inputting the multi-dimensional fusion feature vector into a preset abnormal behavior detection model, and outputting an abnormal probability value of the current driving scene; and if the abnormal probability value exceeds a preset threshold value, generating a graded early warning signal and carrying out early warning prompt through a vehicle-mounted interaction interface and a cloud platform. The problems that an existing vehicle monitoring and abnormal behavior early warning method is single in data source, low in monitoring accuracy and incapable of achieving real-time monitoring are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle monitoring, and in particular to a vehicle monitoring and abnormal behavior early warning method with multi-dimensional information fusion. Background Art

[0002] With the acceleration of urbanization and the continuous increase in traffic flow, vehicle monitoring and abnormal behavior warning have become important links in traffic management. Traditional vehicle monitoring systems mainly rely on a single sensor or data source, such as cameras, radars, etc., but these methods often have problems such as single information and high false alarm rate. In addition, with the development of intelligent transportation systems, the accuracy and real-time requirements for vehicle monitoring are getting higher and higher, and traditional monitoring methods can no longer meet current needs. Therefore, it is particularly important to develop a vehicle monitoring and abnormal behavior warning method that can integrate multiple information sources and improve monitoring accuracy and real-time performance.

[0003] Therefore, the present invention provides a vehicle monitoring and abnormal behavior warning method with multi-dimensional information fusion to solve the above problems. Summary of the invention

[0004] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a vehicle monitoring and abnormal behavior warning method with multi-dimensional information fusion, which solves the problems of the existing vehicle monitoring and abnormal behavior warning methods such as single data source, low monitoring accuracy and inability to monitor in real time.

[0005] In order to achieve the above-mentioned purpose, the technical solution adopted by the present invention is: a vehicle monitoring and abnormal behavior warning method with multi-dimensional information fusion, the warning method comprising: obtaining real-time vehicle operation data, environmental perception data and driver behavior data, the vehicle operation data including vehicle speed, acceleration, braking status and power system parameters, the environmental perception data including camera images, radar point clouds and GPS positioning information, the driver behavior data including steering wheel operation frequency, fatigue status detection results and line of sight direction; performing spatiotemporal synchronization and feature fusion on the vehicle operation data, environmental perception data and driver behavior data to generate a multi-dimensional fusion feature vector; inputting the multi-dimensional fusion feature vector into a preset abnormal behavior detection model to output the abnormal probability value of the current driving scene; if the abnormal probability value exceeds a preset threshold, generating a graded warning signal and issuing a warning prompt through the vehicle-mounted interactive interface and the cloud platform.

[0006] A further improvement of the present application is that the spatiotemporal synchronization and feature fusion of data includes: aligning asynchronous data streams from different sensors based on timestamps; spatially registering vehicle position information with radar point cloud data through a Kalman filter algorithm; extracting lane lines, obstacle contours and traffic sign semantic information from camera images, and performing redundancy verification with radar detection results; inputting the verified data into a multi-layer neural network for feature dimensionality reduction and fusion to generate the multi-dimensional fusion feature vector.

[0007] A further improvement of the present application is that the construction of the abnormal behavior detection model includes: S100, define the multi-dimensional features of the model input as , is the feature dimension, including the normalized vehicle speed , lateral acceleration , steering wheel angle , Driver's gaze deviation , Distance to the vehicle ahead and environmental visibility , and satisfy: (1), In expression (1), and Represent the mean and standard deviation of historical vehicle speeds, Indicates the preset maximum lateral acceleration threshold; S200, builds a model based on a deep reinforcement learning framework, including: S201. Define state space ,in, is the LSTM hidden layer state, Represents vector concatenation; S202, map the action space output abnormal probability through Sigmoid activation function , , whose expression is: (2), represents the weight matrix, represents the bias term; S203, define a reward function, and dynamically adjust it according to the warning accuracy. The reward function expression is: ; S300, combining classification cross entropy and reinforcement learning strategy gradient to determine the joint loss function, which is expressed as: (3), In expression (3), represents the true label, where Indicates that the current moment is normal driving behavior. Indicates that the current moment is judged as abnormal driving behavior. represents the balance coefficient, represents the state-action value function approximated by a deep Q-network; S400, dynamically updating model parameters through online incremental learning to adapt to different road types and driving habits, the road types include highways, urban roads and mountain curves, and the weight adjustment formula is: (4), In expression (4), represents the road type adaptation loss, Indicates the scene weights of highways, urban roads, and mountain curves. and represents the learning rate hyperparameter.

[0008] A further improvement of the present application is that in the step S400, the dynamic update model parameters further include: S401, receiving the group driving behavior statistics shared in the cloud in real time, and the sensitivity factors for speeding, frequent lane changes, and sudden braking Adjust as follows: (5), In expression (5), represents the attenuation coefficient, It represents the count of abnormal events of the type in the current period. Indicates the total number of driving time segments.

[0009] A further improvement of the present application is that the generation of graded warning signals includes: dividing the warning level according to the abnormal probability value, including level one warning, level two warning and level three warning; among which, the level one warning triggers a high-frequency warning sound from the vehicle speaker and a red flashing of the dashboard, and sends an emergency rescue request to the cloud at the same time; the level two warning superimposes a yellow warning box through the head-up display and recommends a safe driving strategy; the level three warning notifies the driver in the form of steering wheel vibration and voice prompts.

[0010] A further improvement of the present application is that the warning method also includes: monitoring the driver's facial expressions and eye movements in real time through a vehicle-mounted camera to calculate a fatigue index; if the fatigue index continuously exceeds a threshold and the vehicle is in automatic driving mode, in response to the driver's confirmation operation, forcibly switching to manual driving mode and recommending the nearest rest area; if the driver does not respond to the forced switch, gradually reducing the vehicle speed and activating the emergency parking assist system.

[0011] A further improvement of the present application is that the warning method also includes: integrating V2X communication data to receive the position, speed and intention information of surrounding vehicles; when predicting potential collision risks, generating a joint obstacle avoidance path through an inter-vehicle collaborative algorithm; superimposing the obstacle avoidance path on the on-board navigation interface in the form of augmented reality, and synchronizing it with the surrounding vehicle control systems.

[0012] A further improvement of the present application is that the warning method also includes: constructing a driving behavior profile on a cloud platform to analyze long-term driving habits and risk tendencies; generating a personalized safety score based on the profile results, and pushing customized training courses through a mobile terminal; marking high-risk driver information and sharing it with the insurance platform and traffic management department.

[0013] A further improvement of the present application is that the warning method also includes: responding to manual adjustments by the driver, supporting customized warning sensitivity, prompting methods and silent periods; recording the driver's response delay and operation correction data to the warning signal for optimizing the model feedback mechanism; if the driver continuously ignores high-risk warnings, activating the remote monitoring service and notifying designated emergency contacts.

[0014] The beneficial effects of the present invention are: significantly improving the accuracy and real-time performance of vehicle monitoring and abnormal behavior warning, and realizing comprehensive monitoring of vehicle operation, environmental perception and driver behavior by integrating multiple information sources. Specifically, the present invention achieves technical improvements through the following aspects: First, the present invention obtains real-time vehicle operation data, environmental perception data and driver behavior data, and performs spatiotemporal synchronization and feature fusion to generate a multi-dimensional fusion feature vector. This step effectively solves the problem of single data source in traditional monitoring methods and improves the comprehensiveness and accuracy of monitoring data.

[0015] Secondly, the present invention uses a deep reinforcement learning framework to build an abnormal behavior detection model, and dynamically updates the model parameters through online incremental learning to adapt to different road types and driving habits. This step not only improves the accuracy of the warning, but also enhances the model's adaptability and robustness.

[0016] In addition, the present invention also realizes the generation of graded warning signals, divides the warning level according to the abnormal probability value, and takes corresponding warning measures. This step not only improves the timeliness of the warning, but also helps the driver to take corresponding countermeasures according to different warning levels, thereby reducing the risk of traffic accidents. These functions further enhance the intelligent level of vehicle monitoring and abnormal behavior warning, and provide a strong guarantee for traffic safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The present invention is a schematic flow chart of a vehicle monitoring and abnormal behavior early warning method with multi-dimensional information fusion. DETAILED DESCRIPTION

[0018] The following will describe various embodiments of the present invention in detail with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0019] Based on the above problems, the inventors provide the following solutions: First, multiple data sources are obtained from the vehicle itself, the environment, and the driver. These data sources include but are not limited to vehicle operation data such as speed, acceleration, braking status, and power system parameters, environmental perception data such as camera images, radar point clouds, and GPS positioning information, and driver behavior data such as steering wheel operation frequency, fatigue status detection results, and line of sight direction. The acquisition of these data sources provides a rich information foundation for subsequent fusion processing.

[0020] Next, the acquired vehicle operation data, environmental perception data, and driver behavior data are processed in time and space to ensure the alignment of timestamps between different data sources and the unification of spatial coordinates. Through spatial registration methods such as the Kalman filter algorithm, the vehicle position information is accurately spatially matched with the radar point cloud data, and key information in the camera image, such as lane lines, obstacle contours, and traffic sign semantic information, is extracted, and redundancy checks are performed with the radar detection results to improve the accuracy and reliability of the data.

[0021] On the basis of data synchronization, we further perform feature fusion on multi-source data. The verified data is input into a multi-layer neural network for feature dimension reduction and fusion processing to generate a multi-dimensional fusion feature vector. This step uses the powerful capabilities of deep learning algorithms to extract feature information that plays a key role in abnormal behavior warning, providing strong support for subsequent abnormal behavior detection.

[0022] Subsequently, the generated multi-dimensional fusion feature vector is input into the preset abnormal behavior detection model for processing. The model is built on the deep reinforcement learning framework and dynamically updates the model parameters through online incremental learning to adapt to different road types and driving habits. The model outputs the abnormal probability value of the current driving scene. If the value exceeds the preset threshold, the early warning mechanism is triggered, a graded early warning signal is generated, and an early warning prompt is issued through the vehicle interactive interface and the cloud platform.

[0023] In terms of the generation of warning signals, the present invention divides different warning levels according to the size of the abnormal probability value, such as level 1 warning, level 2 warning and level 3 warning, and takes corresponding warning measures. The implementation of these warning measures helps to improve the driver's reaction speed and coping ability to abnormal situations.

[0024] In addition, the present invention also provides a wealth of additional functions, such as real-time monitoring of driver fatigue, integration of V2X communication data to predict potential collision risks, and construction of driving behavior profiles to analyze long-term driving habits and risk tendencies. These functions further enhance the intelligence level of vehicle monitoring and abnormal behavior warning, providing more comprehensive protection for traffic safety.

[0025] The technical solution will be described in detail below in conjunction with specific embodiments.

[0026] Example 1 refer to Figure 1 , a vehicle monitoring and abnormal behavior early warning method based on multi-dimensional information fusion, the early warning method comprises the following steps T100-T400: T100, obtaining real-time vehicle operation data, environmental perception data and driver behavior data, wherein the vehicle operation data includes vehicle speed, acceleration, braking status and power system parameters, the environmental perception data includes camera images, radar point clouds and GPS positioning information, and the driver behavior data includes steering wheel operation frequency, fatigue status detection results and line of sight direction; In the embodiment of the present application, vehicle speed: collected in real time through the on-board OBD (on-board diagnostic system) interface or wheel speed sensor, and the pulse signal output by the vehicle ECU (electronic control unit) is parsed in combination with the CAN bus protocol; acceleration: measured by the inertial measurement unit (IMU) integrated in the vehicle chassis or body control module, including real-time data of the three-axis accelerometer; braking status: monitoring the pedal opening and closing degree through the brake pedal position sensor, or reading the hydraulic pressure value from the brake system controller to determine the braking intensity and emergency braking events; power system parameters: obtaining data such as speed, torque, fuel efficiency, battery SOC (state of charge) from the engine control module (ECM) or motor controller, and transmitting it in real time through the CAN bus.

[0027] Camera images are acquired through on-board cameras, which are located in different positions of the vehicle, such as the front, rear, and sides, to collect visual information around the vehicle. Radar point clouds are formed by on-board radars (such as millimeter-wave radars and lidars) that emit electromagnetic waves or laser beams, and reflect echoes to form point cloud data to sense the distance, speed, and direction of surrounding objects. GPS positioning information is determined by receiving satellite signals through an on-board GPS receiver to determine the vehicle's geographic location.

[0028] The steering wheel operation frequency can be obtained by measuring the frequency of the angle change of the steering wheel with the help of the steering wheel angle sensor; the fatigue state detection result usually uses the vehicle-mounted camera to collect the driver's facial image and eye features, and uses image processing and analysis technology, such as detecting the driver's eye closure degree, blinking frequency, head posture, etc., to determine whether the driver is in a fatigue state; it can also be assisted by monitoring the driver's physiological signals, such as heart rate, brain waves, etc. to determine the fatigue state; the line of sight direction can use eye tracking technology to capture the driver's eye gaze direction and eye movement trajectory through the camera in the car. By analyzing these data, the driver's line of sight direction can be determined and the driver's focus point can be understood.

[0029] T200, performing spatiotemporal synchronization and feature fusion on the vehicle operation data, environment perception data and driver behavior data to generate a multi-dimensional fusion feature vector; In the embodiment of the present application, performing spatiotemporal synchronization and feature fusion on data includes: Align asynchronous data streams from different sensors based on timestamps; The vehicle position information and radar point cloud data are spatially aligned using the Kalman filter algorithm; Extract the lane lines, obstacle outlines and traffic sign semantic information from the camera image and perform redundancy check with the radar detection results; The verified data is input into a multi-layer neural network for feature dimension reduction and fusion to generate the multi-dimensional fusion feature vector.

[0030] T300, inputting the multi-dimensional fusion feature vector into a preset abnormal behavior detection model, and outputting an abnormal probability value of the current driving scene; In the embodiment of the present application, the construction of the abnormal behavior detection model includes: S100, define the multi-dimensional features of the model input as , is the feature dimension, including the normalized vehicle speed , lateral acceleration , steering wheel angle , Driver's gaze deviation , Distance to the vehicle ahead and environmental visibility , and satisfy: (1), In expression (1), and Represent the mean and standard deviation of historical vehicle speeds, Indicates the preset maximum lateral acceleration threshold; S200, builds a model based on a deep reinforcement learning framework, including: S201. Define state space ,in, is the LSTM hidden layer state, Represents vector concatenation; S202, map the action space output abnormal probability through Sigmoid activation function , , whose expression is: (2), represents the weight matrix, represents the bias term; S203, define a reward function, and dynamically adjust it according to the warning accuracy. The reward function expression is: ; S300, combining classification cross entropy and reinforcement learning strategy gradient to determine the joint loss function, which is expressed as: (3), In expression (3), represents the true label, where Indicates that the current moment is normal driving behavior. Indicates that the current moment is judged as abnormal driving behavior. represents the balance coefficient, represents the state-action value function approximated by a deep Q-network; S400, dynamically updating model parameters through online incremental learning to adapt to different road types and driving habits, the road types include highways, urban roads and mountain curves, and the weight adjustment formula is: (4), In expression (4), represents the road type adaptation loss, Indicates the scene weights of highways, urban roads, and mountain curves. and represents the learning rate hyperparameter.

[0031] In the step S400, the dynamically updating model parameters further includes: S401, receive the group driving behavior statistics shared in the cloud in real time, and analyze the sensitivity factors of speeding, frequent lane changes, and sudden braking Adjust as follows: (5), In expression (5), represents the attenuation coefficient, It represents the count of abnormal events of the type in the current period. Indicates the total number of driving time segments.

[0032] T400. If the abnormal probability value exceeds a preset threshold, a graded warning signal is generated and a warning prompt is issued through the vehicle-mounted interactive interface and the cloud platform.

[0033] In the embodiment of the present application, generating a graded warning signal includes: The warning levels are divided according to the abnormal probability value, including level 1 warning, level 2 warning and level 3 warning; Among them, the first-level warning triggers a high-frequency warning sound from the vehicle's speakers and a red flashing of the dashboard, while sending an emergency rescue request to the cloud; The second-level warning superimposes a yellow warning box on the head-up display and recommends safe driving strategies. For example, if fatigue driving is detected, the nearest server is displayed and a navigation route is given; The third-level warning notifies the driver by vibrating the steering wheel and providing voice prompts. For example, a prompt tone is issued saying “You are speeding, please slow down.”

[0034] Example 2 Based on Example 1, a vehicle monitoring and abnormal behavior warning method using multi-dimensional information fusion, the warning method further includes: The on-board camera monitors the driver's facial expressions and eye movements in real time to calculate the fatigue index; If the fatigue index exceeds the threshold continuously and the vehicle is in the automatic driving mode, in response to the driver's confirmation operation, the vehicle is forced to switch to the manual driving mode and recommend the nearest rest area; If the driver does not respond to the forced switch, the vehicle speed is gradually reduced and the emergency stop assist system is activated.

[0035] In this embodiment, the driver's fatigue index can be accurately calculated by real-time monitoring of the driver's facial expressions and eye movements. When the fatigue index exceeds the preset threshold continuously and the vehicle is in automatic driving mode, the system will first prompt the driver to confirm the operation. If the driver confirms that he is tired, the system will force the driver to switch to manual driving mode and intelligently recommend the nearest rest area to guide the driver to rest. This design is intended to ensure that the driver can rest in time when he is tired and avoid safety hazards caused by fatigue driving.

[0036] If the driver does not respond after receiving the forced switching prompt, the system will take further safety measures. The system will gradually reduce the speed of the vehicle to reduce the risk of rear-end collisions and other accidents caused by sudden stops. At the same time, the emergency parking assist system will be activated, which can automatically find and guide the vehicle to park safely on the roadside or emergency parking strip to ensure the safety of the vehicle and personnel.

[0037] Example 3 Based on Example 1, a vehicle monitoring and abnormal behavior warning method using multi-dimensional information fusion, the warning method further includes: Integrate V2X communication data to receive location, speed and intention information of surrounding vehicles; When predicting potential collision risks, a joint obstacle avoidance path is generated through the inter-vehicle collaborative algorithm; The obstacle avoidance path is superimposed on the in-vehicle navigation interface in the form of augmented reality and synchronized with the control systems of surrounding vehicles.

[0038] In this embodiment, the integration of V2X communication technology enables the vehicle to receive and process the position, speed and intention information from surrounding vehicles in real time. This information is transmitted through dedicated short-range communication technologies (such as DSRC or C-V2X), ensuring the real-time and reliability of the data. After receiving this information, the vehicle uses the built-in inter-vehicle collaborative algorithm to predict the potential risk of collision. The algorithm comprehensively considers factors such as the current driving status of the vehicle, the movement trajectory of surrounding vehicles, and road conditions, and can accurately assess the potential risk of collision.

[0039] When a potential collision risk is predicted, the inter-vehicle collaborative algorithm will quickly generate a joint obstacle avoidance path. This path is designed to guide the vehicle to safely avoid potential collision points and ensure driving safety. The generated obstacle avoidance path is superimposed on the in-vehicle navigation interface in the form of augmented reality, allowing the driver to intuitively see the obstacles ahead and the obstacle avoidance path. At the same time, the obstacle avoidance path will also be synchronized to the control system of surrounding vehicles so that surrounding vehicles can make corresponding avoidance actions to achieve collaborative obstacle avoidance.

[0040] By integrating V2X communication data in this way, vehicles can not only improve their own driving safety, but also collaborate with surrounding vehicles to jointly improve the safety and efficiency of the entire transportation system.

[0041] Example 4 Based on Example 1, a vehicle monitoring and abnormal behavior warning method using multi-dimensional information fusion, the warning method further includes: Build driving behavior profiles on the cloud platform to analyze long-term driving habits and risk tendencies; Generate personalized safety scores based on profiling results and push customized training courses through mobile terminals; Mark high-risk driver information and share it with insurance platforms and traffic management departments.

[0042] Example 5 Based on Example 1, a vehicle monitoring and abnormal behavior warning method using multi-dimensional information fusion, the warning method further includes: In response to manual adjustments by the driver, support for customizing warning sensitivity, prompting methods and silent periods; Record the driver's response delay and operation correction data to the warning signal to optimize the model feedback mechanism; If the driver continuously ignores high-risk warnings, the remote monitoring service will be activated and the designated emergency contact will be notified.

[0043] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0044] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0045] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0046] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0047] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0048] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0049] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0050] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A vehicle monitoring and abnormal behavior early warning method based on multi-dimensional information fusion, characterized in that: The early warning method comprises: Acquire real-time vehicle operation data, environmental perception data and driver behavior data. The vehicle operation data includes vehicle speed, acceleration, braking status and power system parameters. The environmental perception data includes camera images, radar point clouds and GPS positioning information. The driver behavior data includes steering wheel operation frequency, fatigue status detection results and line of sight direction. Performing spatiotemporal synchronization and feature fusion on the vehicle operation data, environment perception data, and driver behavior data to generate a multi-dimensional fusion feature vector; Inputting the multi-dimensional fusion feature vector into a preset abnormal behavior detection model, and outputting an abnormal probability value of the current driving scene; If the abnormal probability value exceeds the preset threshold, a graded warning signal is generated and a warning prompt is issued through the vehicle-mounted interactive interface and the cloud platform.

2. The vehicle monitoring and abnormal behavior early warning method based on multi-dimensional information fusion according to claim 1 is characterized in that: The spatiotemporal synchronization and feature fusion of data includes: Align asynchronous data streams from different sensors based on timestamps; The vehicle position information and radar point cloud data are spatially aligned using the Kalman filter algorithm; Extract the lane lines, obstacle outlines and traffic sign semantic information from the camera image and perform redundancy check with the radar detection results; The verified data is input into a multi-layer neural network for feature dimension reduction and fusion to generate the multi-dimensional fusion feature vector.

3. The vehicle monitoring and abnormal behavior early warning method based on multi-dimensional information fusion according to claim 1 is characterized in that: The construction of the abnormal behavior detection model includes: S100, define the multi-dimensional features of the model input as , is the feature dimension, including the normalized vehicle speed , lateral acceleration , steering wheel angle , Driver's gaze deviation , Distance to the vehicle ahead and environmental visibility , and satisfy: （1）， In expression (1), and Represent the mean and standard deviation of historical vehicle speeds, Indicates the preset maximum lateral acceleration threshold; S200, builds a model based on a deep reinforcement learning framework, including: S201. Define state space ,in, is the LSTM hidden layer state, Represents vector concatenation; S202, map the action space output abnormal probability through Sigmoid activation function , , whose expression is: （2）， represents the weight matrix, represents the bias term; S203, define a reward function, and dynamically adjust it according to the warning accuracy. The reward function expression is: ； S300, combining classification cross entropy and reinforcement learning strategy gradient to determine the joint loss function, which is expressed as: （3）， In expression (3), represents the true label, where Indicates that the current moment is normal driving behavior. Indicates that the current moment is judged as abnormal driving behavior. represents the balance coefficient, represents the state-action value function approximated by a deep Q-network; S400, dynamically updating model parameters through online incremental learning to adapt to different road types and driving habits, the road types include highways, urban roads and mountain curves, and the weight adjustment formula is: （4）， In expression (4), represents the road type adaptation loss, Indicates the scene weights of highways, urban roads, and mountain curves. and represents the learning rate hyperparameter.

4. The vehicle monitoring and abnormal behavior early warning method based on multi-dimensional information fusion according to claim 3 is characterized in that: In the step S400, the dynamically updating model parameters further includes: S401, receive the group driving behavior statistics shared in the cloud in real time, and analyze the sensitivity factors of speeding, frequent lane changes, and sudden braking Adjust as follows: （5）， In expression (5), represents the attenuation coefficient, It represents the count of abnormal events of the type in the current period. Indicates the total number of driving time segments.

5. The vehicle monitoring and abnormal behavior early warning method based on multi-dimensional information fusion according to claim 1 is characterized in that: Generating a graded warning signal comprises: The warning levels are divided according to the abnormal probability value, including level 1 warning, level 2 warning and level 3 warning; Among them, the first-level warning triggers a high-frequency warning sound from the vehicle's speakers and a red flashing of the dashboard, while sending an emergency rescue request to the cloud; The second-level warning superimposes a yellow warning box on the head-up display and recommends safe driving strategies; The third-level warning notifies the driver by vibrating the steering wheel and providing voice prompts.

6. The vehicle monitoring and abnormal behavior early warning method based on multi-dimensional information fusion according to claim 1 is characterized in that: The early warning method also includes: The on-board camera monitors the driver's facial expressions and eye movements in real time to calculate the fatigue index; If the fatigue index exceeds the threshold continuously and the vehicle is in the automatic driving mode, in response to the driver's confirmation operation, the vehicle is forced to switch to the manual driving mode and recommend the nearest rest area; If the driver does not respond to the forced switch, the vehicle speed is gradually reduced and the emergency stop assist system is activated.

7. The vehicle monitoring and abnormal behavior early warning method based on multi-dimensional information fusion according to claim 1 is characterized in that: The early warning method also includes: Integrate V2X communication data to receive location, speed and intention information of surrounding vehicles; When predicting potential collision risks, a joint obstacle avoidance path is generated through the inter-vehicle collaborative algorithm; The obstacle avoidance path is superimposed on the in-vehicle navigation interface in the form of augmented reality and synchronized with the control systems of surrounding vehicles.

8. The vehicle monitoring and abnormal behavior early warning method based on multi-dimensional information fusion according to claim 1 is characterized in that: The early warning method also includes: Build driving behavior profiles on the cloud platform to analyze long-term driving habits and risk tendencies; Generate personalized safety scores based on profiling results and push customized training courses through mobile terminals; Mark high-risk driver information and share it with insurance platforms and traffic management departments.

9. The vehicle monitoring and abnormal behavior early warning method based on multi-dimensional information fusion according to claim 1 is characterized in that: The early warning method also includes: In response to manual adjustments by the driver, support for customizing warning sensitivity, prompting methods and silent periods; Record the driver's response delay and operation correction data to the warning signal to optimize the model feedback mechanism; If the driver continuously ignores high-risk warnings, the remote monitoring service will be activated and the designated emergency contact will be notified.

Images