Fatigue driving prevention system based on large model

Abstract

The invention provides an anti-fatigue driving method and system based on a large model. The method comprises the following steps: acquiring multi-source heterogeneous data; based on a fatigue state recognition model in the large model, the fatigue state of the driver is output according to the multi-source heterogeneous data; outputting an anti-fatigue strategy based on an anti-fatigue strategy generation model in the large model according to the portrait of the driver, the fatigue state and the vehicle driving data; and generating a control instruction of the vehicle based on the anti-fatigue strategy, and controlling the vehicle to execute an anti-fatigue operation in the anti-fatigue strategy according to the control instruction, so that the fatigue state of the driver can be accurately and efficiently identified, and the anti-fatigue operation of the vehicle can be accurately and efficiently identified based on the anti-fatigue strategy generation model according to the portrait of the driver, the fatigue state and the vehicle driving data. The accuracy and the flexibility of the output anti-fatigue strategy are improved, the requirement of a complex driving scene is met, the safety risk caused by fatigue driving is avoided, and the safety in the vehicle driving process is effectively guaranteed.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent driving technology, specifically to a fatigue-prevention driving system based on a large model. Background Technology

[0002] Driver fatigue is one of the core causes of traffic accidents. According to data from the Ministry of Transport, approximately 20% of highway traffic accidents are directly related to driver fatigue. Existing Driver Monitoring Systems (DMS) mainly use a monocular camera to collect driver facial features and combine them with preset thresholds to identify fatigue states. However, this method cannot accurately identify driver fatigue states, and the output fatigue prevention strategies are general-purpose, failing to consider individual driver differences, thus unable to meet the needs of complex driving scenarios and unable to guarantee vehicle driving safety. Summary of the Invention

[0003] This disclosure aims to at least partially address one of the technical problems in the related art.

[0004] According to a first aspect of this disclosure, a fatigue-prevention driving method based on a large model is provided, comprising: acquiring multi-source heterogeneous data, wherein the multi-source heterogeneous data includes at least driver physiological data and vehicle driving data; a fatigue state recognition model based on the large model outputting the driver's fatigue state based on the multi-source heterogeneous data; a fatigue-prevention strategy generation model based on the large model outputting a fatigue-prevention strategy based on the driver's profile, the fatigue state, and the vehicle driving data; generating control commands for the vehicle based on the fatigue-prevention strategy; and controlling the vehicle to execute fatigue-prevention operations in the fatigue-prevention strategy according to the control commands.

[0005] The fatigue-prevention driving system based on a large model, as proposed in the second aspect of this disclosure, includes: an acquisition module for acquiring multi-source heterogeneous data, wherein the multi-source heterogeneous data includes at least the driver's physiological data and vehicle driving data; a first output module for outputting the driver's fatigue state based on a fatigue state recognition model in the large model and the multi-source heterogeneous data; a second output module for outputting a fatigue-prevention strategy based on a fatigue-prevention strategy generation model in the large model and the driver's profile, the fatigue state, and the vehicle driving data; and a control module for generating control commands for the vehicle based on the fatigue-prevention strategy and controlling the vehicle to execute fatigue-prevention operations in the fatigue-prevention strategy according to the control commands.

[0006] According to a third aspect of this disclosure, an electronic device is proposed, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the large-model-based anti-fatigue driving method proposed in the first aspect above.

[0007] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the fatigue-prevention driving method based on a large model proposed in the first aspect above.

[0008] According to the fifth aspect of this disclosure, a computer program product is proposed, comprising a computer program that, when executed by a processor, implements the fatigue-prevention driving method based on a large model as proposed in the first aspect.

[0009] The fatigue-prevention driving system based on a large-scale model disclosed herein acquires multi-source heterogeneous data, including at least driver physiological data and vehicle driving data. A fatigue state identification model within the large-scale model outputs the driver's fatigue state based on the multi-source heterogeneous data. A fatigue-prevention strategy generation model within the large-scale model outputs a fatigue-prevention strategy based on the driver's profile, fatigue state, and vehicle driving data. Based on the fatigue-prevention strategy, vehicle control commands are generated, and the vehicle is controlled to execute the fatigue-prevention operations within the strategy. Therefore, the fatigue state identification model based on the large-scale model can accurately and efficiently identify the driver's fatigue state based on multi-source heterogeneous data, and the fatigue-prevention strategy generation model can output a fatigue-prevention strategy based on the driver's profile, fatigue state, and vehicle driving data. This improves the accuracy and flexibility of the output fatigue-prevention strategy, meets the needs of complex driving scenarios, avoids safety risks caused by fatigue driving, and effectively ensures safety during vehicle driving.

[0010] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0011] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which: Figure 1 This is a flowchart illustrating a fatigue-prevention driving method based on a large model proposed in an embodiment of this disclosure; Figure 2 This is a flowchart illustrating a fatigue-prevention driving method based on a large model proposed in an embodiment of this disclosure; Figure 3 This is a schematic diagram of the structure of a fatigue driving prevention method based on a large model proposed in an embodiment of this disclosure; Figure 4 This is a schematic diagram of the structure of a fatigue-prevention driving system based on a large model proposed in an embodiment of this disclosure; Figure 5 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown.

[0012] Figure 6 This is a schematic diagram of the structure of a vehicle according to an embodiment of the present disclosure; Detailed Implementation Embodiments of this disclosure are described in detail below, with examples of embodiments illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are used only to explain this disclosure, and should not be construed as limiting this disclosure. Rather, embodiments of this disclosure include all variations, modifications, and equivalents falling within the spirit and scope of the appended claims.

[0013] Figure 1 This is a schematic flowchart of a fatigue-prevention driving method based on a large model proposed in one embodiment of this disclosure.

[0014] like Figure 1 As shown, the fatigue driving prevention method based on a large model proposed in this embodiment specifically includes the following steps: S101. Acquire multi-source heterogeneous data, wherein the multi-source heterogeneous data includes at least the driver's physiological data and vehicle driving data.

[0015] It should be noted that various types of sensors can be deployed to collect driver physiological data and vehicle driving data, thereby obtaining multi-source heterogeneous data.

[0016] For example, infrared binocular cameras collect facial feature data of the driver, such as blinking frequency, eye movement trajectory, facial muscle relaxation, and the wearing status of masks / sunglasses. A photoplethysmo graph (PPG) embedded in the steering wheel collects heart rate, blood oxygen saturation, etc. at a sampling frequency of 50Hz. A seat pressure sensor collects the frequency of posture changes at a sampling frequency of 10Hz. The above data are summarized to obtain the driver's physiological data.

[0017] Optionally, the infrared binocular camera has 2 megapixels, a frame rate of 30fps, and can still image in low light conditions of ≤0.01 lux to ensure that facial feature data of the driver can be obtained.

[0018] For example, vehicle driving behavior data such as steering wheel angle fluctuation frequency, brake / accelerator pedal operation frequency and force, vehicle speed stability, and following distance changes are acquired through the Controller Area Network (CAN) bus / On-Board Diagnostics (OBD) interface at an update frequency of 50 Hz. The vehicle driving route (e.g., highway / urban / mountainous area) is acquired through the vehicle's Global Positioning System (GPS) at a positioning accuracy of 10m and an update frequency of 1 Hz. The vehicle's external front-view camera collects vehicle driving environment data such as light intensity and weather conditions (e.g., sunny / rainy / snowy / foggy) at a frame rate of 25fps. The continuous driving time of the vehicle is collected through the clock module at an accuracy of 1 second. The degree of road congestion is collected through the vehicle radar (detection distance 0-200m). The above data are summarized to obtain vehicle driving data.

[0019] S102. The fatigue state recognition model based on the large model outputs the driver's fatigue state based on multi-source heterogeneous data.

[0020] In this embodiment of the disclosure, after obtaining multi-source heterogeneous data, the multi-source heterogeneous data can be preprocessed to obtain target multi-source heterogeneous data, feature extraction can be performed on the target multi-source heterogeneous data to obtain target feature vectors, and the target feature vectors can be input into the fatigue state recognition model to obtain the driver's fatigue state.

[0021] S103. The anti-fatigue strategy generation model based on the large model outputs an anti-fatigue strategy based on the driver's profile, fatigue state and vehicle driving data.

[0022] Optionally, training samples can be obtained, which include the profiles of sample drivers, sample fatigue states, sample vehicle driving data, and corresponding anti-fatigue strategies. Based on the training samples, the anti-fatigue strategy generation model to be trained is trained to obtain the anti-fatigue strategy generation model.

[0023] In this embodiment of the disclosure, after obtaining the anti-fatigue strategy generation model, the anti-fatigue strategy generation model can output an anti-fatigue strategy based on the driver's profile, fatigue state, and vehicle driving data.

[0024] S104. Based on the anti-fatigue strategy, generate control commands for the vehicle, and control the vehicle to execute the anti-fatigue operation in the anti-fatigue strategy according to the control commands.

[0025] In this embodiment, the fatigue prevention intervention strategy can be decomposed to generate a first control command for the on-board device and a second control command for the vehicle control device. Based on the first control command, the on-board device is controlled to execute the fatigue prevention operation within the fatigue prevention strategy; based on the second control command, the vehicle control device is controlled to execute the fatigue prevention operation within the fatigue prevention strategy. The fatigue-prevention driving method based on a large model proposed in this disclosure acquires multi-source heterogeneous data, including at least driver physiological data and vehicle driving data. A fatigue state identification model within the large model outputs the driver's fatigue state based on the multi-source heterogeneous data. A fatigue-prevention strategy generation model within the large model outputs a fatigue-prevention strategy based on the driver's profile, fatigue state, and vehicle driving data. Based on the fatigue-prevention strategy, vehicle control commands are generated, and the vehicle is controlled to execute the fatigue-prevention operations within the strategy. Therefore, the fatigue state identification model based on the large model can accurately and efficiently identify the driver's fatigue state based on multi-source heterogeneous data, and the fatigue-prevention strategy generation model can output a fatigue-prevention strategy based on the driver's profile, fatigue state, and vehicle driving data. This improves the accuracy and flexibility of the output fatigue-prevention strategy, meets the needs of complex driving scenarios, avoids safety risks caused by fatigue driving, and effectively ensures the safety of vehicle driving.

[0026] Figure 2 This is a schematic flowchart of a fatigue-prevention driving method based on a large model proposed in one embodiment of this disclosure.

[0027] like Figure 2 As shown, the fatigue driving prevention method based on a large model proposed in this embodiment specifically includes the following steps: S201. Obtain multi-source heterogeneous data, wherein the multi-source heterogeneous data includes at least the driver's physiological data and vehicle driving data.

[0028] S202. Preprocess the multi-source heterogeneous data to obtain the target multi-source heterogeneous data.

[0029] Optionally, unstructured data (such as image and video data) in multi-source heterogeneous data can be normalized to a size of 640×480 pixels and denoised using a Gaussian filtering algorithm. Structured data (such as vehicle speed and heart rate) in multi-source heterogeneous data can be outlier removed based on the 3σ principle and standardized to map to the 0-1 interval to obtain the target multi-source heterogeneous data.

[0030] S203. Extract features from the target multi-source heterogeneous data to obtain the target feature vector.

[0031] In this embodiment of the disclosure, after obtaining the target multi-source heterogeneous data, feature extraction is performed on the target multi-source heterogeneous data to obtain the target feature vector.

[0032] The target feature vector can be a 128-dimensional standardized feature vector.

[0033] S204. Input the target feature vector into the fatigue state recognition model and output the driver's fatigue state.

[0034] In this embodiment of the disclosure, training data is acquired, which includes sample multi-source heterogeneous data and corresponding fatigue states. Based on the training data and fine-tuning strategy, the Transformer model is fine-tuned and trained to obtain a fatigue state recognition model.

[0035] It should be noted that the training data covers different driving scenarios (highway / urban / mountainous areas, sunny / rainy / snowy / foggy days, daytime / nighttime), different groups of people (aged 20-60, male-female ratio 1:1), and different fatigue states (awake / mildly fatigued / moderately fatigued / severely fatigued).

[0036] For example, a fatigue state recognition model can learn the combined features of "nighttime + continuous driving duration > 4 hours + steering wheel fine-tuning frequency > 5 times / min + heart rate < 60 beats / minute" to output a severe fatigue state.

[0037] Optionally, a fine-tuning strategy for the multimodal Transformer model can be obtained, freezing the parameters of the bottom 6 layers of the Transformer model, training only the top 4 layers of feature association and classification layers, using the Adam optimizer (learning rate 1e-4), training 50 epochs, batch size=32, and after training, the model parameters are optimized to 480M, and the inference latency is ≤80ms, meeting the real-time operation requirements of in-vehicle hardware.

[0038] In this embodiment of the disclosure, after obtaining the fatigue state recognition model that has been trained, the target feature vector can be input into the fatigue state recognition model, and the driver's fatigue state can be output by associating the features of the target feature vector through an attention mechanism.

[0039] S205. The anti-fatigue strategy generation model based on the large model outputs an anti-fatigue strategy based on the driver's profile, fatigue state, and vehicle driving data.

[0040] In this embodiment of the disclosure, the driver's identity information is obtained. In response to the mismatch between the identity information and the stored driver profile, an information request prompt is sent to the driver. Input information from the driver is received, and a driver profile is constructed based on the input information.

[0041] Optionally, in response to a mismatch between the identity information and the stored driver profile, the driver can input basic information via the in-vehicle central control screen, such as age, dialect preference, sensitivity level to sound / vibration (level 1-5), and whether they have underlying diseases such as heart disease / hypertension. Based on the driver's input information, a driver profile (initial profile) is constructed.

[0042] In this embodiment of the disclosure, a vehicle driving dataset of the driver within a preset time period is obtained, the vehicle driving dataset of the driver within the preset time period is analyzed to obtain the driver's driving habits, and the driver's profile is updated based on the driving habits.

[0043] For example, a vehicle driving dataset can be obtained every 10 hours. The dataset can be automatically analyzed using a large model to obtain the driver's driving habits, such as regular rest intervals, lane preferences, and response speed to anti-fatigue intervention strategies. Based on these driving habits, the driver's profile can be updated.

[0044] It should be noted that, in order to ensure the privacy and security of drivers, after obtaining the driver's profile, an encryption algorithm can be used to encrypt and store the driver's profile.

[0045] In this embodiment of the disclosure, the large model calls the driver's profile, and the anti-fatigue strategy generation model in the large model outputs the anti-fatigue strategy based on the driver's profile, fatigue state and vehicle driving data.

[0046] For example, in response to a fatigue state that is at rest, the fatigue prevention strategy records only normal driving data; in response to a mild fatigue state, the fatigue prevention strategy includes a gradual green light change on the instrument panel with a brightness variation of 50%, personalized voice prompts such as those in the driver's common dialect, and adjusting the air conditioning temperature to 24°C and fan speed to level 2, providing both visual and voice warnings; in response to a moderate fatigue state, the fatigue prevention strategy includes a flashing yellow light on the instrument panel at a frequency of 1Hz, zonal vibration on the left / right side of the seat at a frequency of 2Hz, and navigation guidance prompts (such as three pop-up navigation icons on the central control screen for highway driving). Near service areas, the distance and estimated arrival time are displayed. For example, in urban areas, the central control screen will display three temporary parking spots with their distance and estimated arrival time, providing visual warnings, voice prompts, and navigation guidance. In response to severe fatigue, the fatigue prevention strategy includes flashing red lights on the instrument panel and displaying the text "Rest Now" on the head-up screen, full-area seat vibration and steering wheel vibration, forced intervention (reducing the vehicle speed to 80% of the current road speed limit and activating the hazard lights), switching the air conditioning to external circulation and increasing the fan speed, and emergency linkage (sending location information to preset emergency contacts), providing visual warnings, voice prompts, and forced intervention.

[0047] S206. Decompose the anti-fatigue strategy to generate the first control command for the on-board equipment and the second control command for the vehicle control equipment.

[0048] For example, in-vehicle equipment includes at least in-vehicle visual equipment such as instrument panel, central control screen and head-up display (HUD), in-vehicle auditory equipment such as in-vehicle audio and in-vehicle speakers, in-vehicle tactile equipment such as seats and steering wheel, and in-vehicle sensing equipment such as in-vehicle air conditioning; vehicle control equipment includes at least accelerator, brake, hazard lights, and windows.

[0049] S207. According to the first control command, control the on-board equipment to perform the anti-fatigue operation in the anti-fatigue strategy.

[0050] Optionally, the vehicle-mounted equipment can be controlled to perform anti-fatigue operations in the anti-fatigue strategy via the vehicle-mounted CAN bus / Ethernet according to the first control command.

[0051] For example, in response to a state of severe fatigue, the system controls the flashing of the red lights on the instrument panel and the "Rest Now" text displayed on the head-up screen, controls the vibration of the entire seat and steering wheel, controls the air conditioning to switch to external circulation and increases the fan speed, and controls the sending of location information to preset emergency contacts.

[0052] S208. According to the second control command, control the vehicle control equipment to execute the anti-fatigue operation in the anti-fatigue strategy.

[0053] Optionally, the vehicle control equipment can be controlled to perform anti-fatigue operations in the anti-fatigue strategy via the vehicle CAN bus / Ethernet according to the second control command.

[0054] For example, in response to a state of severe fatigue, the vehicle speed is reduced to 80% of the current road speed limit, and the hazard lights are turned on.

[0055] In this embodiment of the disclosure, driver response data to anti-fatigue operations can be collected, and the anti-fatigue strategy generation model can be incrementally trained based on the newly added multi-source heterogeneous data and the response data.

[0056] For example, response data such as whether drivers follow navigation to service areas and whether their fatigue status has improved can be collected. Based on newly added multi-source heterogeneous data (such as facial feature data when wearing occlusion, heavy rain, etc.), the anti-fatigue strategy generation model can be incrementally trained through a few-shot learning algorithm. The model parameters are automatically updated once a week (only the top classification layer is updated, and the training time is <10 minutes), so as to achieve model self-evolution and enhance scene adaptability.

[0057] The fatigue-prevention driving method based on a large model proposed in this disclosure acquires multi-source heterogeneous data, which includes at least the driver's physiological data and vehicle driving data. The multi-source heterogeneous data is preprocessed to obtain target multi-source heterogeneous data. Feature extraction is performed on the target multi-source heterogeneous data to obtain target feature vectors. These target feature vectors are input into a fatigue state recognition model to obtain the driver's fatigue state. Based on the driver's profile, fatigue state, and vehicle driving data, a fatigue-prevention strategy generation model in the large model outputs a fatigue-prevention strategy. The fatigue-prevention intervention strategy is decomposed to generate a first control command for the on-board equipment and a second control command for the vehicle control equipment. According to the first control command, the on-board equipment is controlled to execute the fatigue-prevention operation in the fatigue-prevention strategy. According to the second control command, the vehicle control equipment is controlled to execute the fatigue-prevention operation. This invention controls the vehicle control equipment to execute anti-fatigue operations within an anti-fatigue strategy. By inputting the target feature vector into a fatigue state recognition model, the driver's fatigue state is obtained, avoiding misjudgments and omissions caused by a single data dimension. This improves the accuracy and anti-interference capability of fatigue state recognition. Based on the driver's profile, fatigue state, and vehicle driving data, a personalized anti-fatigue strategy can be output, avoiding the limitations of a single, fixed anti-fatigue strategy. Furthermore, according to a first control command, the on-board equipment is controlled to execute the anti-fatigue operations within the anti-fatigue strategy; according to a second control command, the vehicle control equipment is controlled to execute the anti-fatigue operations within the anti-fatigue strategy. This improves the effectiveness and intelligence of anti-fatigue driving, reduces safety risks caused by fatigued driving, and enhances vehicle driving safety.

[0058] Figure 3 This is a schematic flowchart of a fatigue-prevention driving method based on a large model proposed in one embodiment of this disclosure.

[0059] like Figure 3 As shown, the fatigue driving prevention method based on a large model proposed in this embodiment specifically includes the following steps: S301. Acquire multi-source heterogeneous data, wherein the multi-source heterogeneous data includes at least the driver's physiological data and vehicle driving data.

[0060] S302. The fatigue state recognition model based on the large model outputs the driver's fatigue state based on multi-source heterogeneous data.

[0061] S303. The anti-fatigue strategy generation model based on the large model outputs an anti-fatigue strategy based on the driver's profile, fatigue state, and vehicle driving scenarios in the vehicle driving data.

[0062] S304. Based on the anti-fatigue strategy, generate control commands for the vehicle, and control the vehicle to execute the anti-fatigue operation in the anti-fatigue strategy according to the control commands.

[0063] S305. Upload the driver's fatigue status, fatigue driving intervention strategies, and response data to the vehicle management platform associated with the vehicle.

[0064] S306, Receive fatigue driving intervention instructions sent by the vehicle management platform.

[0065] S307. In response to the driver's fatigue state and the response data meeting preset conditions, the vehicle's location is reported to the traffic management platform.

[0066] For example, in response to the driver's fatigue state being severe and the driver being unresponsive, if the preset conditions are met, the vehicle's location is automatically reported to the traffic management platform to request roadside assistance.

[0067] The fatigue driving prevention method based on a large model proposed in this disclosure acquires multi-source heterogeneous data, including at least driver physiological data and vehicle driving data. A fatigue state recognition model within the large model outputs the driver's fatigue state based on the multi-source heterogeneous data. A fatigue prevention strategy generation model within the large model outputs a fatigue prevention strategy based on the driver's profile, fatigue state, and vehicle driving scenario from the vehicle driving data. Based on the fatigue prevention strategy, vehicle control commands are generated. According to the control commands, the vehicle executes the fatigue prevention operations within the fatigue prevention strategy. The driver's fatigue state, fatigue driving intervention strategy, and response data are uploaded to a vehicle management platform associated with the vehicle. The method receives fatigue driving intervention commands from the vehicle management platform and, in response to the driver's fatigue state and response data meeting preset conditions, reports the vehicle's location to the traffic management platform. Thus, this disclosure achieves multi-platform linkage, effectively reducing the safety risks caused by fatigue driving, achieving the effect of fatigue driving prevention, and improving the safety of vehicle driving.

[0068] The following uses a heavy commercial vehicle as an example to explain the specific process of the fatigue-prevention driving method based on a large model proposed in this disclosure.

[0069] For example, the hardware deployment for logistics trucks is as follows: (1) Data acquisition hardware deployment: A 2-megapixel infrared binocular camera with a lens angle of 120° is deployed inside the A-pillar of the driver's cab of the logistics truck, covering the driver's face area; two PPG photoplethysmography sensors are embedded at the 3 o'clock and 9 o'clock positions of the steering wheel, with a sampling frequency of 50Hz; four pressure sensors are embedded in the seat cushion and backrest respectively, with a sampling frequency of 10Hz, to collect the driver's physiological data, which is then connected to the vehicle's CAN bus through the OBD interface to read parameters such as steering wheel angle (accuracy ±0.1°), accelerator pedal opening (accuracy ±1%), brake pedal travel (accuracy ±1mm), and vehicle speed (accuracy ±0.5km / h). The update frequency is 50Hz. By integrating the vehicle GPS module, the positioning accuracy is 10m. The driving route is obtained. The external front view camera has a frame rate of 25fps and supports the recognition of light intensity and weather conditions. The vehicle millimeter-wave radar judges the degree of road congestion in order to collect vehicle driving data and obtain multi-source heterogeneous data. (2) Execution layer hardware deployment: The vehicle instrument panel adopts a 12.3-inch LCD screen (supports multi-color light gradient and flashing); HUD head-up display with a projection distance of 2.5m; the seat adopts a partitioned vibration seat and supports left / right / full area vibration; the vehicle audio system supports the import of custom voice packages; the vehicle control equipment supports limiting the throttle opening and turning on the hazard lights through the CAN bus.

[0070] For example, for the high-speed night driving scenario of logistics trucks: (1) Data collection stage: After the vehicle starts, the data collection hardware (perception layer) collects data in real time and transmits standardized data to the processing layer of the AI ​​big model once every 50ms (such as "23:00 at night, continuous driving time of 3.8h, heart rate of 58 beats / minute, steering wheel fine adjustment frequency of 4.5 times / min, vehicle speed of 90km / h, highway section, no precipitation"); (2) Fatigue state recognition stage: After the AI ​​big model receives the data, it outputs the driver's fatigue based on the characteristics of "night + continuous driving time of nearly 4h + low heart rate + frequent steering wheel fine adjustment" through the fatigue state recognition model in the big model. The state is moderate fatigue; (3) Anti-fatigue strategy generation stage: The AI ​​big model calls the driver profile "age 45 years old, male, dialect preference Sichuan dialect, vibration sensitivity level 3, no underlying diseases, regular rest interval 4h", combined with the high-speed driving scenario, generates the anti-fatigue strategy as "yellow light flashing on the dashboard (frequency 1Hz), left / right side vibration of the seat (frequency 2Hz), Sichuan dialect voice prompt 'You have been driving continuously for 3.8h, there is XX service area 5km ahead, do you want to go?', the central control screen pops up the service area location and estimated arrival time (8 minutes)"; (4) Anti-fatigue strategy Slight execution phase: The instrument panel, seat, audio, central control, and vehicle navigation are controlled to simultaneously execute the anti-fatigue operation in the anti-fatigue strategy; (5) Feedback iteration phase: If the driver clicks "Yes" on the central control screen and arrives at the service area to rest 8 minutes later, the system collects "the fatigue state has improved to the awake state after intervention, and the heart rate has risen to 72 beats / minute", and determines that the anti-fatigue strategy is effective. The large model records the effectiveness of the anti-fatigue strategy in the "high-speed night and moderate fatigue" scenario, and recommends it first in the same scenario in the future; If the driver ignores the anti-fatigue strategy, the anti-fatigue strategy is upgraded after 30 seconds "the red light on the instrument panel flashes frequently, and the square The steering wheel vibrates, the vehicle air conditioner switches to external circulation and increases the fan speed to level 4, and plays upbeat music. At the same time, the information "driver is moderately fatigued and unresponsive" is uploaded to the fleet management platform. If the driver still does not respond after 1 minute, the vehicle speed is automatically reduced to 72km / h, the hazard lights are turned on, and the vehicle location and fatigue status information are sent to the preset emergency contact. (6) Model iteration stage: Automatically summarize weekly driving data (including status recognition results, anti-fatigue strategy, and response data), and incrementally train the anti-fatigue strategy generation model through a small sample learning algorithm to complete the parameter update of the anti-fatigue strategy generation model.

[0071] In summary, the fatigue-prevention driving method based on a large model proposed in this disclosure effectively avoids misjudgment and missed judgment of fatigue state by integrating multi-dimensional features through a large model, thereby improving the accuracy and anti-interference ability of fatigue state identification. Based on the driver's profile, fatigue state, and vehicle driving data, it can output personalized fatigue-prevention strategies, improving the driver's experience and avoiding the limitations of a single fatigue-prevention strategy. It achieves closed-loop management through multi-system linkage, enhancing the effectiveness of fatigue-prevention driving. It not only improves the driving safety of individual vehicles but also supports large-scale fleet management, reducing overall operational risks and improving the safety of vehicle driving. It can be applied to various driving vehicles such as passenger cars, commercial vehicles, construction machinery, and rail transit trains.

[0072] Figure 4 This is a schematic diagram of the structure of a fatigue-prevention driving system based on a large model proposed in one embodiment of this disclosure.

[0073] like Figure 4 As shown, the fatigue prevention driving system 1000 based on a large model includes: an acquisition module 101, a first output module 102, a second output module 103, and a control module 104.

[0074] The acquisition module 101 is used to acquire multi-source heterogeneous data, wherein the multi-source heterogeneous data includes at least the driver's physiological data and vehicle driving data; The first output module 102 is used to output the driver's fatigue state based on the fatigue state recognition model in the large model according to the multi-source heterogeneous data; The second output module 103 is used to generate a model based on the anti-fatigue strategy in the large model, and output the anti-fatigue strategy according to the driver's profile, the fatigue state and the vehicle driving data. The control module 104 is used to generate control commands for the vehicle based on the anti-fatigue strategy, and control the vehicle to perform anti-fatigue operations in the anti-fatigue strategy according to the control commands.

[0075] In some embodiments of this disclosure, the first output module 102 is further configured to: preprocess the multi-source heterogeneous data to obtain target multi-source heterogeneous data; extract features from the target multi-source heterogeneous data to obtain target feature vectors; input the target feature vectors into the fatigue state recognition model and output the driver's fatigue state.

[0076] In some embodiments of this disclosure, the training process of the fatigue state recognition model includes: acquiring training data, wherein the training data includes sample multi-source heterogeneous data and corresponding fatigue levels; and fine-tuning the Transformer model according to the training data and a fine-tuning strategy to obtain the fatigue state recognition model.

[0077] In some embodiments of this disclosure, the process of constructing the driver's profile includes: obtaining the driver's identity information; in response to the identity information not matching a stored driver profile, sending an information request prompt to the driver; receiving the driver's input information, and constructing the driver's profile based on the input information.

[0078] In some embodiments of this disclosure, the system 1000 is further configured to: acquire a vehicle driving dataset of the driver within a preset time period; analyze the vehicle driving dataset of the driver within the preset time period to acquire the driver's driving habits; and update the driver's profile based on the driving habits.

[0079] In some embodiments of this disclosure, the control module 104 is further configured to: decompose the anti-fatigue intervention strategy to generate a first control instruction for the vehicle-mounted device and a second control instruction for the vehicle control device; control the vehicle-mounted device to execute the anti-fatigue operation in the anti-fatigue strategy according to the first control instruction; and control the vehicle control device to execute the anti-fatigue operation in the anti-fatigue strategy according to the second control instruction.

[0080] In some embodiments of this disclosure, the system 1000 is further configured to: collect the driver's response data to the anti-fatigue operation; and incrementally train the anti-fatigue strategy generation model based on the newly added multi-source heterogeneous data and the response data.

[0081] In some embodiments of this disclosure, the system 1000 is further configured to: upload the driver's fatigue state, the fatigue driving intervention strategy, and the response data to a vehicle management platform associated with the vehicle; and receive fatigue driving intervention instructions sent by the vehicle management platform.

[0082] In some embodiments of this disclosure, the system 1000 is further configured to: report the location of the vehicle to a traffic management platform in response to the driver's fatigue state and the response data meeting preset conditions.

[0083] It should be noted that the foregoing explanation of the fatigue-prevention method based on large models also applies to the fatigue-prevention system based on large models in this embodiment, and will not be repeated here.

[0084] In this embodiment, by acquiring multi-source heterogeneous data, including at least the driver's physiological data and vehicle driving data, a fatigue state recognition model based on a large model outputs the driver's fatigue state based on the multi-source heterogeneous data. A fatigue prevention strategy generation model based on the large model outputs a fatigue prevention strategy based on the driver's profile, fatigue state, and vehicle driving data. Based on the fatigue prevention strategy, vehicle control commands are generated, and the vehicle is controlled to execute the fatigue prevention operations within the strategy according to the control commands. Therefore, this disclosure, based on the fatigue state recognition model in a large model using multi-source heterogeneous data, can accurately and efficiently identify the driver's fatigue state, and the fatigue prevention strategy generation model in the large model can output a fatigue prevention strategy based on the driver's profile, fatigue state, and vehicle driving data. This improves the accuracy and flexibility of the output fatigue prevention strategy, meets the needs of complex driving scenarios, avoids safety risks caused by fatigued driving, and effectively ensures safety during vehicle driving.

[0085] Figure 5 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Figure 5 The electronic device 2000 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0086] like Figure 5 As shown, the electronic device 2000 is presented in the form of a general-purpose computing device. The components of the electronic device 2000 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0087] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0088] Electronic device 2000 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 2000, including volatile and non-volatile media, removable and non-removable media.

[0089] Memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and / or cache memory 32. Electronic device 2000 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 5 Not shown; usually referred to as a "hard drive".

[0090] although Figure 5 As not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disc drive for reading and writing to a removable non-volatile optical disc (e.g., a compact disc read-only memory (CD-ROM), a digital video disc read-only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this disclosure.

[0091] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of this disclosure.

[0092] Electronic device 2000 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable human interaction with electronic device 2000, and / or with any device that enables electronic device 2000 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, electronic device 2000 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of electronic device 2000 via bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 2000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0093] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the fatigue-prevention driving method based on a large model mentioned in the foregoing embodiments.

[0094] To achieve the above embodiments, this disclosure also proposes a vehicle, such as... Figure 6 As shown, vehicle 3000 includes electronic equipment 2000.

[0095] To implement the above embodiments, this disclosure also proposes a non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements the fatigue-prevention driving method based on a large model as proposed in the foregoing embodiments of this disclosure.

[0096] To implement the above embodiments, this disclosure also proposes a computer program product that, when the instruction processor in the computer program product is executed, performs the fatigue-prevention driving method based on a large model as proposed in the foregoing embodiments of this disclosure.

[0097] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0098] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0099] It should be noted that in the description of this disclosure, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0100] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0101] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0102] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0103] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0104] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0105] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0106] Although embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A fatigue driving prevention method based on a large model, characterized in that, The method includes: Acquire multi-source heterogeneous data, wherein the multi-source heterogeneous data includes at least the driver's physiological data and vehicle driving data; The fatigue state recognition model based on the large model outputs the driver's fatigue state according to the multi-source heterogeneous data; The fatigue prevention strategy generation model based on the large model outputs a fatigue prevention strategy based on the driver's profile, fatigue state, and vehicle driving data. Based on the anti-fatigue strategy, control commands for the vehicle are generated, and the vehicle is controlled to perform anti-fatigue operations in the anti-fatigue strategy according to the control commands.

2. The method according to claim 1, characterized in that, The fatigue state recognition model based on the large model outputs the driver's fatigue state according to the multi-source heterogeneous data, including: The multi-source heterogeneous data is preprocessed to obtain the target multi-source heterogeneous data; Feature extraction is performed on the target multi-source heterogeneous data to obtain the target feature vector; The target feature vector is input into the fatigue state recognition model, and the driver's fatigue state is output.

3. The method according to any one of claims 1-2, characterized in that, The training process of the fatigue state recognition model includes: Acquire training data, wherein the training data includes sample multi-source heterogeneous data and corresponding fatigue levels; Based on the training data and fine-tuning strategy, the Transformer model is fine-tuned to obtain the fatigue state recognition model.

4. The method according to claim 1, characterized in that, The process of constructing the driver's profile includes: Obtain the driver's identification information; In response to the fact that the identity information does not match the stored driver profile, an information request prompt message is sent to the driver; Receive input information from the driver and construct a profile of the driver based on the input information.

5. The method according to claim 4, characterized in that, The method further includes: Obtain the vehicle driving dataset of the driver within a preset time period; The driver's vehicle driving data set within a preset time period is analyzed to obtain the driver's driving habits; The driver's profile is updated based on the driving habits described.

6. The method according to claim 1, characterized in that, The step of generating control commands for the vehicle based on the anti-fatigue strategy, and controlling the vehicle to execute anti-fatigue operations in the anti-fatigue strategy according to the control commands, includes: The fatigue prevention intervention strategy is decomposed to generate a first control command for the on-board equipment and a second control command for the vehicle control equipment. According to the first control command, the vehicle-mounted equipment is controlled to perform the anti-fatigue operation in the anti-fatigue strategy; According to the second control command, the vehicle control device is controlled to perform the anti-fatigue operation in the anti-fatigue strategy.

7. The method according to claim 6, characterized in that, The method further includes: Collect the driver's response data to the anti-fatigue operation; Based on the newly added multi-source heterogeneous data and the response data, the fatigue prevention strategy generation model is incrementally trained.

8. The method according to claim 7, characterized in that, The method further includes: The driver's fatigue status, the fatigue driving intervention strategy, and the response data are uploaded to the vehicle management platform associated with the vehicle. Receive fatigue driving intervention instructions sent by the vehicle management platform.

9. The method according to claim 7, characterized in that, The method further includes: In response to the driver's fatigue state and the response data meeting preset conditions, the vehicle's location is reported to the traffic management platform.

10. A fatigue-prevention driving device based on a large model, characterized in that, The device includes: The acquisition module is used to acquire multi-source heterogeneous data, wherein the multi-source heterogeneous data includes at least the driver's physiological data and vehicle driving data; The first output module is used to output the driver's fatigue state based on the fatigue state recognition model in the large model according to the multi-source heterogeneous data; The second output module is used to generate a model based on the anti-fatigue strategy in the large model, and output the anti-fatigue strategy according to the driver's profile, the fatigue state and the vehicle driving data. The control module is used to generate control commands for the vehicle based on the anti-fatigue strategy, and control the vehicle to perform anti-fatigue operations in the anti-fatigue strategy according to the control commands.

11. An electronic device, characterized in that, include: The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the fatigue-prevention driving method based on a large model as described in any one of claims 1-9.

12. A vehicle, characterized in that, Including the electronic device as described in claim 11.

13. A computer-readable storage medium, characterized in that, It stores a computer program, characterized in that, when the program is executed by a processor, it implements the fatigue-prevention driving method based on a large model as described in any one of claims 1-9.

14. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the fatigue-prevention driving method based on a large model according to any one of claims 1-9.

Images