In-vehicle intelligent cockpit terminal and system based on AI large model

Abstract

An embodiment of the present invention relates to the field of intelligent driving technology, and discloses an in-vehicle intelligent cockpit terminal and system based on an AI large model. The system includes: a sensing data acquisition module for real-time acquisition of various types of sensing time-series data of the vehicle; a cockpit monitoring data acquisition module for acquiring vehicle state data sent by the central control device in real time; a danger judgment module for inputting the sensing sequence data and vehicle state data into a danger prediction large model in real time to determine whether it is in a dangerous state, the danger level, and the corresponding strategy; a control module for recording and not actively intervening when the judged danger level is normal; starting the roof vortex fan and injecting fresh air through the A-pillar air duct when the judged high-risk state occurs; and controlling the release of the electronic restriction of all door locks and displaying a danger warning prompt message on the display screen of the central control when the judged emergency state occurs. The embodiment of the present invention realizes the ability to predict the current dangerous state in advance by combining the large model and perform corresponding danger prompts.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of intelligent driving, and particularly to an in-vehicle intelligent cockpit terminal and system based on an AI large model. Background Art

[0002] Currently, with the development of new energy vehicles, the automotive driving cockpit is becoming more and more intelligent. However, with increasing intelligence, the safety performance of the vehicle is also becoming more and more important.

[0003] The mechanism of battery thermal runaway is as follows: The thermal runaway of modern high-energy density batteries (such as NCM811) usually undergoes a chain reaction: SEI film decomposition (~90°C) → negative electrode reacts with electrolyte (~120°C) → separator melts (~135°C) → positive electrode releases oxygen (~200°C), and the whole process can be completed within 60 seconds. Therefore, existing safety standards require that the battery pack does not catch fire within 5 minutes after thermal diffusion occurs. The voltage needs to drop below 60V within 1 second after the high-voltage system is powered off, etc. However, the inventors of the present application have found that the current BMS usually samples temperature at a frequency of 1Hz and only monitors the module level (not single cells), resulting in the inability to capture a steep rise of 3°C / s, causing blind spots in temperature detection and lag in response. And the human-machine interaction is not intuitive, and when the power is cut off, corresponding alarm and indication operations cannot be performed. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention provide an in-vehicle intelligent cockpit terminal and system based on an AI large model, which are used to solve the problems of lag in response and inability to alarm and indicate operations in a timely manner in the prior art.

[0005] According to one aspect of the embodiments of the present invention, an in-vehicle intelligent cockpit system based on an AI large model is provided. The system is respectively connected to the central control device of the in-vehicle intelligent driving cockpit and at least two backup batteries distributed at different positions in the cockpit. The system includes:

[0006] A sensing data acquisition module, configured to acquire various types of sensing time-series data of the vehicle in real time; the sensing time-series data includes sensing data of temperature sensors, smoke detectors, sound sensors, system battery internal pressure monitors, collision acceleration sensors, vehicle body deformation sensors, and cameras inside and outside the vehicle;

[0007] A cockpit monitoring data acquisition module, configured to acquire vehicle state data sent by the central control device in real time; the vehicle state data includes vehicle control data;

[0008] A danger judgment module is used to input the sensing sequence data and the vehicle status data into the danger prediction model in real time to determine whether it is in a dangerous state, the danger level and the corresponding strategy; wherein the danger prediction model includes a battery temperature curve prediction module and a multi-task output module; the temperature curve prediction module includes a lightweight Llama3-8B superimposed LSTM timing head; wherein the vehicle data sample includes a sensing timing data sample and a vehicle status data sample, and the sensing data sample includes the battery historical temperature data, the door simulation deformation data and the acceleration history data and the corresponding danger level label within a preset time period before the danger occurs;

[0009] The control module is used for recording the sensor sequence data, the dangerous state and the dangerous level without active intervention when it is judged that the dangerous state is in a dangerous state and the dangerous level is normal; starting the roof vortex fan and injecting fresh air through the A-pillar air duct when the dangerous state is judged to be a high-risk state; and forcibly taking over the control of the central control device through the safety watchdog circuit when the dangerous state is judged to be an emergency state, controlling the release of the electronic restrictions of the door locks of the entire vehicle, and displaying the danger warning prompt information on the display screen of the central control.

[0010] In an optional manner, the system also includes a training module; the training module is used to obtain automobile data samples, and train the lightweight Llama3-8B superimposed LSTM timing head according to the training data to obtain a large risk prediction model.

[0011] In an optional manner, the training module further includes:

[0012] A sample acquisition unit, used to acquire original sample data, pre-process the original sample data, and obtain a car data sample;

[0013] A model building unit is used to obtain Llama3-8B and an LSTM time series model, preprocess the Llama3-8B to obtain a lightweight Llama3-8B, and superimpose the lightweight Llama3-8B and the LSTM time series model to obtain a battery temperature curve prediction module; and obtain a multi-task learning module through a regression task output head and a classification task output head;

[0014] The iterative training unit is used to iteratively input the automobile data samples into the lightweight Llama3-8B superimposed LSTM timing head for training, adjust the model parameters of the lightweight Llama3-8B superimposed LSTM timing head, and obtain the trained large risk prediction model.

[0015] In an optional manner, the sample acquisition unit is specifically used to:

[0016] Build a virtual test field to simulate the sensing timing data samples and vehicle state data samples under vehicle impacts in extreme working conditions, and perform label annotation to obtain the original simulated sample data;

[0017] Obtain the historical data of vehicle accidents by using differential privacy processing from the accident databases of vehicle manufacturers, the claim databases of insurance companies, and the publicly disclosed accident reports of the government, and collect the sensing timing data samples and vehicle state data samples under historical vehicle accidents. After verifying the data integrity and performing label annotation, obtain the original historical sample data;

[0018] After preprocessing the original simulated sample data and the original historical sample data, use them as the vehicle data samples.

[0019] In an alternative approach, the model construction unit is further specifically configured to:

[0020] Use a cloud large model as the teacher model to extract features from the vehicle sample data and distill the student model Llama3-8B;

[0021] Remove the attention heads irrelevant to safety in Llama3-8B and retain 20% of the cross-modal attention layers to obtain the lightweight Llama3-8B;

[0022] Add an LSTM timing head to the lightweight Llama3-8B to predict the battery temperature curve and the risk level.

[0023] In an alternative approach, the system further includes an identification module for indicating the position of the door handle through a laser device placed at a preset position.

[0024] In an alternative approach, the system further includes a voice module for providing voice prompts to the user for corresponding operations in high-risk or emergency states.

[0025] According to another aspect of the embodiments of the present invention, there is provided an in-vehicle intelligent cockpit terminal based on an AI large model, including: a plurality of sensors, a camera, one or more processors, and a communication interface;

[0026] The processor includes the in-vehicle intelligent cockpit system based on the AI large model.

[0027] According to yet another aspect of the embodiments of the present invention, there is provided a vehicle, including a vehicle body and an in-vehicle intelligent cockpit terminal based on an AI large model disposed within the vehicle body.

[0028] In the embodiments of the present invention, the system is respectively connected to the central control device of the in-vehicle intelligent driving cockpit and at least two backup batteries distributed at different positions in the cockpit. The system includes a sensing data acquisition module, a cockpit monitoring data acquisition module, a danger judgment module, and a control module. It is possible to connect multiple additional backup power supplies to the system, enabling the vehicle to combine the large model to perceive the battery temperature change in advance through the vehicle deformation data, sensing data, and battery temperature, perform danger classification and early warning in advance, and forcibly take over the control right in an emergency state through the system, supply power using the backup batteries at different positions, and combine the setting of door handle identification to enable the user to quickly open the door and escape quickly in a dangerous state.

[0029] The above description is only an overview of the technical solution of the embodiments of the present invention. In order to be able to understand the technical means of the embodiments of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features, and advantages of the embodiments of the present invention more obvious and understandable, the following specifically describes the embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings are only used to illustrate the embodiments and are not considered as a limitation of the present invention. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0031] Figure 1 shows a schematic structural diagram of an in-vehicle intelligent cockpit system based on an AI large model provided by an embodiment of the present invention;

[0032] Figure 2 shows a schematic structural diagram of an in-vehicle intelligent cockpit system based on an AI large model provided by another embodiment of the present invention;

[0033] Figure 3 shows a schematic structural diagram of a vehicle provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] Hereinafter, the exemplary embodiments of the present invention will be described in more detail with reference to the drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein.

[0035] Figure 1 shows a schematic structural diagram of an in-vehicle intelligent cockpit system based on an AI large model provided by an embodiment of the present invention. Among them, the system is set in the in-vehicle intelligent cockpit terminal, and the in-vehicle intelligent cockpit terminal is respectively connected to the central control device of the in-vehicle intelligent driving cockpit and at least two backup batteries distributed at different positions in the cockpit, so as to avoid the problem of inability to work due to the interruption of the main battery power supply in a dangerous state. As Figure 1As shown, the system 100 includes: a sensing data acquisition module 110, a cockpit monitoring data acquisition module 120, a hazard determination module 130, and a control module 140.

[0036] As Figure 2 shown, in an embodiment of the present invention, each module of the system works and performs data interaction in the following specific ways:

[0037] The sensing data acquisition module 110 is used to obtain various types of sensing time-series data of the vehicle in real time. The sensing time-series data includes sensing data of temperature sensors inside and outside the vehicle, smoke detectors, sound sensors, system battery internal pressure monitors, collision acceleration sensors, vehicle body deformation sensors, and cameras.

[0038] Among them, the temperature sensors inside and outside the vehicle are used to monitor the temperature changes inside and outside the vehicle, providing a basis for judging whether the vehicle is in an abnormal state such as high temperature. The smoke detector is used to detect whether there is smoke in the cockpit, and timely discover potential fire hazards. The sound sensor is used to capture the sound information in the cockpit, such as abnormal collision sounds and abnormal noises, to assist in judging whether the vehicle has an accident or other abnormal conditions. The system battery internal pressure monitor is used to monitor the internal pressure of the battery to prevent safety problems caused by abnormal pressure in the battery. The collision acceleration sensor is used to measure the acceleration of the vehicle during a collision to judge the severity of the collision. The vehicle body deformation sensor is used to detect whether the vehicle body has deformed to judge whether the vehicle has been impacted by an external force. The camera is used to obtain image information inside and outside the cockpit for visual monitoring, such as driver behavior monitoring and road condition monitoring. Among them, the data is temporarily stored in the DDR4 memory of the vehicle terminal through a shared memory pool (Ring Buffer structure), and the storage area is divided according to the data type. For example, it can be divided into a time-series data area and an image data area. Among them, the time-series data area includes numerical data such as temperature and acceleration, which is stored in matrix form (dimension: N×6, N is the time step); the image data area includes video streams, which are cached frame by frame in H.264 format.

[0039] In the embodiments of the present invention, various types of sensed data collected are preprocessed, such as filtering, denoising, data format conversion, etc., so that subsequent modules can efficiently perform analysis and processing. Specifically, environmental data, mechanical data, and visual data are polled at a period of 100 ms through the vehicle CAN bus, LIN bus, and dedicated sensor interfaces. Among them, the environmental data includes the temperature inside and outside the vehicle (accuracy ±0.5 °C), smoke concentration (unit: mg / m³), and environmental noise decibel value (range: 30 - 120 dB). The mechanical data includes the internal pressure of the battery (unit: kPa), the deformation of the vehicle body (millimeter-level laser ranging data), and the collision acceleration (triaxial G value). The visual data includes a camera that acquires the postures of the vehicle occupants inside the vehicle and the images of obstacles outside the vehicle at 15 fps (resolution 1280×720). In the embodiments of the present invention, voice data is also acquired to obtain the voice data inside the cockpit in real time. The original data is normalized, such as mapping the temperature to the range of 0 - 1, adding a timestamp with an accuracy of 1 ms.

[0040] Among them, for the convenience of subsequent prediction, in the embodiments of the present invention, various types of sensed data obtained in real time are concatenated with the sensed data within the previous 5S to form sensed time-series data.

[0041] In the embodiments of the present invention, the sensed data acquisition module 110 synchronizes the real-time data to the hazard determination module 130, triggers through a hardware interrupt, and sends a data ready signal to the hazard determination module 130 at a period of every 200 ms. After the sensed data acquisition module 110 completes the following operations, a hardware interrupt signal is automatically triggered: 1. Store the sensor data collected within 200 ms into a shared memory pool (Ring Buffer); 2. Complete data format verification and timestamp alignment. The data format verification can be CRC verification. Among them, through the hardware interrupt, the data itself is not directly transmitted, but a high-level pulse (lasting 1 μs) is sent through a hardware pin (such as the GPIO of an FPGA). The embodiments of the present invention consider that the automotive safety scenario requires a response delay <500 ms (ISO26262 standard), and also consider avoiding excessive CPU load caused by frequent interruptions, which reduces data integrity. Therefore, in one embodiment of the present invention, a redundant time is reserved for multiple predictions at an interval of 200 ms. Among them, if a certain interruption is lost, such as being affected by electromagnetic interference, etc., the hazard determination module 130 will actively pull the sensed time-series data when it does not receive a signal within 400 ms.

[0042] The cockpit monitoring data acquisition module 120 is used to acquire the vehicle status data sent by the central control device in real time; the vehicle status data includes vehicle control data. Obtaining the real-time status data of the vehicle from the in-vehicle central control device, including but not limited to vehicle control data, such as vehicle speed, steering angle, braking status, etc. Ensure stable and real-time data transmission with the central control device so that the danger judgment module can obtain complete and accurate vehicle status information. Specifically, monitor the CAN bus messages of the central control device (ID range 0x500 - 0x5FF), extract the real-time status data, including vehicle speed, battery, door lock status, brake pedal opening, etc. Bind the parsed data with the GPS positioning information to generate a JSON format message, where the GPS positioning information is longitude and latitude, updated at 10Hz. After obtaining the vehicle status data, push it to the input queue of the danger judgment module 130 through the ZeroMQ protocol.

[0043] The danger judgment module 130 is used to input the sensing sequence data and the vehicle status data into the danger prediction large model in real time to determine whether it is in a dangerous state, the danger level, and the corresponding strategy. Among them, the danger prediction large model includes a battery temperature curve prediction module and a multi-task output module; the temperature curve prediction module includes a lightweight Llama3 - 8B stacked with an LSTM time series head; among them, the vehicle data samples include sensing time series data samples and vehicle status data samples, and the sensing data samples include the battery historical temperature data, door simulated deformation data, acceleration historical data, and corresponding danger level labels within a preset time period before a danger occurs.

[0044] Specifically, the danger judgment module 130 performs data preprocessing on various types of sensing data and vehicle status data received, and time series alignment: uses the DTW algorithm for dynamic time warping of the timestamps of the sensing data and the central control data to eliminate clock deviation. Then, calculate the first-order difference (ΔT / Δt) for the battery temperature data to capture the mutation trend; perform FFT transformation on the acceleration data to extract the side collision characteristic frequency band energy, and the side collision characteristic frequency band energy is the energy in the 5 - 15Hz frequency band. Use the YOLOv5s model to detect in real time whether the occupants in the vehicle are wearing seat belts, etc. When not wearing a seat belt, issue a seat belt warning.

[0045] The risk prediction large model of the risk judgment module 130 includes a battery temperature curve prediction module and a multi-task output module. The temperature curve prediction module includes a lightweight Llama3-8B stacked with an LSTM time series head. Among them, based on the Transformer architecture of Llama3-8B, the first 6 layers are retained for general feature extraction. The LSTM time series head is connected with 2 layers of bidirectional LSTM after the output of the Transformer architecture to capture long-term dependencies. Specifically, before the input to the temperature curve prediction module, the input data is also preprocessed. According to a certain time window, the data is segmented with a preset step size, which can be a 500ms window and 50 time steps (10ms interval). The data dimension within each time step includes 8-dimensional data such as battery temperature, voltage, and deformation sensors. The 8-dimensional data is input into the lightweight Llama3-8B (Transformer encoding), with the input shape (batch_size, 50, 8), linearly projected to 768 dimensions, obtaining (batch_size, 50, 768), and spatio-temporal features are extracted through 6 layers of Transformer. After obtaining the spatio-temporal features, they are input into the LSTM time series head for time series prediction. The output of the last time step of the Transformer (batch_size, 768) is taken and input into the LSTM to predict the temperature curve for the next 50 time steps, thus realizing the temperature curve prediction.

[0046] Among them, in the embodiment of the present invention, the temperature curve is obtained based on the Beernardi electrochemistry-thermal coupling equation: : where ρ represents density; C p represents specific heat capacity; represents the temperature change rate; k represents the thermal conductivity of the battery material; q gen represents the heat generated per unit volume of the battery per unit time. Among them, . I represents current, ; represents the open-circuit voltage temperature coefficient, and T represents absolute temperature.

[0047] In the embodiment of the present invention, the multi-task output module includes a regression task and a classification task. The temperature prediction curve, the aforementioned sensing sequence data, and the vehicle state data are input into the multi-task output module. In order to be able to quickly predict the occurrence of a runaway accident, 50 time steps are set to 5 seconds. For example, it is predicted that thermal runaway may occur 5 seconds later according to the battery temperature curve. The prediction target of the regression task is to predict the temperature values within the next 50 time steps. When the predicted temperature exceeds 80°C, a high-temperature warning is triggered; the classification task outputs the probabilities of three classifications, including normal state, high-risk state, and emergency state. Among them, the threshold is set in the following way. High-risk is: P≥0.7 and the temperature prediction ≥70°C; Emergency is: P≥0.9 or the detected door deformation >50mm.

[0048] Among them, in the embodiment of the present invention, the receiving end of the risk judgment module 130 subscribes to the Ring Buffer interrupt signal of the sensing module and the central control data queue; the output end writes the decision result in the form of a structure to the control module through shared memory.

[0049] Among them, the system further includes a training module. The training module is used to obtain automotive data samples and train the lightweight Llama3-8B stacked with LSTM time series headers based on the training data to obtain a risk prediction large model.

[0050] In the embodiment of the present invention, the training module further includes:

[0051] A sample acquisition unit, which is used to obtain original sample data and preprocess the original sample data to obtain automotive data samples. Among them, the sample acquisition unit is specifically used for: building a virtual test field, simulating the sensing time series data samples and automotive state data samples under vehicle impacts in extreme working conditions, and performing label annotation to obtain original simulated sample data; obtaining automotive accident historical data from the accident databases of automobile enterprises, the claim databases of insurance companies, and the publicly disclosed accident reports of the government by using differential privacy processing methods, and collecting the sensing time series data samples and automotive state data samples under historical automotive accidents, verifying the data integrity, and performing label annotation to obtain original historical sample data; preprocessing the original simulated sample data and the original historical sample data and using them as the automotive data samples. In the embodiment of the present invention, encrypted data uploaded by the vehicle is regularly obtained through the 4G module to the cloud training platform. Among them, during data annotation, for accidents, the data in the 5 minutes before the accident is automatically marked as "urgent", and abnormal segments are manually reviewed.

[0052] The model construction unit is used to obtain the Llama3-8B and LSTM time series models, preprocess the Llama3-8B to obtain a lightweight Llama3-8B, stack the lightweight Llama3-8B and the LSTM time series model to obtain a battery temperature curve prediction module; and obtain a multi-task learning module through a regression task output head and a classification task output head. Among them, the model construction unit is also specifically used for: using the cloud large model Llama3-70B as a teacher model to extract features from the automotive sample data including temperature, voltage, deformation, etc., and distilling the student model Llama3-8B; performing structural pruning, removing the safety-irrelevant attention heads in the Llama3-8B, and retaining 20% of the cross-modal attention layers to focus on time series pattern recognition, obtaining a lightweight Llama3-8B; adding an lstm time series head on the basis of the lightweight Llama3-8B to obtain a battery temperature curve prediction module for predicting the battery temperature curve. Specifically, for the battery temperature curve prediction module, the base model is based on the Transformer architecture of Llama3-8B, and the first 6 layers are retained for general feature extraction. After the Transformer output, 2 layers of bidirectional LSTM are connected as the LSTM time series head to capture long-term dependencies for predicting the battery temperature curve.

[0053] The iterative training unit is used to iteratively input automotive data samples into the lightweight Llama3-8B stacked with the LSTM time series head for training, adjust the model parameters of the lightweight Llama3-8B stacked with the LSTM time series head, and obtain a trained large model for danger prediction. Among them, the Huber Loss is used as the loss function to enhance the robustness to outliers.

[0054] The control module 140 is used to record the sensing sequence data, the danger state and the danger level without active intervention when it is judged to be in a danger state and the danger level is normal; start the roof eddy current fan and inject fresh air through the A-pillar air duct when it is judged that the danger state is a high-risk state; forcibly take over the control right of the central control device through the safety watchdog circuit, control the release of the electronic restriction of all door locks, and display a danger warning prompt message on the display screen of the central control when it is judged that the danger state is an emergency state.

[0055] Specifically, in the normal state, danger_level = 0, the original data and decision results are encrypted and written into the black box (eMMC storage chip); the connection with the central control is maintained through a heartbeat packet (1Hz).

[0056] Under the high-risk state where danger_level = 1, eddy current fan control and A-pillar air duct control are performed. Specifically, a PWM signal (with an adjustable duty cycle of 0 - 100%) is sent to the fan drive circuit to control the eddy current fan. The formula for the target wind speed is: duty_cycle = min(100, (T_current - 60) * 2). The micro air pump (with a flow rate of 5L / s) is activated, and the PID algorithm is used to maintain the CO2 concentration in the vehicle < 1000ppm, thereby performing A-pillar air duct control.

[0057] In the emergency state (danger_level = 2), a Hard Reset signal (a 50ms low-level pulse) is sent to the vehicle center console through the safety watchdog circuit to forcibly take over control; the door unlock instruction (message ID 0x7FF, data field 0x0F) is broadcast through the HS-CAN bus. Multimodal warnings are also carried out. For example, the screen of the vehicle center console displays a red flashing interface at a frequency of 2Hz, and AR guidance for the escape route is provided. By activating the seat vibration motor, such as 3 short vibrations + 1 long vibration, the user is prompted of the emergency state. In the embodiment of the present invention, after receiving the decision structure of the danger judgment module 130, the control module 140 parses the action_code within 5ms, and ensures the priority execution of the emergency instruction (response delay < 10ms) through the hardware priority arbiter (implemented by FPGA).

[0058] In the embodiment of the present invention, the system further includes an identification module for indicating the position of the door handle through a laser device placed at a preset position. Among them, the trigger condition of the identification module is that danger_level ≥ 1 and the ambient light sensor detects that the light intensity < 50lux. The laser projection is through a 532nm green laser in the A-pillar, generating a rectangular light spot with adjustable brightness at the position of the door handle, with a size of 30cm × 10cm.

[0059] In the embodiment of the present invention, the system further includes a voice module for providing voice prompts to the user for corresponding operations in the high-risk state or the emergency state. Among them, the multi-language TTS engine can automatically select the language according to the vehicle registration information associated with the VIN code; through sound field focusing, the seat headrest speakers are used for directional broadcasting (beamforming technology) to reduce external interference.

[0060] In the embodiment of the present invention, the system is respectively connected to the central control device of the in-vehicle intelligent driving cockpit and at least two backup batteries distributed at different positions in the cockpit. The system includes a sensing data acquisition module, a cockpit monitoring data acquisition module, a danger judgment module, and a control module. By setting multiple additional backup power sources to be connected to the system, the vehicle can combine the large model to sense the battery temperature change in advance through vehicle deformation data, sensing data, and battery temperature, conduct danger classification and early warning in advance, and forcibly take over the control right in an emergency state through the system, supply power using backup batteries at different positions, and combine the setting of door handle identifiers to enable the user to be prompted to quickly open the door and escape quickly in a dangerous state.

[0061] Figure 3 FIG. shows a schematic structural diagram of an in-vehicle intelligent cockpit terminal based on an AI large model provided by an embodiment of the present invention. As Figure 3 shown, the device 300 includes: a sensor 305, a camera 307, one or more processors 302, and a communication interface 304;

[0062] Multiple sensors 305, camera 307, one or more processors 302, and communication interface 304 interact through a communication bus 308;

[0063] It can be understood that the in-vehicle intelligent cockpit system based on the AI large model can be composed of a program 310, which is stored in a memory 306. The processor calls the program 310 in the memory 306 to execute the functions of the aforementioned in-vehicle intelligent cockpit system based on the AI large model. The processor executes the functions of each module in the above embodiment.

[0064] In the embodiment of the present invention, the system is respectively connected to the central control device of the in-vehicle intelligent driving cockpit and at least two backup batteries distributed at different positions in the cockpit. The system includes a sensing data acquisition module, a cockpit monitoring data acquisition module, a danger judgment module, and a control module. By setting multiple additional backup power sources to be connected to the system, the vehicle can combine the large model to sense the battery temperature change in advance through vehicle deformation data, sensing data, and battery temperature, conduct danger classification and early warning in advance, and forcibly take over the control right in an emergency state through the system, supply power using backup batteries at different positions, and combine the setting of door handle identifiers to enable the user to be prompted to quickly open the door and escape quickly in a dangerous state.

[0065] An embodiment of the present invention further provides an automobile, including an automobile body and the in-vehicle intelligent cockpit terminal based on the AI large model provided in the automobile body.

[0066] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. A variety of general-purpose systems can also be used in conjunction with the teachings based herein. The structure required to construct such systems will be apparent from the above description. In addition, embodiments of the present invention are not directed to any particular programming language. It should be understood that the content of the present invention described herein can be implemented using various programming languages, and the description of a particular language above is for the purpose of disclosing the best mode of the present invention.

[0067] In the specification provided herein, a number of specific details are set forth. However, it is understood that embodiments of the present invention may be practiced without these specific details. In some instances, well-known systems, structures, and techniques have not been shown in detail so as not to obscure the understanding of the present specification.

[0068] Similarly, it should be understood that, in order to streamline the present invention and assist in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present invention, the various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed systems should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.

[0069] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and disposed in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all the features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all the processes or units of any system or device so disclosed. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) can be replaced by an alternative feature that provides the same, equivalent, or similar purpose.

[0070] It should be noted that the above embodiments are illustrative of the present invention and not restrictive, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim listing several devices, several of these devices may be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. An in-vehicle intelligent cockpit system based on an AI large model, characterized in that, The system is arranged in a vehicle-mounted intelligent cockpit terminal, and the vehicle-mounted intelligent cockpit terminal is respectively connected to a central control device of a vehicle-mounted intelligent driving cockpit and at least two backup batteries distributed at different positions in the cockpit, and the system includes: Sensor data acquisition module, used to obtain various sensor time series data of the car in real time; The cockpit monitoring data acquisition module is used to obtain the vehicle status data sent by the central control device in real time; The danger judgment module is used to input the sensor time series data and the vehicle status data into the danger prediction model in real time to determine whether it is in a dangerous state, the danger level and the corresponding strategy; wherein the danger prediction model includes a battery temperature curve prediction module and a multi-task output module; the temperature curve prediction module includes a lightweight Llama3-8B superimposed LSTM time series head; wherein the vehicle data sample includes a sensor time series data sample and a vehicle status data sample, and the sensor time series data sample includes the battery historical temperature data, the door simulation deformation data and the acceleration history data and the corresponding danger level label within a preset time period before the danger occurs; the system also includes a training module; the training module is used to obtain vehicle data samples, and train the lightweight Llama-3-8B superimposed LSTM time series head according to the training data to obtain the danger level label. The training module further includes: a sample acquisition unit, which is used to acquire original sample data, pre-process the original sample data, and obtain automobile data samples; a model construction unit, which is used to acquire Llama3-8B and LSTM time series model, pre-process the Llama3-8B to obtain lightweight Llama3-8B, and superimpose the lightweight Llama3-8B with the LSTM time series model to obtain a battery temperature curve prediction module; a multi-task learning module is obtained through a regression task output head and a classification task output head; an iterative training unit, which is used to iteratively input automobile data samples into the lightweight Llama3-8B superimposed LSTM time series head for training, adjust the model parameters of the lightweight Llama3-8B superimposed LSTM time series head, and obtain a trained danger prediction model; The control module is used for recording the sensor timing data, the dangerous state and the dangerous level, and not actively intervening when it is judged that the dangerous state is in a dangerous state and the dangerous level is normal; starting the roof vortex fan and injecting fresh air through the A-pillar air duct when the dangerous state is judged to be a high-risk state; and forcibly taking over the control of the central control device through the safety watchdog circuit when the dangerous state is judged to be an emergency state, controlling the release of the electronic restrictions of the door locks of the entire vehicle, and displaying danger warning prompt information on the display screen of the central control.

2. The system according to claim 1, characterized in that The sample acquisition unit is specifically used for: Build a virtual test field to simulate the sensor time series data samples and vehicle status data samples under vehicle collision under extreme working conditions, and label them to obtain the original simulation sample data; Obtain historical automotive accident data using differential privacy processing from automotive enterprise accident databases, insurance company claims databases, and government public accident reports, and collect sensor time-series data samples and vehicle state data samples under historical automotive accidents. After verifying data integrity and performing label annotation, obtain the original historical sample data; After preprocessing the original simulation sample data and the original historical sample data, use them as the automotive data samples.

3. The system according to any one of claims 1-2, characterized in that, The model construction unit is further specifically configured to: Use a cloud large model as the teacher model to extract features from the automotive data samples and distill the student model Llama3-8B; Remove the attention heads unrelated to safety in Llama3-8B and retain 20% of the cross-modal attention layers to obtain the lightweight Llama3-8B; Add an LSTM time-series head on the basis of the lightweight Llama3-8B to predict the battery temperature curve.

4. The system according to any one of claims 1-2, characterized in that, The system further includes an identification module for indicating the position of the door handle through a laser device placed at a preset position.

5. The system according to any one of claims 1-2, characterized in that, The system further includes a voice module for providing voice prompts to the user for corresponding operations in high-risk or emergency states.

6. An in-vehicle intelligent cockpit terminal based on an AI large model, characterized in that, Comprising: Multiple sensors, cameras, one or more processors, and a communication interface; The processor includes the operations corresponding to the in-vehicle intelligent cockpit system based on the AI large model according to any one of claims 1-5.

7. An automobile, characterized in that, Comprising an automotive body and the in-vehicle intelligent cockpit terminal based on the AI large model according to claim 6 provided in the automotive body.

Images