Vehicle-mounted intelligent cabin terminal and system based on AI large model

Abstract

The embodiment of the invention relates to the technical field of intelligent driving, and discloses a vehicle-mounted intelligent cabin terminal and system based on an AI large model, and the system comprises a sensing data obtaining module which is used for obtaining various kinds of sensing time sequence data of an automobile in real time; the cockpit monitoring data acquisition module is used for acquiring automobile state data sent by the central control device in real time; the danger judgment module is used for inputting sensing sequence data and automobile state data into the danger prediction large model in real time, and determining whether the automobile is in a dangerous state or not, a danger level and a corresponding strategy; the control module is used for recording and not actively intervening when the danger level is judged to be normal; when the high-risk state is judged, a car roof vortex fan is started, and fresh air is injected through an A-column air duct; and when the emergency state is judged, the electronic limitation of the whole vehicle door lock is controlled to be relieved, and danger warning prompt information is displayed to a display screen of a central controller. According to the embodiment of the invention, the current dangerous state can be predicted in advance by combining a large model, and corresponding danger prompt is carried out.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of intelligent driving technology, and specifically to a vehicle-mounted intelligent cockpit terminal and system based on an AI big model. Background Art

[0002] At present, with the development of new energy vehicles, car cockpits are becoming more and more intelligent. However, as they become more and more intelligent, the safety performance of cars is becoming more and more important.

[0003] The mechanism of thermal runaway of batteries is as follows: The thermal runaway of modern high-energy density batteries (such as NCM811) usually undergoes a chain reaction: SEI film decomposition (~90°C) → reaction of the negative electrode with the electrolyte (~120°C) → meltdown of the diaphragm (~135°C) → oxygen release from the positive electrode (~200°C), and the entire process can be completed within 60 seconds. Therefore, existing safety standards require that the battery pack does not catch fire within 5 minutes after thermal diffusion occurs. The voltage needs to drop below 60V within 1 second after the high-voltage system is powered off. However, the inventors of this application found that the current BMS usually samples the temperature at a frequency of 1Hz and only monitors the module level (not a single cell), resulting in the inability to capture a steep rise of 3°C / s, resulting in a blind spot in temperature detection and a lag in response. In addition, the human-computer interaction is not intuitive, and when the power is off, the corresponding alarm and indication operations cannot be performed. Summary of the invention

[0004] In view of the above problems, an embodiment of the present invention provides an on-board intelligent cockpit terminal and system based on an AI big model, which is used to solve the problems in the prior art of delayed response and inability to promptly alarm and indicate operations.

[0005] According to one aspect of an embodiment of the present invention, there is provided an in-vehicle intelligent cockpit system based on an AI large model, the system being connected to a central control device of an in-vehicle intelligent driving cockpit and at least two backup batteries distributed at different positions in the cockpit, respectively, the system comprising: The sensor data acquisition module is used to acquire various sensor time series data of the car in real time; the sensor time series data includes sensor data of temperature sensors inside and outside the car, smoke detectors, sound sensors, system battery internal pressure monitors, collision acceleration sensors, body deformation sensors, and cameras; The cockpit monitoring data acquisition module is used to acquire the vehicle status data sent by the central control device in real time; the vehicle status data includes vehicle control data; A danger judgment module is used to input the sensing sequence data and the vehicle status data into the danger prediction model in real time to determine whether it is in a dangerous state, the danger level and the corresponding strategy; wherein the danger prediction model includes a battery temperature curve prediction module and a multi-task output module; the temperature curve prediction module includes a lightweight Llama3-8B superimposed LSTM timing head; wherein the vehicle data sample includes a sensing timing data sample and a vehicle status data sample, and the sensing data sample includes the battery historical temperature data, the door simulation deformation data and the acceleration history data and the corresponding danger level label within a preset time period before the danger occurs; The control module is used for recording the sensor sequence data, the dangerous state and the dangerous level without active intervention when it is judged that the dangerous state is in a dangerous state and the dangerous level is normal; starting the roof vortex fan and injecting fresh air through the A-pillar air duct when the dangerous state is judged to be a high-risk state; and forcibly taking over the control of the central control device through the safety watchdog circuit when the dangerous state is judged to be an emergency state, controlling the release of the electronic restrictions of the door locks of the entire vehicle, and displaying the danger warning prompt information on the display screen of the central control.

[0006] In an optional manner, the system also includes a training module; the training module is used to obtain automobile data samples, and train the lightweight Llama3-8B superimposed LSTM timing head according to the training data to obtain a large risk prediction model.

[0007] In an optional manner, the training module further includes: A sample acquisition unit, used to acquire original sample data, pre-process the original sample data, and obtain a car data sample; A model building unit is used to obtain Llama3-8B and an LSTM time series model, preprocess the Llama3-8B to obtain a lightweight Llama3-8B, and superimpose the lightweight Llama3-8B and the LSTM time series model to obtain a battery temperature curve prediction module; and obtain a multi-task learning module through a regression task output head and a classification task output head; The iterative training unit is used to iteratively input the automobile data samples into the lightweight Llama3-8B superimposed LSTM timing head for training, adjust the model parameters of the lightweight Llama3-8B superimposed LSTM timing head, and obtain the trained large risk prediction model.

[0008] In an optional manner, the sample acquisition unit is specifically used to: Build a virtual test field to simulate the sensor time series data samples and vehicle status data samples under vehicle collision under extreme working conditions, and label them to obtain the original simulation sample data; The historical data of automobile accidents is obtained from the accident database of automobile companies, the claims database of insurance companies and the public accident reports of the government using differential privacy processing. The sensor time series data samples and automobile status data samples under historical automobile accidents are collected, and the data integrity is verified. After labeling, the original historical sample data is obtained; The original simulation sample data and the original historical sample data are preprocessed to serve as the automobile data samples.

[0009] In an optional manner, the model building unit is further specifically used for: Use the cloud-based large model as the teacher model, perform feature extraction on the car sample data, and distill the student model Llama3-8B; Remove the attention heads not related to security in the Llama3-8B, retain 20% of the cross-modal attention layers, and obtain the lightweight Llama3-8B; Based on the lightweight Llama3-8B, an LSTM timing header is added to predict the battery temperature curve and risk level.

[0010] In an optional manner, the system further includes an identification module for indicating the position of the door handle via a laser device placed at a preset position.

[0011] In an optional manner, the system further includes a voice module for providing voice prompts to the user for corresponding operations in a high-risk or emergency state.

[0012] According to another aspect of an embodiment of the present invention, there is provided an in-vehicle intelligent cockpit terminal based on an AI big model, comprising: a plurality of sensors, a camera, one or more processors and a communication interface; The processor includes the in-vehicle intelligent cockpit system based on the AI ​​big model.

[0013] According to another aspect of an embodiment of the present invention, a car is provided, comprising a car body, and an on-board intelligent cockpit terminal based on an AI big model arranged in the car body.

[0014] The embodiment of the present invention is connected to the central control device of the vehicle-mounted intelligent driving cockpit and at least two backup batteries distributed at different positions in the cockpit through the system. The system includes a sensor data acquisition module, a cockpit monitoring data acquisition module, a danger judgment module, and a control module. It can be connected to the system by setting up additional multiple backup power supplies, so that the vehicle can sense the battery temperature change in advance through the vehicle deformation data and sensor data and battery temperature in combination with the large model, and make a danger classification warning in advance. The system can forcibly take over control in an emergency state, use backup batteries in different positions to supply power, and combine with the setting of door handle signs to prompt users to quickly open the door and escape quickly in a dangerous state.

[0015] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings: Figure 1 A schematic diagram of the structure of an on-vehicle intelligent cockpit system based on an AI big model provided by an embodiment of the present invention is shown; Figure 2 A schematic diagram of the structure of an on-vehicle intelligent cockpit system based on an AI big model provided by another embodiment of the present invention is shown; Figure 3 A structural schematic diagram of a car provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0017] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0018] Figure 1 The schematic diagram of the structure of the vehicle-mounted intelligent cockpit system based on the AI ​​big model provided by an embodiment of the present invention is shown. The system is set in the vehicle-mounted intelligent cockpit terminal, which is respectively connected to the central control device of the vehicle-mounted intelligent driving cockpit and at least two backup batteries distributed in different positions in the cockpit, so that in dangerous conditions, the problem of failure to work due to interruption of the main battery power supply can be avoided. Figure 1As shown, the system 100 includes: a sensor data acquisition module 110 , a cockpit monitoring data acquisition module 120 , a danger judgment module 130 , and a control module 140 .

[0019] like Figure 2 As shown, in one embodiment of the present invention, each module of the system works and performs data interaction in the following manner: The sensor data acquisition module 110 is used to acquire various sensor time series data of the vehicle in real time, including sensor data of temperature sensors inside and outside the vehicle, smoke detectors, sound sensors, system battery internal pressure monitors, collision acceleration sensors, body deformation sensors, and cameras.

[0020] Among them, the temperature sensors inside and outside the vehicle are used to monitor the temperature changes inside and outside the vehicle, providing a basis for judging whether the vehicle is in an abnormal state such as high temperature. The smoke detector is used to detect whether there is smoke in the cockpit and timely discover fire hazards. The sound sensor is used to capture the sound information in the cockpit, such as abnormal collision sounds, abnormal noises, etc., to assist in judging whether the vehicle has an accident or other abnormal conditions. The system battery internal pressure monitor is used to monitor the internal pressure of the battery to prevent safety problems caused by abnormal pressure. The collision acceleration sensor is used to measure the acceleration of the vehicle during a collision to judge the severity of the collision. The body deformation sensor is used to detect whether the body is deformed and judge whether the vehicle is impacted by external forces. The camera is used to obtain image information inside and outside the cockpit for visual monitoring, such as driver behavior monitoring, road condition monitoring, etc. Among them, the data is temporarily stored in the DDR4 memory of the vehicle terminal through a shared memory pool (Ring Buffer structure), and the storage area is divided according to the data type. For example, it can be divided into a time series data area and an image data area. Among them, the time series data area includes numerical data such as temperature and acceleration, which are stored in matrix form (dimension: N×6, N is the time step); the image data area includes video streams, which are cached in frames in H.264 format.

[0021] In an embodiment of the present invention, the various types of sensor data collected are preprocessed, such as filtering, denoising, data format conversion, etc., so that subsequent modules can efficiently analyze and process. Specifically, environmental data, mechanical data, and visual data are polled with a period of 100ms through the vehicle CAN bus, LIN bus, and dedicated sensor interface. Among them, environmental data include vehicle internal and external temperature (accuracy ±0.5℃), smoke concentration (unit mg / m³), and ambient noise decibel value (range 30-120dB). Mechanical data include battery internal pressure (unit kPa), vehicle body shape (millimeter-level laser ranging data), and collision acceleration (three-axis G value). Visual data includes cameras that collect the posture of passengers in the car and the image of obstacles outside the car at 15fps (resolution 1280×720). In an embodiment of the present invention, voice data is also obtained for real-time acquisition of voice data in the cockpit. The original data is normalized, such as mapping the temperature to the 0-1 interval, adding a timestamp, and an accuracy of 1ms.

[0022] In order to facilitate subsequent predictions, the embodiment of the present invention splices various sensor data acquired in real time with the sensor data within the previous 5S to form sensor time series data.

[0023] In an embodiment of the present invention, the sensor data acquisition module 110 synchronizes the real-time data to the danger judgment module 130, and sends a data ready signal to the danger judgment module 130 every 200ms through a hardware interrupt trigger. When the sensor data acquisition module 110 completes the following operations, the hardware interrupt signal is automatically triggered: 1. The sensor data collected within 200ms is stored in the shared memory pool (Ring Buffer); 2. The data format check and timestamp alignment are completed. The data format check can be a CRC check. Among them, through the hardware interrupt, the data itself is not directly transmitted, but a high-level pulse (lasting 1μs) is sent through the hardware pin (such as the GPIO of the FPGA). The embodiment of the present invention takes into account the response delay required by the automotive safety scenario to be less than 500ms (ISO26262 standard), and takes into account avoiding frequent interrupts that cause excessive CPU load, which reduces data integrity. Therefore, in one embodiment of the present invention, redundant time is reserved for multiple predictions at intervals of 200ms. Among them, if a certain interrupt is lost, such as electromagnetic interference, the danger judgment module 130 will actively pull the sensor timing data when no signal is received in 400ms.

[0024] The cockpit monitoring data acquisition module 120 is used to obtain the vehicle status data sent by the central control device in real time; the vehicle status data includes vehicle control data. The real-time status data of the vehicle is obtained from the vehicle-mounted central control device, including but not limited to vehicle control data, such as vehicle speed, steering angle, braking status, etc. Ensure that the data transmission between the central control device is stable and real-time so that the danger judgment module can obtain complete and accurate vehicle status information. Specifically, monitor the CAN bus message (ID range 0x500-0x5FF) of the central control device to extract real-time status data, including vehicle speed, battery, door lock status, brake pedal opening, etc. Bind the parsed data with the GPS positioning information to generate a JSON format message, where the GPS positioning information is longitude and latitude, updated at 10Hz. After obtaining the vehicle status data, push it to the input queue of the danger judgment module 130 through the ZeroMQ protocol.

[0025] The danger judgment module 130 is used to input the sensor sequence data and the vehicle status data into the danger prediction model in real time to determine whether it is in a dangerous state, the danger level and the corresponding strategy. The danger prediction model includes a battery temperature curve prediction module and a multi-task output module; the temperature curve prediction module includes a lightweight Llama3-8B superimposed LSTM timing head; the vehicle data sample includes a sensor timing data sample and a vehicle status data sample, and the sensor data sample includes the battery historical temperature data, door simulation deformation data and acceleration history data and the corresponding danger level label within a preset time period before the danger occurs.

[0026] Specifically, the danger judgment module 130 performs data preprocessing and time alignment on the various types of sensor data and vehicle status data received: the DTW algorithm is used to dynamically time warp the timestamps of the sensor data and the central control data to eliminate clock deviation. After that, the first-order difference (ΔT / Δt) of the battery temperature data is calculated to capture the mutation trend; the acceleration data is transformed by FFT to extract the energy of the side collision characteristic frequency band, which is the energy of the 5-15Hz frequency band. The YOLOv5s model is used to detect in real time whether the occupants in the in-car picture are wearing seat belts, etc. If the seat belts are not fastened, a seat belt warning is issued.

[0027] The large danger prediction model of the danger judgment module 130 includes a battery temperature curve prediction module and a multi-task output module. The temperature curve prediction module includes a lightweight Llama3-8B superimposed LSTM timing head. Among them, based on the Transformer architecture of Llama3-8B, the first 6 layers are retained for general feature extraction. The LSTM timing head is connected to a 2-layer bidirectional LSTM after the output of the Transformer architecture to capture long-term dependencies. Specifically, before entering the temperature curve prediction module, the input data is also preprocessed, and the data is segmented according to a certain time window and a preset step size, which can be a 500ms window, 50 time steps (10ms interval), and the data dimensions in each time step include 8-dimensional data such as battery temperature, voltage, and deformation sensor. The 8-dimensional data is input into the lightweight Llama3-8B (Transformer encoding), the input shape is (batch_size, 50, 8), linearly projected to 768 dimensions, and (batch_size, 50, 768) is obtained, and the spatiotemporal features are extracted through a 6-layer Transformer. After obtaining the spatiotemporal features, they are input into the LSTM time series head for time series prediction. The Transformer output (batch_size, 768) of the last time step is taken and input into the LSTM to predict the temperature curve for the next 50 time steps, thereby realizing temperature curve prediction.

[0028] The temperature curve in the embodiment of the present invention is obtained based on the Bernardi electrochemical-thermal coupling equation: : where ρ represents density; C p represents specific heat capacity; represents the temperature change rate; k represents the thermal conductivity of the battery material; q gen It represents the heat generated by the battery per unit volume per unit time. I represents the current, ; represents the open circuit voltage temperature coefficient, and T represents the absolute temperature.

[0029] In an embodiment of the present invention, the multi-task output module includes a regression task and a classification task, and the temperature prediction curve and the aforementioned sensor sequence data and the vehicle status data are input into the multi-task output module. In order to quickly predict the occurrence of a runaway accident, 50 time steps are set to 5 seconds. For example, according to the battery temperature curve, thermal runaway may occur in 5 seconds. The prediction target of the regression task is to predict the temperature value within the next 50 time steps, and trigger a high temperature warning when the predicted temperature exceeds 80°C; the classification task outputs three classification probabilities, including normal state, high-risk state and emergency state. Among them, the threshold is set in the following way, high risk is: P≥0.7 and the temperature prediction is ≥70°C; emergency is: P≥0.9 or the door deformation is detected to be greater than 50mm.

[0030] Among them, in the embodiment of the present invention, the receiving end of the danger judgment module 130 subscribes to the Ring Buffer interrupt signal and the central control data queue of the sensor module; the output end writes the decision result in the form of a structure to the control module through the shared memory.

[0031] The system also includes a training module, which is used to obtain automobile data samples and train the lightweight Llama3-8B superimposed LSTM timing head according to the training data to obtain a large risk prediction model.

[0032] In the embodiment of the present invention, the training module further includes: The sample acquisition unit is used to obtain the original sample data, pre-process the original sample data, and obtain the automobile data sample. Among them, the sample acquisition unit is specifically used to: build a virtual test field, simulate the sensor time series data samples and automobile status data samples under vehicle collision under extreme working conditions, and label them to obtain the original simulation sample data; use differential privacy processing to obtain automobile accident history data from the automobile company accident database, insurance company claims database and government public accident report, and collect sensor time series data samples and automobile status data samples under historical automobile accidents, verify data integrity, label them, and obtain the original historical sample data; pre-process the original simulation sample data and the original historical sample data as the automobile data sample. In the embodiment of the present invention, the encrypted data uploaded by the vehicle is also regularly obtained through the 4G module to the cloud training platform. Among them, when the data is labeled, for an accident, the data 5 minutes before the accident is automatically marked as "urgent", and the abnormal fragments are manually reviewed.

[0033] The model construction unit is used to obtain Llama3-8B and LSTM timing model, obtain lightweight Llama3-8B after preprocessing the Llama3-8B, superimpose the lightweight Llama3-8B and LSTM timing model to obtain a battery temperature curve prediction module; obtain a multi-task learning module through the regression task output head and the classification task output head. Among them, the model construction unit is also specifically used to: use the cloud-based large model Llama3-70B as a teacher model, extract features of the automobile sample data including temperature, voltage, deformation, etc., and distill out the student model Llama3-8B; perform structural pruning, remove the attention heads that are not related to safety in the Llama3-8B, retain 20% of the cross-modal attention layer to focus on timing pattern recognition, and obtain a lightweight Llama3-8B; add the lstm timing head on the basis of the lightweight Llama3-8B to obtain a battery temperature curve prediction module for predicting the battery temperature curve. Specifically, for the battery temperature curve prediction module, the base model is based on the Transformer architecture of Llama3-8B, retaining the first 6 layers for general feature extraction. A 2-layer bidirectional LSTM is connected after the Transformer output as the LSTM timing head to capture long-term dependencies for predicting the battery temperature curve.

[0034] The iterative training unit is used to iteratively input the automobile data samples into the lightweight Llama3-8B superimposed LSTM time-series head for training, adjust the model parameters of the lightweight Llama3-8B superimposed LSTM time-series head, and obtain the trained hazard prediction model. Among them, Huber Loss is used as the loss function to enhance the robustness to outliers.

[0035] The control module 140 is used to record the sensor sequence data, the dangerous state and the danger level, and not actively intervene when it is judged to be a dangerous state and the danger level is normal; when it is judged that the dangerous state is a high-risk state, start the roof vortex fan and inject fresh air through the A-pillar air duct; when it is judged that the dangerous state is an emergency state, forcibly take over the control of the central control device through the safety watchdog circuit, control the release of the electronic restrictions of the door locks of the entire vehicle, and display danger warning prompt information on the display screen of the central control.

[0036] Specifically, under normal conditions, danger_level = 0, the original data and decision results are encrypted and written into the black box (eMMC storage chip); the connection with the central control is maintained through the heartbeat packet (1Hz).

[0037] In the high-risk state danger_level=1, the eddy current fan and A-pillar air duct are controlled. Specifically, a PWM signal is sent to the fan drive circuit (duty cycle 0-100% adjustable) to control the eddy current fan. The target wind speed calculation formula is: duty_cycle = min(100, (T_current - 60) * 2). The micro air pump (flow rate 5L / s) is activated to maintain the CO2 level in the vehicle through the PID algorithm. 2 The concentration is less than 1000ppm, thus controlling the A-pillar air duct.

[0038] In an emergency state (danger_level=2), a Hard Reset signal (50ms low-level pulse) is sent to the central control through the safety watchdog circuit to forcibly take over control; the door unlocking command is broadcast through the HS-CAN bus (message ID0x7FF, data field 0x0F). And multi-modal warnings are performed, such as the central control screen displays a red flashing interface at a frequency of 2Hz, and provides AR guidance on escape routes. By starting the seat vibration motor, such as 3 short vibrations + 1 long vibration, the user is prompted to an emergency state. In an embodiment of the present invention, after receiving the decision structure of the danger judgment module 130, the control module 140 parses the action_code within 5ms, and ensures that the emergency command is executed first through the hardware priority arbitrator (FPGA implementation) (response delay <10ms).

[0039] In an embodiment of the present invention, the system further includes an identification module for indicating the position of the door handle through a laser device placed at a preset position. The triggering condition of the identification module is danger_level ≥ 1 and the ambient light sensor detects that the light is less than 50 lux. The laser projection is through a 532nm green laser in the A-pillar to generate a 30cm×10cm rectangular light spot with adjustable brightness at the door handle position.

[0040] In the embodiment of the present invention, the system also includes a voice module, which is used to give voice prompts to the user for corresponding operations in a high-risk state or an emergency state. Among them, the multi-language TTS engine can automatically select the language according to the vehicle registration information associated with the VIN code; through sound field focusing, the seat headrest speaker is used for directional broadcasting (beamforming technology) to reduce external interference.

[0041] The embodiment of the present invention is connected to the central control device of the vehicle-mounted intelligent driving cockpit and at least two backup batteries distributed at different positions in the cockpit through the system. The system includes a sensor data acquisition module, a cockpit monitoring data acquisition module, a danger judgment module, and a control module. It can be connected to the system by setting up additional multiple backup power supplies, so that the vehicle can sense the battery temperature change in advance through the vehicle deformation data and sensor data and battery temperature in combination with the large model, and make a danger classification warning in advance. The system can forcibly take over control in an emergency state, use backup batteries in different positions to supply power, and combine with the setting of door handle signs to prompt users to quickly open the door and escape quickly in a dangerous state.

[0042] Figure 3 FIG. 1 shows a schematic diagram of the structure of an on-board intelligent cockpit terminal based on an AI large model provided by an embodiment of the present invention. Figure 3 As shown, the device 300 includes: a sensor 305, a camera 307, one or more processors 302 and a communication interface 304; Multiple sensors 305 , camera 307 , one or more processors 302 , and communication interface 304 interact via a communication bus 308 ; It can be understood that the vehicle-mounted intelligent cockpit system based on the AI ​​large model can be composed of a program 310, which is stored in the memory 306, and the processor calls the program 310 in the memory 306 to perform the functions of the vehicle-mounted intelligent cockpit system based on the AI ​​large model. The processor performs the functions of each module in the above embodiment.

[0043] The embodiment of the present invention is connected to the central control device of the vehicle-mounted intelligent driving cockpit and at least two backup batteries distributed at different positions in the cockpit through the system. The system includes a sensor data acquisition module, a cockpit monitoring data acquisition module, a danger judgment module, and a control module. It can be connected to the system by setting up additional multiple backup power supplies, so that the vehicle can sense the battery temperature change in advance through the vehicle deformation data and sensor data and battery temperature in combination with the large model, and make a danger classification warning in advance. The system can forcibly take over control in an emergency state, use backup batteries in different positions to supply power, and combine with the setting of door handle signs to prompt users to quickly open the door and escape quickly in a dangerous state.

[0044] An embodiment of the present invention also provides a car, including a car body, and the AI ​​big model-based vehicle-mounted intelligent cockpit terminal arranged in the car body.

[0045] The algorithm or display provided herein is not inherently related to any particular computer, virtual system or other equipment. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the embodiment of the present invention is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description made to specific languages ​​above is for disclosing the best mode of the present invention.

[0046] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known systems, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0047] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed system should not be interpreted as reflecting the following intention: that the claimed invention requires more features than those expressly recited in each claim.

[0048] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and set in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and may be divided into multiple submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any system or device disclosed in this manner may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0049] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. An in-vehicle intelligent cockpit system based on an AI large model, characterized in that: The system is arranged in a vehicle-mounted intelligent cockpit terminal, and the vehicle-mounted intelligent cockpit terminal is respectively connected to a central control device of a vehicle-mounted intelligent driving cockpit and at least two backup batteries distributed at different positions in the cockpit, and the system includes: Sensor data acquisition module, used to obtain various sensor time series data of the car in real time; The cockpit monitoring data acquisition module is used to obtain the vehicle status data sent by the central control device in real time; A danger judgment module is used to input the sensing sequence data and the vehicle status data into the danger prediction model in real time to determine whether it is in a dangerous state, the danger level and the corresponding strategy; wherein the danger prediction model includes a battery temperature curve prediction module and a multi-task output module; the temperature curve prediction module includes a lightweight Llama3-8B superimposed LSTM timing head; wherein the vehicle data sample includes a sensing timing data sample and a vehicle status data sample, and the sensing data sample includes the battery historical temperature data, the door simulation deformation data and the acceleration history data and the corresponding danger level label within a preset time period before the danger occurs; The control module is used for recording the sensor sequence data, the dangerous state and the dangerous level without active intervention when it is judged that the dangerous state is in a dangerous state and the dangerous level is normal; starting the roof vortex fan and injecting fresh air through the A-pillar air duct when the dangerous state is judged to be a high-risk state; and forcibly taking over the control of the central control device through the safety watchdog circuit when the dangerous state is judged to be an emergency state, controlling the release of the electronic restrictions of the door locks of the entire vehicle, and displaying the danger warning prompt information on the display screen of the central control.

2. The system according to claim 1, characterized in that The system also includes a training module; the training module is used to obtain automobile data samples, and train the lightweight Llama3-8B superimposed LSTM timing head according to the training data to obtain a large risk prediction model.

3. The system according to claim 2, characterized in that The training module further comprises: A sample acquisition unit, used to acquire original sample data, pre-process the original sample data, and obtain a car data sample; A model building unit is used to obtain Llama3-8B and an LSTM time series model, preprocess the Llama3-8B to obtain a lightweight Llama3-8B, and superimpose the lightweight Llama3-8B and the LSTM time series model to obtain a battery temperature curve prediction module; and obtain a multi-task learning module through a regression task output head and a classification task output head; The iterative training unit is used to iteratively input the automobile data samples into the lightweight Llama3-8B superimposed LSTM timing head for training, adjust the model parameters of the lightweight Llama3-8B superimposed LSTM timing head, and obtain the trained large risk prediction model.

4. The system according to claim 3, characterized in that The sample acquisition unit is specifically used for: Build a virtual test field to simulate the sensor time series data samples and vehicle status data samples under vehicle collision under extreme working conditions, and label them to obtain the original simulation sample data; The historical data of automobile accidents is obtained from the accident database of automobile companies, the claims database of insurance companies and the public accident reports of the government using differential privacy processing. The sensor time series data samples and automobile status data samples under historical automobile accidents are collected, and the data integrity is verified. After labeling, the original historical sample data is obtained; The original simulation sample data and the original historical sample data are preprocessed to serve as the automobile data samples.

5. The system according to any one of claims 1 to 4, characterized in that: The model building unit is also specifically used for: Use the cloud-based large model as the teacher model, perform feature extraction on the car sample data, and distill the student model Llama3-8B; Remove the attention heads not related to security in the Llama3-8B, retain 20% of the cross-modal attention layers, and obtain the lightweight Llama3-8B; Based on the lightweight Llama3-8B, an LSTM timing header is added to predict the battery temperature curve.

6. The system according to any one of claims 1 to 4, characterized in that: The system further comprises an identification module for indicating the position of the door handle by means of a laser device placed at a preset position.

7. The system according to any one of claims 1 to 4, characterized in that: The system also includes a voice module, which is used to voice prompt the user to perform corresponding operations in a high-risk state or an emergency state.

8. An in-vehicle intelligent cockpit terminal based on an AI large model, characterized in that: include: multiple sensors, cameras, one or more processors, and communication interfaces; The processor includes operations corresponding to the in-vehicle intelligent cockpit system based on the AI ​​big model as described in any one of claims 1-7.

9. A car, characterized in that: It includes a car body, and an on-board intelligent cockpit terminal based on the AI ​​big model as described in claim 8, which is arranged in the car body.

Images