Driver state monitoring method and system for intelligent driving

Abstract

The invention discloses a driver state monitoring method and system for intelligent driving, and the method comprises the steps that a cabin chip obtains driver state information based on a received driver video stream, and transmits the driver state information to a cabin microcontroller; the cabin microcontroller receives the driver state information and sends the driver state information to the intelligent driving domain controller; and the intelligent driving domain controller receives the driver state information and determines a corresponding intelligent driving control instruction according to the driver state information. According to the method, a cabin microcontroller is introduced, and cooperative monitoring is constructed with a cabin chip and an intelligent driving domain controller, so that decoupling of high-performance state analysis and high-reliability safety monitoring is realized, and a single-point fault risk caused by dependence of a single chip is effectively avoided, so that stability and reliability of intelligent driving are maintained, and a DMS can continuously obtain an effective driver state.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving safety of vehicles, and in particular to a driver state monitoring method and system for intelligent driving. BACKGROUND

[0002] At present, with the development of intelligent driving, major vehicle OEMs (Original Equipment Manufacturer) begin to develop L3 (conditional intelligent driving) intelligent driving systems. Among them, the L3 system is responsible for all dynamic driving tasks after activation, but when the intelligent driving system detects an anomaly and requests the driver to take over, the driver must respond in a timely manner within a certain time. This means that the intelligent driving system must have the ability to continuously and reliably monitor the state of the driver, such as whether the driver is concentrating, whether the driver is tired, and whether the driver is distracted, to ensure that the driver can be effectively reminded and handed over the control of the vehicle when needed.

[0003] Therefore, the driver state monitoring system (Driver Monitoring System, DMS) has become a key component to ensure the safety of L3 intelligent driving functions. The DMS needs to meet the specified automotive safety integrity level (Automotive Safety Integrity Level, ASIL) - ASIL B level to ensure that the driver's state can be reliably detected when the intelligent driving system requests to take over. The existing DMS scheme directly connects the camera to the system chip (System on Chip, SoC) of the cabin domain controller, and the SoC completes image processing and state analysis.

[0004] However, the existing DMS scheme faces severe challenges in achieving the above safety goals and cannot guarantee the continuous reliability of the DMS. On the one hand, the SoC as the core computing unit is mainly designed for non-safety critical functions such as infotainment, and generally lacks the hardware safety mechanisms required to meet the ASIL B level. On the other hand, the architecture of integrating DMS functions completely in a single SoC has a single point of failure risk. Once the SoC fails due to failure, it will directly cause the driver monitoring function to be interrupted, endangering road safety.

[0005] Therefore, there is an urgent need for a highly reliable DMS safety monitoring scheme to break through the bottleneck of existing technology and provide key protection for the safety of L3 intelligent driving. SUMMARY

[0006] In order to maintain the stability and reliability of the intelligent driving system and enable the DMS to continuously obtain effective driver states, a driver state monitoring method and system for intelligent driving are provided in the embodiments of the present application.

[0007] In a first aspect, embodiments of the present application provide a driver state monitoring method for intelligent driving, applied to a driver state monitoring system for intelligent driving, the driver state monitoring system for intelligent driving comprising a cockpit chip, a cockpit microcontroller and an intelligent driving domain controller, and the method comprising: The cockpit chip obtains driver state information based on a received driver video stream, and sends the driver state information to the cockpit microcontroller; The cockpit microcontroller receives the driver state information, and sends the driver state information to the intelligent driving domain controller; The intelligent driving domain controller receives the driver state information, and determines a corresponding intelligent driving control instruction according to the driver state information.

[0008] In one or some optional embodiments of the present application, the following are further included: The cockpit microcontroller performs CRC check and serial number check on communication data between the cockpit microcontroller and the cockpit chip; The cockpit microcontroller performs CRC check and serial number check on communication data between the cockpit microcontroller and the intelligent driving domain controller.

[0009] In one or some optional embodiments of the present application, the following are further included: If the cockpit microcontroller detects CRC check failure or serial number check failure during communication data transmission with the cockpit chip, the cockpit microcontroller sends a communication data retransmission instruction to the cockpit chip, and records a current retransmission number; If the current retransmission number is greater than a preset maximum retransmission number, the cockpit microcontroller determines that the cockpit chip has a communication fault, and determines a corresponding fault code, fault state and timestamp to form first fault information.

[0010] In one or some optional embodiments of the present application, the following are further included: The cockpit microcontroller receives a heartbeat signal and a state flag of the cockpit chip, and performs heartbeat monitoring counting based on the heartbeat signal to obtain a heartbeat health state; If at least any one of the heartbeat health state and the state flag is detected to have a fault, the cockpit microcontroller determines a corresponding fault code, fault state and timestamp to form second fault information.

[0011] In one or some optional embodiments of the present application, the following are further included: The cockpit microcontroller performs self-checking according to a preset period to obtain self-checking information.

[0012] In one or some optional embodiments of the present application, the following are further included: The cabin microcontroller determines a fault level according to the first fault information, the second fault information and the self-check information; the fault level includes a light fault, a medium fault, a serious fault and a fatal fault.

[0013] In one or some optional embodiments of the embodiments of the application, the method further comprises: The cabin microcontroller sends the fault information to the intelligent driving domain controller when the fault level is the light fault. The cabin microcontroller sends the fault information to the intelligent driving domain controller and instructs the intelligent driving domain controller to trigger intelligent driving system function degradation when the fault level is the medium fault. The cabin microcontroller sends a hard reset instruction or a power-off instruction to a fault unit corresponding to the fault information when the fault level is the serious fault. The cabin microcontroller activates a preset fault safety output pin to force a fault unit corresponding to the fault information to perform reset or power-off when the fault level is the fatal fault.

[0014] In a second aspect, the embodiments of the application provide a driver state monitoring system for intelligent driving, comprising a cabin chip, a cabin microcontroller and an intelligent driving domain controller, The cabin chip is configured to obtain driver state information based on a received driver video stream and send the driver state information to the cabin microcontroller. The cabin microcontroller is configured to receive the driver state information and send the driver state information to the intelligent driving domain controller. The intelligent driving domain controller is configured to receive the driver state information and determine a corresponding intelligent driving control instruction according to the driver state information.

[0015] In a third aspect, the embodiments of the application provide a computer readable storage medium having a computer program / instruction stored thereon, the computer program / instruction being executed by a processor to implement the driver state monitoring method for intelligent driving as described above.

[0016] In a fourth aspect, the embodiments of the application provide a computer program product comprising a computer program / instruction, the computer program / instruction being executed by a processor to implement the driver state monitoring method for intelligent driving as described above.

[0017] In a fifth aspect, the embodiments of the application provide a computer device comprising a memory, a processor and a computer program stored on the memory, the processor executing the computer program to implement the driver state monitoring method for intelligent driving as described above.

[0018] The beneficial effects of the above technical solutions provided by the embodiments of the present application at least include: The embodiments of the present application provide a driver state monitoring method for intelligent driving, which is applied to a driver state monitoring system for intelligent driving, and the driver state monitoring system for intelligent driving comprises a cockpit chip, a cockpit microcontroller and an intelligent driving domain controller. The method introduces the cockpit microcontroller to jointly build a cooperative monitoring mechanism with the cockpit chip and the intelligent domain controller. The method decouples the high-performance driver state analysis function and the high-reliability safety monitoring and communication forwarding function, the cockpit chip is responsible for executing complex image algorithms to obtain driver state information, and the cockpit microcontroller is responsible for building an independent and reliable data channel to ensure that the driver state information can be safely and accurately transmitted to the intelligent driving domain controller for decision-making to determine the intelligent driving control instruction. This design effectively solves the single-point failure risk caused by the high dependence of the existing scheme on a single non-safety level cockpit chip, enables the DMS to continuously obtain effective driver state, and further improves the robustness and functional safety level of the driver state monitoring method, thereby providing a solid foundation for the safe application of L3 and above intelligent driving systems.

[0019] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be achieved and obtained by the structure particularly pointed out in the written description and the accompanying drawings.

[0020] The technical solutions of the present application will be further described in detail below with the help of the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings are intended to provide a further understanding of the present application, and constitute a part of the specification, together with the embodiments of the present application, to explain the present application, and do not constitute a limitation of the present application. In the drawings: Figure 1 A flowchart of the driver state monitoring method for intelligent driving provided by the embodiments of the present application is shown in the figure; Figure 2 A data verification mechanism flowchart of the cockpit microcontroller provided by the embodiments of the present application is shown in the figure; Figure 3 A safety monitoring mechanism flowchart of the cockpit microcontroller provided by the embodiments of the present application is shown in the figure; Figure 4 An architecture diagram provided by the embodiments of the present application is shown in the figure; Figure 5 A flowchart provided by the embodiments of the present application is shown in the figure; Figure 6A structural schematic diagram of a driver state monitoring system for intelligent driving provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0022] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0023] The inventors have found that in the prior art, a common DMS solution directly connects a camera to a SoC of a cabin domain controller, and the SoC completes image processing and driver state analysis. However, this architecture has obvious functional safety defects: First, the functional safety level is insufficient. Commercial cabin SoCs are usually designed for non-safety-critical tasks such as infotainment, and only support a lower ASIL level such as QM or ASIL A level, lacking hardware safety mechanisms.

[0024] At the same time, all data is processed by a single SoC, and once the SoC fails, the entire DMS will be directly disabled, which cannot effectively monitor the driver's state and may cause the driver to be unable to take over the vehicle when needed, resulting in serious safety risks.

[0025] In addition, some existing solutions use a high-cost single camera plus a dual-camera signal conversion chip to make up for the lack of safety level, that is, to simulate two video streams from a single camera for redundant processing, which undoubtedly increases the complexity and overall cost of the DMS system, making it difficult to popularize in mainstream vehicles.

[0026] Therefore, there is an urgent need in the industry for a low-cost and high-reliability DMS access and safety monitoring solution to break through the bottleneck of existing technology and provide key protection for the safe landing of L3 intelligent driving. Based on this, the inventors have made the present application after further research and development, and provide a driver state monitoring method and system for intelligent driving.

[0027] Embodiment One In an embodiment of the present application, a driver state monitoring method for intelligent driving is provided, which is applied to a driver state monitoring system for intelligent driving. As shown in Figure 1 The driver state monitoring system for intelligent driving includes a cabin chip, a cabin microcontroller, and an intelligent driving domain controller. The method includes steps S101-S103: S101: The cabin chip obtains driver state information based on the received driver video stream, and sends the driver state information to the cabin microcontroller.

[0028] S102: The cabin microcontroller receives the driver state information, and sends the driver state information to the intelligent driving domain controller.

[0029] S103: The intelligent driving domain controller receives the driver state information, and determines the corresponding intelligent driving control instruction according to the driver state information.

[0030] The embodiment of the application provides a driver state monitoring method for intelligent driving, which is applied to a driver state monitoring system for intelligent driving, and the driver state monitoring system for intelligent driving comprises a cabin chip, a cabin microcontroller and an intelligent driving domain controller. The method introduces the cabin microcontroller, and cooperates with the cabin chip and the intelligent domain controller to construct a cooperative monitoring mechanism. The method decouples the high-performance driver state analysis function and the high-reliability safety monitoring and communication forwarding function, the cabin chip is responsible for executing complex image algorithms to obtain driver state information, and the cabin microcontroller is responsible for constructing an independent and reliable data channel, so that the driver state information can be safely and accurately transmitted to the intelligent driving domain controller to make decisions and determine the intelligent driving control instruction. This design effectively solves the single-point failure risk caused by the high dependence of the existing scheme on a single non-safety level cabin chip, enables the DMS to continuously obtain effective driver state, and further improves the robustness and functional safety level of the driver state monitoring method, thereby providing a solid foundation for the safe application of the L3 and above intelligent driving system.

[0031] In the embodiment of the application, the camera module can obtain the driver video stream and send the driver video stream to the cabin chip.

[0032] Specifically, the camera module can be a camera arranged in the vehicle cabin and pointing to the driver's face or upper body area. The camera is responsible for real-time acquisition of original video data containing driver facial details, such as eye opening degree, line of sight direction, etc.

[0033] The camera module can also integrate an image signal processor to perform preprocessing such as noise reduction, exposure correction and format conversion on the original video data, to form a driver video stream meeting the requirements of subsequent processing. The driver video stream will be transmitted to the cabin chip in real time and with low delay as the original data source for driver state analysis.

[0034] In the above step S101, the cabin chip obtains driver state information based on the received driver video stream, and sends the driver state information to the cabin microcontroller.

[0035] Specifically, the cockpit chip can be a high-performance computing core of the method, which is essentially a system-on-chip (SoC) that follows a typical software and hardware layered architecture: At the hardware layer, it provides dedicated computing units for image processing, neural network acceleration, etc., providing computing power for the upper-layer algorithms. At the bottom software and operating system layer, it runs complex multi-task operating systems such as Linux and QNX, responsible for system resource management and scheduling.

[0036] At the application layer, there is a dedicated DMS algorithm. Based on the computing power provided by the hardware, the DMS algorithm analyzes and calculates the driver video stream from the camera module frame by frame, and through the analysis of the driver's facial features, eye movements (such as eye opening and closing frequency, gaze direction), head posture, etc., the driver's state result is obtained in real time.

[0037] The cockpit chip encapsulates the current state result of the driver as structured data information, i.e., driver state information, which can represent that the driver is currently in a state of concentration, distraction, fatigue, etc.

[0038] Subsequently, the cockpit chip sends the driver state information to the cockpit microcontroller through the established independent hard-wired communication link (such as SPI or I2C) between the cockpit chip and the cockpit microcontroller. This direct communication based on the hard-wired link can effectively avoid the delay and congestion problems caused by the vehicle network bus, providing a high-deterministic data transmission channel for safety monitoring.

[0039] In the embodiments of the present application, the cockpit chip can also monitor the state information of the camera module, and send camera fault information to the cockpit microcontroller when the state information of the camera module indicates that the camera module has a fault.

[0040] Specifically, the cockpit chip can continuously query or receive heartbeat signals, data stream status, and internal diagnostic codes, etc. state information from the camera module through a dedicated interface (such as MIPI CSI) or a general I / O interface between the cockpit chip and the camera module. These state information are used to determine the working health status of the camera module. For example, if the cockpit chip does not receive valid heartbeat signals for a number of monitoring periods, or detects that the driver data stream is continuously interrupted or abnormal in format, or analyzes a specific internal fault diagnostic code reported by the camera module, it is determined that the camera module has a fault. Subsequently, the cockpit chip generates structured data containing fault type, fault level, and timestamp as camera fault information, and synchronously sends the camera fault information to the cockpit microcontroller through the independent hard-wired communication link, so as to perform unified safety management and fault reporting.

[0041] Among them, the fault type corresponding to the camera module mainly covers various abnormalities in its function, such as communication interruption caused by physical connection problem, which is manifested as complete loss of video signal; loss of data validity caused by sensor or processor abnormality, which is manifested as persistent image of screen, black frame or serious distortion; and self-checking failure triggered by abnormal working state of internal components, such as sensor overheating, initialization timeout or power supply abnormality. Based on the threat degree of these faults to the functional safety of the method, they can be divided into different fault levels: for slight abnormalities that occur instantaneously and can be quickly self-recovered, such as single communication error code or instantaneous frame loss, they can be classified as light-level faults; for performance degradation that leads to continuous decline in image quality but can still be partially used for state analysis, they can be classified as medium-level faults; and for faults that lead to complete and continuous interruption of the driver video stream, causing substantial failure of the DMS core function, they can be classified as serious faults.

[0042] In the above step S102, the cabin microcontroller receives the driver state information and sends the driver state information to the intelligent driving domain controller.

[0043] Specifically, the cabin microcontroller can be a microcontroller unit (MCU) that is independent of the cabin chip and has an ASIL B functional safety level. Its core hardware selection can be a vehicle-grade MCU such as Infineon TC3xx series or NXP S32K series. The MCU internally integrates key hardware modules that meet functional safety requirements, such as lockstep cores for implementing high-reliability computing, hardware security modules for providing encryption and secure boot functions, and rich peripheral communication interfaces. Based on this hardware foundation, the cabin microcontroller can play the role of an independent security guard in the method, independent of the operating system and scheduling resources of the cabin chip, to achieve high-deterministic safety monitoring and execution. This safety monitoring architecture based on independent hardware not only strictly controls the response time of fault detection and processing, meeting the stringent requirements of ASIL B level on time constraints, but also is easy to integrate into existing cabin domain controller architectures, without the need for significant changes to existing hardware and software designs.

[0044] In the embodiments of the present application, for the convenience of understanding, all the functions and methods contained in the cabin microcontroller are explained based on three mechanisms: communication data verification mechanism, safety monitoring mechanism and fault grading response mechanism. The cabin microcontroller is described with respect to the three mechanisms as follows: Communication data verification mechanism: The cabin microcontroller, as the security hub in the communication link of the method, is responsible for performing strict end-to-end protection verification on all the communication data flowing through it. Specifically, this communication data verification mechanism is embodied in two aspects: Firstly, on the data receiving side, when the cabin microcontroller obtains the communication data from the cabin chip through the hard link such as SPI, the communication data received by the cabin microcontroller not only includes the driver state information, but also includes a series of safety monitoring data for health diagnosis in the method. These data jointly constitute the communication content between the two, and the safety monitoring data includes but is not limited to: heartbeat signal reflecting normal operation of cabin chip software task, state flag reflecting internal diagnosis state of cabin chip itself, and camera failure information of camera module. After receiving each frame of communication data, the cabin microcontroller will immediately start the verification process, and perform cyclic redundancy check (CRC) on the communication data frame to verify whether bit error or tampering occurs in the transmission process. At the same time, the cabin microcontroller also checks the serial number contained in the communication data frame to determine whether the data is lost, repeated or out of order.

[0045] Similarly, on the data sending side, when the cabin microcontroller prepares to send the integrated communication data including the driver state information, self-checking information, etc. to the intelligent driving domain controller through the bus, the cabin microcontroller will actively attach CRC check code and serial number to the communication data frame before sending, thereby building complete integrity protection on the entire communication data link.

[0046] Secondly, the communication data verification mechanism has error recovery and fault diagnosis capability. When the cabin microcontroller detects CRC check failure or discontinuous serial number in the communication with the cabin chip, the cabin microcontroller will judge that the communication data is invalid, discard the communication data, and immediately reply a negative acknowledgement to the cabin chip, send a communication data retransmission instruction to the cabin chip, and request the cabin chip to retransmit the communication data. At the same time, an internal counter is started to record the current retransmission number. If the communication is restored after several retries, the process continues; if the communication still fails after the current retransmission number accumulates more than the preset maximum retransmission number (for example, 3 times), the cabin microcontroller will determine that a persistent communication fault occurs. At this time, the cabin microcontroller will generate a data packet containing a specific fault code (such as 0x10 representing communication timeout), fault state (such as "permanent fault") and timestamp according to the preset rules, and use it as the first fault information for subsequent fault management and reporting.

[0047] For the convenience of those skilled in the art, the above communication data verification mechanism is more clearly explained based on the flowchart as follows: refer to Figure 2As shown, after the MCU (i.e., the cabin microcontroller) receives the data frame, it enters the "CRC check & serial number check" link. If the check is passed, it is determined that the data is valid and the subsequent process of "processing data" is executed, and sent to the intelligent driving domain controller. If the check is not passed, it is determined that the data is invalid, and the data is discarded immediately, and a reply not ok is returned to request the cabin chip to retransmit. Among them, the reply not ok and the triggering of the retransmission mechanism, i.e., the communication data retransmission instruction mentioned above, are the negative acknowledgement and the retransmission instruction sent to the cabin chip. At the same time, the "retransmission mechanism i++" is triggered, i.e., the current retransmission number is recorded, as mentioned above. Among them, i is the current retransmission number. Subsequently, it is determined whether the current retransmission number has reached the "maximum retransmission number", i.e., the preset maximum retransmission number mentioned above. If not, wait for the SoC to retransmit, accept the data frame, and return to the starting CRC check & serial number check link. If it has reached, it is "determined that the communication is permanently faulty", and finally "safety fault handling" is executed, i.e., the fault code is recorded and reported to the intelligent driving domain controller, completing the closed-loop management of the communication link fault.

[0048] Safety monitoring mechanism: In addition to performing communication data verification, the cabin microcontroller also undertakes the task of continuous safety monitoring of the running state of the cabin chip and its own health state. This safety monitoring mechanism is embodied in two aspects: monitoring of external units and monitoring of itself.

[0049] First, in terms of external unit monitoring, the cabin microcontroller periodically reads the heartbeat signal and status flag from the cabin chip through an independent hard-wired link. The cabin microcontroller will perform heartbeat monitoring counting based on the received heartbeat signal, and analyze the heartbeat health status by judging whether the counting value increases as expected and whether the response is completed within the preset time window. If the cabin microcontroller detects at least any one of heartbeat health state abnormalities, such as heartbeat monitoring counting stagnation or response timeout, or identifies that the status flag reported by the cabin chip indicates that there is a fault in it, it will immediately trigger the fault handling process. The cabin microcontroller will determine the corresponding fault code (e.g., 0x20 represents heartbeat timeout) and fault state according to the preset mapping rule, and attach a timestamp, and combine these information into the second fault information, which provides the basis for subsequent fault diagnosis and response.

[0050] Among them, the fault type corresponding to the cockpit chip mainly covers various abnormalities on the running state of the cockpit chip, such as heartbeat signal abnormalities caused by task scheduling deadlock or calculation core lock, which are manifested as heartbeat count stagnation or permanent loss; state flag abnormalities caused by internal hardware module (such as memory, bus) error or software diagnosis trigger, which are manifested as reporting specific internal error code or fault identification; and chip function integrity failure caused by power supply, clock and other basic running environment problems. Based on the threat degree of these faults to the functional safety of the method, they can be divided into different fault levels: for slight abnormalities that occur instantaneously and can be quickly self-recovered, such as single heartbeat response slight delay or transient false alarm of state flag, which can be classified as light level fault; for state abnormalities that cause cockpit chip performance to fluctuate continuously but the DMS algorithm can still run, such as heartbeat periodic jitter or intermittent alarm of state flag, which can be classified as medium level fault; and for faults that cause cockpit chip function to completely lose, resulting in substantial interruption of the driver state recognition function, such as permanent loss of heartbeat signal or reporting of unrecoverable serious hardware error, which are classified as serious fault.

[0051] Secondly, in terms of self-monitoring, the cockpit microcontroller will perform a self-checking program once every preset period (for example, every 10 milliseconds) to ensure its reliability as a safety monitoring unit. The self-checking program covers its core functional modules, such as program memory, data memory and communication interface, etc., thereby generating self-checking information reflecting its own health status.

[0052] For the convenience of those skilled in the art to understand, the above safety monitoring mechanism is more clearly explained based on the flowchart as follows: referring to Figure 3 As shown in Figure 3The third step from the right, first read the SoC heartbeat, check code, status flag through SPI, that is, the heartbeat signal and status flag obtained from the cockpit chip as described above, and the check code in the figure is the CRC check in the communication data check mechanism, which is not described here. Then, the success of the communication is judged: if the communication fails, the communication failure error is directly recorded and the fault flag is set; if the communication is successful, the data analysis and processing of the read data are carried out, and the logic judgment of "heartbeat monitoring count increment & response timeout" is entered, wherein the response timeout is the judgment of whether the response is completed within the preset time window based on the heartbeat signal as described above. As long as any of the above checks fails, it is determined as a fault. Then, the cockpit microcontroller records the specific fault type and timestamp, and evaluates the fault severity level (instantaneous, continuous, multiple), then the cockpit microcontroller assembles the fault information frame (fault code, state, timestamp), that is, the second fault information as described above, and finally sends it to the intelligent driving domain controller through CAN FD (bus) communication. After processing, it will wait for the next monitoring period, thus starting a new round of safety monitoring cycle, realizing continuous and periodic monitoring and fault reporting of the cockpit chip state.

[0053] Fault classification response mechanism: The cockpit microcontroller integrates complete fault classification and response logic, which determines the fault level based on the first fault information, the second fault information and the self-checking information. Among them, the fault level is divided based on the severity and time urgency of the fault, including light level fault, medium level fault, serious fault and fatal fault, the definition and requirements of which are shown in Table 1 as follows: Table 1 Fault level

[0054] Based on the fault level defined in the above table, the cockpit microcontroller executes the following classification response strategy: If the fault level is a light fault, for example, the cabin chip transient high load causes the heartbeat to slightly jitter, the scene is that the cabin microcontroller detects that the heartbeat signal of the cabin chip has once exceeded the expected normal receiving window, but has not reached the timeout standard, and it is judged that the cabin chip may have processed a complex scene, the CPU load is temporarily too high, and it can soon recover by itself. The cabin microcontroller sends the corresponding second fault information to the intelligent driving domain controller for recording and diagnosis, but does not interrupt the operation of the intelligent driving system function. A corresponding counter (such as crc_error_count) can be incremented in the internal non-volatile memory (NVM) of the cabin microcontroller, and a time stamp and possibly related data are recorded. The purpose of this operation is to enable post-sale diagnosis and health monitoring, and if a certain type of light fault occurs frequently within a short period of time, it can be regarded as a precursor to a systemic problem, and the user can be prompted to repair in advance.

[0055] If the fault level is a medium fault, for example, repeated failures occur in communication with the cabin chip, the scene is that the cabin microcontroller communicates with the cabin chip through a single data frame, and after a maximum number of retransmissions (such as 3 times), the communication still fails, but the communication link is then restored by itself, and it is judged that the communication link is seriously and continuously disturbed, but has not been completely interrupted. The cabin microcontroller sends the corresponding first fault information to the intelligent driving domain controller, and at the same time instructs the intelligent driving domain controller to trigger the intelligent driving system function degradation, such as limiting the use of high-level automatic driving function. Detailed fault codes, occurrence times and time stamps can be recorded and immediately reported to the intelligent driving domain controller, and the fault level flag is raised. The purpose of this operation is to enable the intelligent driving domain controller to know that the performance has been degraded, and to decide to limit part of the intelligent driving function accordingly, and to remind the driver that the system performance is limited, and to drive carefully.

[0056] If the fault level is a serious fault, for example, the heartbeat of the cabin chip is permanently lost, the scene is that the cabin microcontroller has not received any heartbeat signal from the cabin chip in a plurality of continuous monitoring periods, but the intelligent driving domain controller can receive the self-check signal from the cabin microcontroller, and it is judged that the cabin chip may have completely crashed or powered off. The cabin microcontroller will immediately send a hard reset instruction or power off instruction to the corresponding fault unit, i.e. the cabin chip, through a separate hardware watchdog circuit or GPIO pin, forcing it to enter a harmless state to achieve "fault silence", and recording fault information as a highest priority task. At the same time or after taking safety actions, the cabin microcontroller immediately sends a highest priority fault message to the intelligent driving domain controller, declaring that the cabin chip has failed and has been reset. The purpose of this operation is to force the cabin chip into a harmless state before violating safety goals.

[0057] If the fault level is a fatal fault, that is, the cabin microcontroller self-check fails, its scenario is that the cabin microcontroller does not receive any heartbeat signal of the cabin chip in a plurality of continuous monitoring periods, and the intelligent driving domain controller also cannot receive the self-check signal of the cabin microcontroller, it is judged that the cabin microcontroller itself has failed and cannot reliably monitor the cabin chip. At this time, since the cabin microcontroller itself has failed, it may not be able to send valid data, and the intelligent driving domain controller will detect the fault because it cannot receive the self-check signal of the cabin microcontroller. The cabin microcontroller will activate a preset fail-safe output pin (FSO), and through an independent hardware circuit, the fault unit, that is, the cabin microcontroller and the cabin chip, is forced to activate its final hardware safety output. The pin is directly connected to the reset or power-off circuit of the cabin microcontroller and the cabin chip. Even if the software of the cabin microcontroller has completely stuck, this hardware circuit will also forcibly pull down / pull up the pin level, thereby ensuring that the cabin microcontroller and the cabin chip are reset or powered off.

[0058] In the embodiment of the application, the cabin microcontroller is further configured to receive camera fault information and send the camera fault information to the intelligent driving domain controller.

[0059] Specifically, after monitoring that the camera module fails, the cabin chip generates camera fault information containing the fault type, level and timestamp, and sends it out through an independent hard-wired communication link (such as SPI) between the cabin chip and the cabin microcontroller. After receiving the information, the cabin microcontroller integrates it with other fault information (such as the first fault information and the second fault information) obtained by itself. Then, the cabin microcontroller sends the integrated information or the separate camera fault information to the intelligent driving domain controller through the bus, thereby ensuring that the intelligent driving system can comprehensively and timely obtain the health status of all key components of the method, and providing a complete basis for the safety decision of the whole vehicle.

[0060] In step S103, the intelligent driving domain controller receives the driver state information, and determines the corresponding intelligent driving control instruction according to the driver state information.

[0061] Specifically, the intelligent driving domain controller can be the decision core of vehicle intelligent driving, continuously receiving the driver state information forwarded by the cabin microcontroller and verified for safety. The driver state information is the key input data for judging whether the driver has the ability to take over. The intelligent driving domain controller is internally preset with a safety strategy logic corresponding to the L3 level automatic driving function.

[0062] When the received driver status information indicates that the driver is focused, alert, and has their eyes on the road, meeting the requirements for takeover, the intelligent driving domain controller determines that the driver is in good condition and then determines the intelligent driving control command to maintain or enter autonomous driving mode, and the intelligent driving system continues to be responsible for vehicle control.

[0063] When the received driver status information indicates that the driver is distracted, fatigued (e.g., closing their eyes, frequently nodding), or has their gaze deviating from the road for an extended period, thus failing to meet the takeover requirements, the intelligent driving domain controller determines that the driver's condition is poor. At this point, the intelligent driving domain controller will determine whether to disengage from or disable the intelligent driving control command for autonomous driving, and simultaneously trigger a tiered warning mechanism. This may involve prompting the driver to take over immediately through sound, visual, or tactile means, while simultaneously preparing to execute a minimum-risk strategy, such as smoothly decelerating the vehicle, activating hazard warning lights, and safely pulling the vehicle to the side of the road.

[0064] To facilitate understanding by those skilled in the art, this method is explained more clearly here based on architecture diagrams and flowcharts: (Refer to...) Figure 4 The architecture diagram shown and Figure 5 As shown in the flowchart, this method begins with the DMS camera (i.e., the camera module) to capture the driver's video stream. Subsequently, the driver's video stream is sent to the cockpit controller SoC (i.e., the cockpit chip), where the DMS algorithm runs internally to accurately monitor the driver's state, obtain driver state information, and determine the camera's own state (i.e., acquire camera fault information). The driver state information is then sent to the cockpit controller MCU (i.e., the cockpit microcontroller), which is ASIL B level and responsible for monitoring the SoC's operating status (such as power supply voltage, internal faults, etc.) and receiving the driver state it identifies (i.e., driver state information). All critical data transmissions between the SoC and the MCU, and between the MCU and the intelligent driving controller (i.e., the intelligent driving domain controller), are protected by E2E (end-to-end) protection. E2E protection, as described above, includes a communication protection mechanism with CRC checksum and sequence number verification, ensuring data integrity and reliability throughout the entire link. Ultimately, the intelligent driving controller integrates all information to determine the fault status of this method (covering cameras, SoC, and MCU), and determines whether the driver meets the L3 driving requirements based on the driver's status information. It then determines and executes the corresponding intelligent driving control commands (such as maintaining autonomous driving, triggering graded warnings, or executing the minimum risk strategy), thus forming a complete closed loop from state perception and safety monitoring to vehicle decision-making.

[0065] This method introduces a low-cost microcontroller (MCU) with ASIL B functional safety level as the cockpit microcontroller, which collaborates with the cockpit chip to construct a dual-path monitoring architecture. Without significantly increasing costs, it effectively compensates for the inherent functional safety deficiencies of existing cockpit chips, ensuring that the entire driver status monitoring method meets ASIL B requirements. This application constructs an end-to-end integrity protection mechanism from the camera to the intelligent driving domain controller. Through redundancy checks and independent monitoring, it ensures the accuracy and reliability of driver status information transmission. Simultaneously, the hardware-based safety monitoring mechanisms (such as independent watchdogs and hardwired communication) are unaffected by the complex software environment of the cockpit chip, achieving high determinism and strict controllability in fault detection and handling response time, fully meeting the stringent time constraints of ASIL B. Furthermore, this method is compatible with mainstream cockpit domain controller architectures, requiring no significant modifications to existing hardware and software designs, and exhibits good integration and implementation feasibility.

[0066] Example 2 Based on the same inventive concept, embodiments of the present invention also provide a driver state monitoring system for intelligent driving, referring to... Figure 6 As shown, the system includes: a cockpit chip 11, a cockpit microcontroller 12, and an intelligent driving domain controller 13. The cockpit chip 11 is configured to obtain driver status information based on the received driver video stream and send the driver status information to the cockpit microcontroller 12. The cockpit microcontroller 12 is configured to receive the driver status information and send the driver status information to the intelligent driving domain controller 13; The intelligent driving domain controller 13 is configured to receive the driver status information and determine the corresponding intelligent driving control command based on the driver status information.

[0067] Example 3 Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program / instruction thereon, which, when executed by a processor, implements the driver state monitoring method for intelligent driving as described in Embodiment 1 above.

[0068] Example 4 Based on the same inventive concept, embodiments of the present invention also provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the driver state monitoring method for intelligent driving as described in Embodiment 1 above.

[0069] Example 5 Based on the same inventive concept, the embodiment of the present application also provides a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to realize the intelligent driving-oriented driver state monitoring method as described in the above embodiment one.

[0070] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage, etc.) containing computer-usable program code.

[0071] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices generate a device implemented in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in the flow(s) or block(s).

[0072] These computer program instructions can also be stored in a computer-readable memory capable of guiding the computer or other programmable data processing devices to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product comprising instruction apparatus, which realizes the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in the flow(s) or block(s).

[0073] These computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide a process for realizing the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in the flow(s) or block(s).

[0074] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A driver state monitoring method for intelligent driving, applied to a driver state monitoring system for intelligent driving, characterized in that, The driver status monitoring system for intelligent driving includes a cockpit chip, a cockpit microcontroller, and an intelligent driving domain controller; the method includes: The cockpit chip obtains driver status information based on the received driver video stream and sends the driver status information to the cockpit microcontroller; The cockpit microcontroller receives the driver status information and sends the driver status information to the intelligent driving domain controller; The intelligent driving domain controller receives the driver status information and determines the corresponding intelligent driving control command based on the driver status information.

2. The method according to claim 1, characterized in that, Also includes: The cockpit microcontroller will perform CRC check and serial number check on the communication data between itself and the cockpit chip; The cockpit microcontroller will perform CRC check and serial number check on the communication data between itself and the intelligent driving domain controller.

3. The method according to claim 2, characterized in that, Also includes: If the cockpit microcontroller detects a CRC check failure or a serial number check failure during communication data transmission with the cockpit chip, it sends a communication data retransmission instruction to the cockpit chip and records the current retransmission count. If the current number of retransmissions is greater than the preset maximum number of retransmissions, the cockpit chip is determined to have a communication failure, and the corresponding fault code, fault status and timestamp are determined to form the first fault information.

4. The method according to claim 3, characterized in that, Also includes: The cockpit microcontroller will receive the heartbeat signal and status flag from the cockpit chip, and perform heartbeat monitoring and counting based on the heartbeat signal to obtain the heartbeat health status; If a fault is detected in at least one of the heartbeat health status and the status flag, the corresponding fault code, fault status, and timestamp are determined to form the second fault information.

5. The method according to claim 4, characterized in that, Also includes: The cockpit microcontroller will perform self-tests according to a preset cycle and obtain self-test information.

6. The method according to claim 5, characterized in that, Also includes: The cockpit microcontroller determines the fault level based on the first fault information, the second fault information, and the self-test information; the fault level includes minor fault, medium fault, serious fault, and fatal fault.

7. The method according to claim 6, characterized in that, Also includes: When the fault level is a minor fault, the cockpit microcontroller sends the fault information to the intelligent driving domain controller. When the fault level is medium, the cockpit microcontroller sends the fault information to the intelligent driving domain controller and instructs the intelligent driving domain controller to trigger the intelligent driving system function degradation. When the fault level is a serious fault, the cockpit microcontroller sends a hard reset command or a power-down command to the fault unit corresponding to the fault information. When the fault level is a fatal fault, the cockpit microcontroller activates a preset failsafe output pin to force the fault unit corresponding to the fault information to perform a reset or power-off.

8. A driver status monitoring system for intelligent driving, characterized in that, include: Cockpit chips, cockpit microcontrollers, and intelligent driving domain controllers, The cockpit chip is configured to obtain driver status information based on the received driver video stream and send the driver status information to the cockpit microcontroller. The cockpit microcontroller is configured to receive the driver status information and send the driver status information to the intelligent driving domain controller. The intelligent driving domain controller is configured to receive the driver status information and determine the corresponding intelligent driving control command based on the driver status information.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instruction is executed by the processor, it implements the driver state monitoring method for intelligent driving as described in any one of claims 1-7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the driver state monitoring method for intelligent driving as described in any one of claims 1-7.

Images