Driver state detection method, device and electronic equipment

Abstract

The application provides a driver state detection method and device and electronic equipment, comprising: acquiring image information, physiological information of a driver and running state information of a driving vehicle; pre-processing the image information to obtain a pre-processed image; determining facial features, skin color features and human feature point position information of the driver according to the pre-processed image; determining vital sign information of the driver according to the skin color features and the physiological information; inputting the facial features and the human feature point position information into a machine learning model to output a limb stiffness score and a posture stability score of the driver; determining whether the driver is in a critical illness state according to the running state information and the vital sign information; and if not, determining a driving state score of the driver according to the limb stiffness score and the posture stability score. The method can effectively prevent traffic accidents.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicles, in particular to a driver state detection method and device and electronic equipment. BACKGROUND

[0002] The existing commercial field vehicles cannot timely distinguish and take measures when the driver state is abnormal. However, when driving a motor vehicle, the driver may be fatigued or have a sudden illness, which may easily cause a traffic accident.

[0003] Therefore, the problem of driver state detection needs to be solved. SUMMARY

[0004] The present application aims to provide a driver state detection method and device and electronic equipment to alleviate the technical problem that when driving a motor vehicle, the driver may be fatigued or have a sudden illness, which may easily cause a traffic accident, and to improve driving safety.

[0005] In a first aspect, the present application provides a driver state detection method, comprising: acquiring image information of a driver through a vehicle-mounted image acquisition device; acquiring physiological information of the driver through a wearable device; and acquiring running state information of a running vehicle through a vehicle-mounted sensor; preprocessing the image information to obtain a preprocessed image; determining facial features, skin color features, and human feature point position information of the driver according to the preprocessed image; determining vital sign information of the driver according to the skin color features and the physiological information; inputting the facial features and the human feature point position information into a pre-trained machine learning model to output a limb rigidity score and a posture stability score of the driver; the machine learning model is pre-trained based on an image carrying a preset limb rigidity score and a preset posture stability score; determining whether the driver is in a critical illness state according to the running state information and the vital sign information; if not, determining a driving state score of the driver according to the limb rigidity score and the posture stability score.

[0006] In a preferred embodiment of the present application, after the step of determining the driving state score of the driver according to the limb rigidity score and the posture stability score, the method comprises: calculating the size relationship between the driving state score and a preset score threshold; when the driving state score is less than the score threshold, generating a warning information and controlling the running vehicle to stop within a preset lane line and turning on the double flash of the running vehicle; and sending the warning information and the running state information to a designated terminal device.

[0007] In the preferred embodiment of the present application, the specified terminal is a TSP platform; after the step of sending the warning information and the running state information to the specified terminal device, the method further comprises: sending the warning information and the running state information to a preset medical system through the TSP platform.

[0008] In the preferred embodiment of the present application, after the step of determining the driving state score of the driver, the method comprises: calculating the size relationship between the driving state score and a preset abnormal score range; when the driving state score is greater than the minimum value of the abnormal score range and less than the maximum value of the abnormal score range, determining the driving state score as a driver abnormal score and recording the current duration of the driver abnormal score; when the current duration is greater than or equal to a preset duration, determining whether the running vehicle is in an assisted running state; if the running vehicle is in the assisted running state, stopping calling a preset assisted running system and controlling the running vehicle to play a preset prompt information.

[0009] In the preferred embodiment of the present application, after the step of stopping calling the preset assisted running system and controlling the running vehicle to play the preset prompt information if the running vehicle is in the assisted running state, the method further comprises: sending the driver abnormal score to the TSP platform and, through the TSP platform, counting a total duration of the driver abnormal score according to the current duration of the driver abnormal score; when the total duration is greater than a preset abnormal threshold, generating the warning information and controlling the running vehicle to stop in the lane line and turn on the double flash of the running vehicle; sending the warning information and the running state information to the TSP platform; and sending the warning information and the running state information to a preset medical system through the TSP platform.

[0010] In the preferred embodiment of the present application, the machine learning model comprises: an upper limb stiffness score model, a neck stiffness score model, and a posture stability score model; the step of inputting the facial features and the human body feature point position information into the pre-trained machine learning model to output the limb stiffness score and the posture stability score comprises: inputting the human body feature point position information into the upper limb stiffness score model to output an upper limb stiffness score; inputting the facial features and the human body feature point position information into the neck stiffness score model to output a neck stiffness score; inputting the human body feature point position information into the posture stability score model to output a posture stability score; and determining the limb stiffness score of the driver according to the upper limb stiffness score and the neck stiffness score and outputting the limb stiffness score of the driver.

[0011] In the preferred embodiment of the present application, the running state information includes acceleration, speed, driving direction and driving environment temperature and humidity information of the vehicle; and the step of determining whether the driver is in a critical illness state according to the running state information and the vital sign information includes: determining whether the vehicle is in a variable speed driving state within a preset parameter range according to the acceleration, speed and driving direction; and determining whether the driver is in a critical illness state according to the vital sign information, the driving environment temperature and humidity information and the variable speed driving state.

[0012] In the preferred embodiment of the present application, the variable speed driving state includes an acceleration driving state and a deceleration driving state; and the step of determining whether the driver is in a critical illness state according to the vital sign information, the driving environment temperature and humidity information and the variable speed driving state includes: determining whether the driver is in a critical illness state according to the vital sign information, the driving environment temperature and humidity information, the acceleration driving state and the deceleration driving state.

[0013] In a second aspect, the embodiments of the present application further provide a driver state detection device, which comprises: a data acquisition module, configured to acquire image information of a driver through a vehicle-mounted image acquisition device, and acquire physiological information of the driver through a wearable device, and acquire running state information of a vehicle through a vehicle-mounted sensor; a preprocessing module, configured to preprocess the image information to obtain a preprocessed image; a feature extraction module, configured to determine facial features, skin color features and human feature point position information of the driver according to the preprocessed image; an identification module, configured to determine vital sign information of the driver according to the skin color features and the physiological information, and input the facial features and the human feature point position information into a pre-trained machine learning model to output a limb rigidity score and a posture stability score of the driver; the machine learning model is obtained by being trained in advance based on an image carrying a preset limb rigidity score and a preset posture stability score; and a driving state determination block, configured to determine whether the driver is in a critical illness state according to the running state information and the vital sign information, and determine a driving state score of the driver according to the limb rigidity score and the posture stability score if the driver is not in a critical illness state.

[0014] In a third aspect, the embodiments of the present application further provide an electronic device, which comprises a processor and a memory, the memory stores computer executable instructions capable of being executed by the processor, and the processor executes the computer executable instructions to implement the method.

[0015] The embodiments of the present application have the following beneficial technical effects:

[0016] This invention provides a driver state detection method, apparatus, and electronic device, comprising: acquiring image information of the driver through an in-vehicle image acquisition device; acquiring physiological information of the driver through a wearable device; and acquiring the operating status information of the vehicle through an in-vehicle sensor; preprocessing the image information to obtain a preprocessed image; determining the driver's facial features, skin color features, and human feature point location information based on the preprocessed image; determining the driver's vital signs based on the skin color features and physiological information; inputting the facial features and human feature point location information into a pre-trained machine learning model to output the driver's limb stiffness score and posture stability score; the machine learning model is pre-trained based on images carrying preset limb stiffness scores and preset posture stability scores; determining whether the driver is in a critical illness state based on the operating status information and vital sign information; if not, determining the driver's driving state score based on the limb stiffness score and posture stability score. This method, by fusing multi-source information such as images, physiological data, vehicle operating status, and machine learning technology, can accurately assess the driver's health status and driving state, thereby effectively preventing traffic accidents. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a driver state detection method provided in an embodiment of the present invention.

[0019] Figure 2 A flowchart illustrating another driver state detection method provided in an embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of a driver status detection device provided in an embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0022] Icons: 31-Data acquisition module; 32-Preprocessing module; 33-Feature extraction module; 34-Recognition module; 35-Driving state determination block; 41-Memory; 42-Processor; 43-Bus; 44-Communication interface. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0024] Existing commercial vehicles are unable to promptly identify and respond to abnormal driver conditions. However, driver fatigue and sudden illness can easily lead to traffic accidents. Therefore, driver condition detection is a pressing issue that needs to be addressed.

[0025] Based on this, embodiments of the present invention provide a driver state detection method, device, and electronic device. This method, through the fusion of multi-source information—images, physiological data, vehicle operating status, and machine learning techniques—can accurately assess the driver's health and driving state, thereby effectively preventing traffic accidents. For ease of understanding, a driver state detection method is first introduced.

[0026] Example 1

[0027] In this embodiment, Figure 1 This is a flowchart illustrating a driver state detection method provided in an embodiment of the present invention.

[0028] Depend on Figure 1 As seen, the method includes:

[0029] Step S101: Acquire the driver's image information through the vehicle-mounted image acquisition device; acquire the driver's physiological information through the wearable device; and acquire the vehicle's operating status information through the vehicle-mounted sensors.

[0030] In this embodiment, the vehicle-mounted image acquisition device includes a DMS camera, an OMS camera, and a panoramic camera; the vehicle-mounted sensors include a forward-facing 4D millimeter-wave radar, an angle radar, and a monocular integrated camera.

[0031] Step S102: Preprocess the above image information to obtain a preprocessed image.

[0032] In this embodiment, step S102 includes: performing contrast enhancement, noise reduction, and size adjustment on the above image information to obtain a preprocessed image.

[0033] Step S103: Based on the preprocessed image above, determine the driver's facial features, skin color features, and the location information of human feature points.

[0034] Step S104: Based on the skin color features and physiological information, determine the driver's vital signs; and input the facial features and human body feature point location information into a pre-trained machine learning model to output the driver's limb stiffness score and posture stability score; the machine learning model is pre-trained based on images carrying preset limb stiffness scores and preset posture stability scores.

[0035] Step S105: Based on the above operating status information and the above vital signs information, determine whether the above driver is in a critical illness state; if not, determine the above driver's driving status score based on the above limb stiffness score and posture stability score.

[0036] In this embodiment, when it is determined that the driver is in a critical illness state, a warning message is generated and the vehicle is controlled to stop within a preset lane line, and the hazard lights of the vehicle are turned on; the warning message and the operating status information are sent to a preset medical system.

[0037] This invention provides a driver state detection method, comprising: acquiring driver image information through an in-vehicle image acquisition device; acquiring the driver's physiological information through a wearable device; and acquiring the vehicle's operating status information through an in-vehicle sensor; preprocessing the image information to obtain a preprocessed image; determining the driver's facial features, skin color features, and human feature point location information based on the preprocessed image; determining the driver's vital signs based on the skin color features and physiological information; inputting the facial features and human feature point location information into a pre-trained machine learning model to output the driver's limb stiffness score and posture stability score; the machine learning model is pre-trained based on images carrying preset limb stiffness scores and preset posture stability scores; determining whether the driver is in a critical illness state based on the operating status information and vital sign information; if not, determining the driver's driving state score based on the limb stiffness score and posture stability score. This method, by fusing multi-source information such as images, physiological data, vehicle operating status, and machine learning technology, can accurately assess the driver's health status and driving state, thereby effectively preventing traffic accidents.

[0038] Example 2

[0039] In this embodiment, Figure 2 This is a flowchart illustrating another driver state detection method provided in an embodiment of the present invention.

[0040] Depend on Figure 2As seen, the method includes:

[0041] Step S201: Acquire the driver's image information through the vehicle-mounted image acquisition device; acquire the driver's physiological information through a wearable device; and acquire the vehicle's operating status information through vehicle-mounted sensors.

[0042] Step S202: Preprocess the above image information to obtain a preprocessed image.

[0043] Step S203: Based on the preprocessed image above, determine the driver's facial features, skin color features, and the location information of human feature points.

[0044] Step S204: Based on the skin color features and physiological information, determine the driver's vital signs; and input the facial features and human body feature point location information into a pre-trained machine learning model to output the driver's limb stiffness score and posture stability score; the machine learning model is pre-trained based on images carrying preset limb stiffness scores and preset posture stability scores.

[0045] In practice, assuming a driver experiences a myocardial infarction, their skin color will turn gray and their lips will turn bluish. Since lighting and makeup can affect skin color, the driver's vital signs are determined by combining this physiological information with the aforementioned physical characteristics.

[0046] In practice, the driver's vital signs are first assessed based on the skin color characteristics described above. If so, the driver's vital signs are determined by combining the physiological information described above.

[0047] In this embodiment, the above machine learning model is built based on LightGBM.

[0048] Furthermore, prior to step S204, the method includes: inputting the facial features and the location information of the human body feature points into a preset initial machine learning model and outputting information gain; sorting the information gains to obtain sorted information gains; grouping the sorted information gains according to preset parameters to obtain multiple feature groups; training the initial machine learning model with the target facial features and the location information of the target human body feature points corresponding to the multiple feature groups respectively to obtain multiple trained initial machine learning models; evaluating the performance of the multiple trained initial machine learning models using leave-one-out cross-validation to obtain multiple performance values; and determining the initial machine learning model whose performance value reaches a preset threshold as the pre-trained machine learning model.

[0049] Step S205: Based on the above operating status information and the above vital signs information, determine whether the above driver is in a critical illness state; if not, determine the above driver's driving status score based on the above limb rigidity score and posture stability score.

[0050] Step S206: Calculate the relationship between the above driving status score and the preset score threshold.

[0051] Step S207: When the driving status score is less than the score threshold, generate a warning message and control the vehicle to stop within the preset lane line, and turn on the vehicle's hazard lights.

[0052] Step S208: Send the above warning information and the above operating status information to the designated terminal device.

[0053] In this embodiment, the designated terminal is a TSP platform; after the step of sending the warning information and the operating status information to the designated terminal device, the method further includes: sending the warning information and the operating status information to a preset medical system through the TSP platform.

[0054] Furthermore, after determining the driver's driving status score, the method includes: calculating the relationship between the driving status score and a preset abnormal score range; when the driving status score is greater than the minimum value of the abnormal score range and less than the maximum value of the abnormal score range, determining the driving status score as an abnormal driver score and recording the current duration of the abnormal driver score; when the current duration is greater than or equal to a preset duration, determining whether the vehicle is in an assisted driving state; if the vehicle is in an assisted driving state, stopping the invocation of the preset assisted driving system and controlling the vehicle to broadcast preset prompt information.

[0055] Furthermore, if the vehicle is in an assisted driving state, after stopping the invocation of the preset assisted driving system and controlling the vehicle to broadcast preset prompt information, the method further includes: sending the driver abnormality score to the TSP platform, and calculating the total duration of the driver abnormality score based on the current duration of the driver abnormality score through the TSP platform; when the total duration exceeds a preset abnormality threshold, generating the warning information and controlling the vehicle to stop within the lane lines, and activating the vehicle's hazard lights; sending the warning information and the operating status information to the TSP platform; and sending the warning information and the operating status information to a preset medical system through the TSP platform.

[0056] In one embodiment, the machine learning model includes: an upper limb stiffness scoring model, a neck stiffness scoring model, and a posture stability scoring model; the step of inputting the facial features and the location information of the human feature points into the pre-trained machine learning model and outputting the limb stiffness score and posture stability score includes: inputting the location information of the human feature points into the upper limb stiffness scoring model and outputting an upper limb stiffness score; inputting the facial features and the location information of the human feature points into the neck stiffness scoring model and outputting a neck stiffness score; inputting the location information of the human feature points into the posture stability scoring model and outputting a posture stability score; determining the driver's limb stiffness score based on the upper limb stiffness score and the neck stiffness score, and outputting the driver's limb stiffness score.

[0057] In this embodiment, the aforementioned operational status information includes: the acceleration, speed, driving direction, and ambient temperature and humidity information of the vehicle; the step of determining whether the driver is in a critical illness state based on the aforementioned operational status information and the aforementioned vital signs information includes: determining whether the vehicle is in a variable speed driving state within a preset parameter range based on the aforementioned acceleration, speed, and driving direction; and determining whether the driver is in a critical illness state based on the aforementioned vital signs information, ambient temperature and humidity information, and variable speed driving state.

[0058] Furthermore, the aforementioned variable speed driving state includes: acceleration driving state and deceleration driving state; the step of determining whether the driver is in a critical illness state based on the aforementioned vital signs information, the aforementioned driving environment temperature and humidity information, and the aforementioned variable speed driving state includes: determining whether the driver is in a critical illness state based on the aforementioned vital signs information, the aforementioned driving environment temperature and humidity information, the aforementioned acceleration driving state, and the aforementioned deceleration driving state.

[0059] This invention provides a driver state detection method, comprising: acquiring image information of the driver through an in-vehicle image acquisition device; acquiring physiological information of the driver through a wearable device; acquiring operating status information of the vehicle through an in-vehicle sensor; preprocessing the image information to obtain a preprocessed image; determining the driver's facial features, skin color features, and human feature point location information based on the preprocessed image; determining the driver's vital signs based on the skin color features and physiological information; and inputting the facial features and human feature point location information into a pre-trained machine learning model to output the driver's limb stiffness. The method involves a multi-source information fusion model, which integrates images, physiological data, vehicle operating status data, and machine learning techniques to accurately assess a driver's health and driving status. When the driving status score is low, appropriate safety procedures are performed to further prevent traffic accidents. The model is pre-trained on images with preset limb stiffness and postural stability scores. Based on the operational status information and vital signs, it determines whether the driver is in a critical illness state. If not, it determines the driver's driving status score based on the limb stiffness and postural stability scores. The relationship between the driving status score and a preset scoring threshold is calculated. If the driving status score is lower than the threshold, a warning message is generated, and the vehicle is stopped within a preset lane, with the hazard lights activated. The warning message and operational status information are then sent to a designated terminal device.

[0060] Example 3

[0061] Based on the above embodiments, Figure 3 This is a schematic diagram of a driver status detection device provided in an embodiment of the present invention.

[0062] Depend on Figure 3 As seen, the device includes:

[0063] The data acquisition module 31 is used to acquire the driver's image information through the vehicle-mounted image acquisition device; and to acquire the driver's physiological information through the wearable device; and to acquire the operating status information of the vehicle through the vehicle-mounted sensors.

[0064] The preprocessing module 32 is used to preprocess the above image information to obtain a preprocessed image.

[0065] The feature extraction module 33 is used to determine the driver's facial features, skin color features, and human feature point location information based on the preprocessed image.

[0066] The recognition module 34 is used to determine the driver's vital signs information based on the skin color features and physiological information; and input the facial features and the human body feature point location information into a pre-trained machine learning model to output the driver's limb stiffness score and posture stability score; the machine learning model is pre-trained based on images carrying preset limb stiffness scores and preset posture stability scores.

[0067] The driving status determination block 35 is used to determine whether the driver is in a critical illness state based on the above-mentioned operating status information and vital sign information; if not, it determines the driver's driving status score based on the above-mentioned limb stiffness score and posture stability score.

[0068] The data acquisition module 31, preprocessing module 32, feature extraction module 33, recognition module 34, and driving state determination block 35 are connected in sequence.

[0069] The driver state detection device provided in this embodiment of the invention has the same technical features as the driver state detection method provided in the above embodiments, and therefore can solve the same technical problems and achieve the same technical effects. Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0070] Example 4

[0071] This embodiment provides an electronic device, including a processor and a memory. The memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the steps of a driver state detection method.

[0072] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a driver state detection method.

[0073] See Figure 4 The diagram shows the structure of an electronic device, which includes a memory 41 and a processor 42. The memory 41 stores a computer program that can run on the processor 42. When the processor executes the computer program, it implements the steps provided by the driver state detection method described above.

[0074] like Figure 4 As shown, the device also includes a bus 43 and a communication interface 44, with the processor 42, the communication interface 44 and the memory 41 connected via the bus 43; the processor 42 is used to execute executable modules, such as computer programs, stored in the memory 41.

[0075] The memory 41 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 44 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0076] Bus 43 can be an ISA bus, PCI bus, or EISA bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0077] The memory 41 stores the program, and the processor 42 executes the program after receiving the execution instruction. The method executed by the driver state detection device disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 42, or implemented by the processor 42. The processor 42 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor 42 or by instructions in the form of software. The processor 42 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this invention can be directly manifested as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 41, and processor 42 reads information from memory 41 and, in conjunction with its hardware, completes the steps of the above method.

[0078] Furthermore, this embodiment of the invention also provides a machine-readable storage medium storing machine-executable instructions. When the machine-executable instructions are invoked and executed by the processor 42, the machine-executable instructions cause the processor 42 to implement the above-described driver state detection method.

[0079] The electronic devices and computer-readable storage media provided in the embodiments of the present invention have the same technical features, so they can also solve the same technical problems and achieve the same technical effects.

[0080] Furthermore, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0081] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

Claims

1. A driver state detection method, characterized in that, include: Acquire driver image information using in-vehicle image acquisition equipment; In addition, the driver's physiological information is obtained through wearable devices; In addition, the vehicle's operating status information is obtained through onboard sensors; The image information is preprocessed to obtain a preprocessed image; Based on the preprocessed image, the driver's facial features, skin color features, and the location information of human feature points are determined; Based on the skin color characteristics and the physiological information, the driver's vital signs information are determined; Furthermore, the facial features and the location information of the human body feature points are input into a pre-trained machine learning model, which outputs the driver's limb stiffness score and posture stability score; the machine learning model is pre-trained based on images carrying preset limb stiffness scores and preset posture stability scores. Based on the operational status information and the vital signs information, determine whether the driver is in a critical illness state; If not, determine the driver's driving status score based on the limb stiffness score and posture stability score; Calculate the relationship between the driving status score and the preset abnormal score range; When the driving status score is greater than the minimum value of the abnormal score range and less than the maximum value of the abnormal score range, the driving status score is determined as an abnormal driver score, and the current duration of the abnormal driver score is recorded. When the current duration is greater than or equal to a preset duration, it is determined whether the vehicle is in an assisted driving state; If the vehicle is in assisted driving mode, the preset assisted driving system will be stopped, and the vehicle will be controlled to broadcast preset prompts.

2. The driver state detection method according to claim 1, characterized in that, After determining the driver's driving status score based on the limb stiffness score and postural stability score, the method includes: Calculate the relationship between the driving status score and a preset score threshold; When the driving status score is less than the score threshold, a warning message is generated and the vehicle is controlled to stop within the preset lane line, and the vehicle's hazard lights are turned on. The warning information and the operating status information are sent to the designated terminal device.

3. The driver state detection method according to claim 2, characterized in that, The designated terminal is the TSP platform; After the step of sending the warning information and the operating status information to the designated terminal device, the method further includes: The TSP platform sends the warning information and the operational status information to a preset medical system.

4. The driver state detection method according to claim 3, characterized in that, If the vehicle is in an assisted driving state, after stopping the invocation of the preset assisted driving system and controlling the vehicle to broadcast preset prompt information, the method further includes: The driver abnormality score is sent to the TSP platform, and the TSP platform calculates the total duration of the driver abnormality score based on the current duration of the driver abnormality score. When the total duration exceeds a preset abnormal threshold, the warning information is generated and the vehicle is controlled to stop within the lane line, and the hazard lights of the vehicle are turned on. The warning information and the operating status information are sent to the TSP platform; The TSP platform sends the warning information and the operational status information to a preset medical system.

5. The driver state detection method according to claim 1, characterized in that, The machine learning model includes: an upper limb stiffness scoring model, a neck stiffness scoring model, and a posture stability scoring model; the steps of inputting the facial features and the location information of the human body feature points into the pre-trained machine learning model and outputting the limb stiffness score and posture stability score include: The human body feature point location information is input into the upper limb stiffness scoring model, and the upper limb stiffness score is output. The facial features and the location information of the human body feature points are input into the neck stiffness scoring model, and the neck stiffness score is output. The human body feature point location information is input into the posture stability scoring model, and the posture stability score is output. Based on the upper limb stiffness score and the neck stiffness score, the driver's limb stiffness score is determined and output.

6. The driver state detection method according to claim 1, characterized in that, The operating status information includes: the acceleration, speed, driving direction of the vehicle, and the temperature and humidity information of the driving environment; The step of determining whether the driver is in a critical illness state based on the operational status information and the vital signs information includes: Based on the acceleration, the speed, and the direction of travel, determine whether the vehicle is in a variable speed driving state within a preset parameter range; Based on the vital signs information, the temperature and humidity information of the driving environment, and the speed change driving status, it is determined whether the driver is in a critical illness state.

7. The driver state detection method according to claim 6, characterized in that, The variable speed driving state includes: acceleration driving state and deceleration driving state; the step of determining whether the driver is in a critical illness state based on the vital signs information, the temperature and humidity information of the driving environment, and the variable speed driving state includes: Based on the vital signs information, the temperature and humidity information of the driving environment, the acceleration driving state, and the deceleration driving state, it is determined whether the driver is in a critical illness state.

8. A driver status detection device, characterized in that, include: The data acquisition module is used to acquire image information of the driver through the vehicle-mounted image acquisition device; In addition, the driver's physiological information is obtained through wearable devices; In addition, the vehicle's operating status information is obtained through onboard sensors; The preprocessing module is used to preprocess the image information to obtain a preprocessed image; The feature extraction module is used to determine the driver's facial features, skin color features, and human feature point location information based on the preprocessed image. The identification module is used to determine the driver's vital signs information based on the skin color characteristics and the physiological information. Furthermore, the facial features and the location information of the human body feature points are input into a pre-trained machine learning model, which outputs the driver's limb stiffness score and posture stability score; the machine learning model is pre-trained based on images carrying preset limb stiffness scores and preset posture stability scores. The driving status determination block is used to determine whether the driver is in a critical illness state based on the operating status information and the vital signs information; if not, the driving status score of the driver is determined based on the limb rigidity score and the posture stability score. The driving state determination block is also used for: Calculate the relationship between the driving status score and the preset abnormal score range; When the driving status score is greater than the minimum value of the abnormal score range and less than the maximum value of the abnormal score range, the driving status score is determined as an abnormal driver score, and the current duration of the abnormal driver score is recorded. When the current duration is greater than or equal to a preset duration, it is determined whether the vehicle is in an assisted driving state; If the vehicle is in assisted driving mode, the preset assisted driving system will be stopped, and the vehicle will be controlled to broadcast preset prompts.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the method of any one of claims 1 to 7.

Images