Driver state detection method and device and electronic equipment

Abstract

The invention provides a driver state detection method and device and electronic equipment. The method comprises the steps that image information and physiological information of a driver and running state information of a running vehicle are acquired; preprocessing the image information to obtain a preprocessed image; determining facial features, skin color features and human body feature point position information of the driver according to the preprocessed image; determining vital sign information of the driver according to the skin color features and the physiological information; inputting the position information of the facial features and the human body feature points into a machine learning model, and outputting a limb stiffness score and a posture stability score of the driver; determining whether the driver is in a critical disease state or not according to the running state information and the vital sign information; if not, the driving state score of the driver is determined according to the limb stiffness score and the posture stability score. The method can effectively prevent traffic accidents.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to a driver status detection method, device and electronic equipment. Background Art

[0002] Existing commercial vehicles are unable to promptly identify and respond to abnormal driver conditions. However, when driving a motor vehicle, drivers may experience fatigue driving or sudden illness, which can easily lead to traffic accidents.

[0003] Therefore, the problem of driver status detection needs to be solved urgently. Summary of the invention

[0004] The purpose of the present invention is to provide a driver status detection method, device and electronic equipment to alleviate the technical problems of traffic accidents that are prone to occur when the driver is driving a motor vehicle due to fatigue driving and sudden illness of the driver, thereby improving driving safety.

[0005] In a first aspect, an embodiment of the present invention provides a driver status detection method, comprising: obtaining image information of a driver through an on-board image acquisition device; and, obtaining physiological information of the driver through a wearable device; and, obtaining operating status information of a moving vehicle through an on-board sensor; preprocessing the image information to obtain a preprocessed image; determining facial features, skin color features, and position information of human feature points of the driver based on the preprocessed image; determining vital sign information of the driver based on the skin color features and the physiological information; and, inputting the facial features and the position information of the human feature points into a pre-trained machine learning model, and outputting a limb stiffness score and a posture stability score of the driver; the machine learning model is pre-trained based on an image carrying a preset limb stiffness score and a preset posture stability score; determining whether the driver is in a critical illness state based on the operating status information and the vital sign information; if not, determining the driving status score of the driver based on the limb stiffness score and the posture stability score.

[0006] In a preferred embodiment of the present invention, after the step of determining the driving status score of the driver according to the limb stiffness score and the posture stability score, the method includes: calculating the relationship between the driving status score and a preset score threshold; when the driving status score is less than the score threshold, generating a warning message and controlling the moving vehicle to stop within a preset lane line, and turning on the double flash of the moving vehicle; and sending the warning message and the operating status information to a designated terminal device.

[0007] In a preferred embodiment of the present invention, the above-mentioned designated terminal is a TSP platform; after the step of sending the above-mentioned warning information and the above-mentioned operating status information to the designated terminal device, the above-mentioned method also includes: sending the above-mentioned warning information and the above-mentioned operating status information to a preset medical system through the above-mentioned TSP platform.

[0008] In a preferred embodiment of the present invention, after the step of determining the driving status score of the above-mentioned driver, the above-mentioned method includes: calculating the size relationship between the above-mentioned driving status score and a preset abnormal score range; when the above-mentioned driving status score is greater than the minimum value of the above-mentioned abnormal score range and less than the maximum value of the above-mentioned abnormal score range, determining the above-mentioned driving status score as the driver's abnormal score, and recording the current duration of the above-mentioned driver's abnormal score; when the above-mentioned current duration is greater than or equal to the preset duration, judging whether the above-mentioned driving vehicle is in an assisted driving state; if the above-mentioned driving vehicle is in an assisted driving state, stopping calling the preset assisted driving system, and controlling the above-mentioned driving vehicle to broadcast a preset prompt message.

[0009] In a preferred embodiment of the present invention, if the above-mentioned driving vehicle is in an assisted driving state, then after the step of stopping calling the preset assisted driving system and controlling the above-mentioned driving vehicle to broadcast a preset prompt message, the above-mentioned method further includes: sending the above-mentioned driver abnormality score to the above-mentioned TSP platform, and through the above-mentioned TSP platform, according to the current duration of the above-mentioned driver abnormality score, counting the total duration of the above-mentioned driver abnormality score; when the above-mentioned total duration is greater than the preset abnormal threshold, generating the above-mentioned warning information and controlling the above-mentioned driving vehicle to stop within the above-mentioned lane line, and turning on the double flash of the above-mentioned driving vehicle; sending the above-mentioned warning information and the above-mentioned operating status information to the above-mentioned TSP platform; sending the above-mentioned warning information and the above-mentioned operating status information to the preset medical system through the above-mentioned TSP platform.

[0010] In a preferred embodiment of the present invention, the above-mentioned machine learning model includes: an upper limb stiffness scoring model, a neck stiffness scoring model and a posture stability scoring model; the steps of inputting the above-mentioned facial features and the above-mentioned human body feature point position information into a pre-trained machine learning model and outputting the above-mentioned limb stiffness score and posture stability score include: inputting the above-mentioned human body feature point position information into the above-mentioned upper limb stiffness scoring model, and outputting the upper limb stiffness score; inputting the above-mentioned facial features and the above-mentioned human body feature point position information into the above-mentioned neck stiffness scoring model, and outputting the neck stiffness score; inputting the above-mentioned human body feature point position information into the above-mentioned posture stability scoring model, and outputting the posture stability score; determining the above-mentioned limb stiffness score of the above-mentioned driver based on the above-mentioned upper limb stiffness score and the above-mentioned neck stiffness score, and outputting the above-mentioned driver's limb stiffness score.

[0011] In a preferred embodiment of the present invention, the above-mentioned operating status information includes: the acceleration, speed, driving direction of the above-mentioned driving vehicle and the temperature and humidity information of the driving environment; the step of determining whether the above-mentioned driver is in a critical illness state based on the above-mentioned operating status information and the above-mentioned vital signs information includes: determining whether the above-mentioned driving vehicle is in a speed-changing driving state within a preset parameter range based on the above-mentioned acceleration, the above-mentioned speed and the above-mentioned driving direction; determining whether the above-mentioned driver is in a critical illness state based on the above-mentioned vital signs information, the temperature and humidity information of the driving environment and the above-mentioned speed-changing driving state.

[0012] In a preferred embodiment of the present invention, the above-mentioned speed-changing driving state includes: an accelerating driving state and a decelerating driving state; the step of determining whether the above-mentioned driver is in a critical illness state based on the above-mentioned vital signs information, the above-mentioned driving environment temperature and humidity information, and the above-mentioned speed-changing driving state includes: determining whether the above-mentioned driver is in a critical illness state based on the above-mentioned vital signs information, the above-mentioned driving environment temperature and humidity information, the above-mentioned accelerating driving state, and the above-mentioned decelerating driving state.

[0013] In a second aspect, an embodiment of the present invention further provides a driver status detection device, comprising: a data acquisition module, used to acquire the image information of the driver through an on-board image acquisition device; and, to acquire the physiological information of the driver through a wearable device; and, to acquire the running status information of the moving vehicle through an on-board sensor; a preprocessing module, used to preprocess the image information to obtain a preprocessed image; a feature extraction module, used to determine the facial features, skin color features and human feature point position information of the driver according to the preprocessed image; an identification module, used to determine the vital sign information of the driver according to the skin color features and the physiological information; and, input the facial features and the human feature point position information into a pre-trained machine learning model, and output the limb stiffness score and posture stability score of the driver; the machine learning model is pre-trained based on an image carrying a preset limb stiffness score and a preset posture stability score; a driving status determination block, used to determine whether the driver is in a critical disease state according to the running status information and the vital sign information; if not, determine the driving status score of the driver according to the limb stiffness score and the posture stability score.

[0014] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the method.

[0015] The embodiments of the present invention have the following beneficial technical effects:

[0016] The embodiment of the present invention provides a driver status detection method, device and electronic device, including: obtaining the driver's image information through a vehicle-mounted image acquisition device; and obtaining the driver's physiological information through a wearable device; and obtaining the running status information of the driving vehicle through a vehicle-mounted sensor; preprocessing the image information to obtain a preprocessed image; determining the driver's facial features, skin color features and human feature point position information according to the preprocessed image; determining the driver's vital sign information according to the skin color features and the physiological information; and inputting the facial features and the human feature point position information into a pre-trained machine learning model to output the driver's limb stiffness score and posture stability score; the machine learning model is pre-trained based on an image with a preset limb stiffness score and a preset posture stability score; determining whether the driver is in a critical disease state according to the running status information and the vital sign information; if not, determining the driver's driving status score according to the limb stiffness score and the posture stability score. The method can accurately assess the driver's health status and driving status by fusing multi-source information images, physiology, vehicle running status and machine learning technology, thereby effectively preventing traffic accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A flowchart of a driver status detection method provided by an embodiment of the present invention;

[0019] Figure 2 A schematic flow chart of another driver status detection method provided by an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of the structure of a driver status detection device provided by an embodiment of the present invention;

[0021] Figure 4 A schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0022] Icons: 31 - data acquisition module; 32 - preprocessing module; 33 - feature extraction module; 34 - recognition module; 35 - driving state determination block; 41 - memory; 42 - processor; 43 - bus; 44 - communication interface. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0024] Existing commercial vehicles are unable to identify and respond to abnormal driver status in a timely manner. However, when driving a motor vehicle, drivers may experience fatigue driving or sudden illness, which can easily lead to traffic accidents. Therefore, the problem of driver status detection needs to be solved urgently.

[0025] Based on this, an embodiment of the present invention provides a driver status detection method, device and electronic device, which can accurately assess the driver's health status and driving status through multi-source information fusion images, physiology, vehicle operation status and machine learning technology, thereby effectively preventing traffic accidents. For ease of understanding, a driver status detection method is first introduced.

[0026] Example 1

[0027] In this embodiment, Figure 1 A flowchart of a driver status detection method provided by an embodiment of the present invention.

[0028] Depend on Figure 1 As can be seen, the method includes:

[0029] Step S101: acquiring the image information of the driver through the vehicle-mounted image acquisition device; and acquiring the physiological information of the driver through the wearable device; and acquiring the running status information of the moving vehicle through the vehicle-mounted sensor.

[0030] In this embodiment, the above-mentioned vehicle-mounted image acquisition equipment includes a DMS camera, an OMS camera, and a panoramic camera; the above-mentioned vehicle-mounted sensors include: a forward 4D millimeter-wave radar, a corner radar, and a monocular integrated machine.

[0031] Step S102: pre-process the above image information to obtain a pre-processed image.

[0032] In this embodiment, step S102 includes: performing contrast enhancement, denoising and size adjustment on the above image information to obtain a pre-processed image.

[0033] Step S103: Determine the facial features, skin color features and body feature point position information of the driver according to the preprocessed image.

[0034] Step S104: Determine the vital sign information of the driver based on the skin color features and the physiological information; and input the facial features and the position information of the body feature points into a pre-trained machine learning model to output the driver's limb stiffness score and posture stability score; the machine learning model is pre-trained based on images carrying preset limb stiffness scores and preset posture stability scores.

[0035] Step S105: Determine whether the driver is in a critical illness state based on the operating status information and the vital sign information; if not, determine the driver's driving status score based on the limb stiffness score and posture stability score.

[0036] In this embodiment, when it is determined that the driver is in a critical illness state, a warning message is generated and the moving vehicle is controlled to stop within a preset lane line, and the double flashes of the moving vehicle are turned on; the warning message and the operating status information are sent to a preset medical system.

[0037] The embodiment of the present invention provides a driver status detection method, comprising: obtaining the driver's image information through a vehicle-mounted image acquisition device; and obtaining the driver's physiological information through a wearable device; and obtaining the running status information of the driving vehicle through a vehicle-mounted sensor; preprocessing the image information to obtain a preprocessed image; determining the driver's facial features, skin color features and human feature point position information according to the preprocessed image; determining the driver's vital sign information according to the skin color features and the physiological information; and inputting the facial features and the human feature point position information into a pre-trained machine learning model to output the driver's limb stiffness score and posture stability score; the machine learning model is pre-trained based on an image with a preset limb stiffness score and a preset posture stability score; determining whether the driver is in a critical disease state according to the running status information and the vital sign information; if not, determining the driver's driving status score according to the limb stiffness score and the posture stability score. The method can accurately assess the driver's health status and driving status by fusing multi-source information images, physiology, vehicle running status and machine learning technology, thereby effectively preventing traffic accidents.

[0038] Example 2

[0039] In this embodiment, Figure 2 A flowchart of another driver status detection method provided by an embodiment of the present invention.

[0040] Depend on Figure 2As can be seen, the method includes:

[0041] Step S201: acquiring the image information of the driver through the vehicle-mounted image acquisition device; and acquiring the physiological information of the driver through the wearable device; and acquiring the running status information of the moving vehicle through the vehicle-mounted sensor.

[0042] Step S202: pre-process the above image information to obtain a pre-processed image.

[0043] Step S203: Determine the facial features, skin color features and body feature point position information of the driver according to the preprocessed image.

[0044] Step S204: Determine the vital sign information of the driver based on the skin color features and the physiological information; and input the facial features and the position information of the body feature points into a pre-trained machine learning model to output the driver's limb stiffness score and posture stability score; the machine learning model is pre-trained based on images carrying preset limb stiffness scores and preset posture stability scores.

[0045] In actual operation, if the driver has myocardial infarction, the driver's skin color will turn gray and the lip color will turn blue. Since light and driver makeup may affect the skin color, the above physiological information is combined to jointly determine the driver's vital sign information.

[0046] In actual operation, firstly, it is determined whether the driver's vital signs information is suspected to be affected based on the above skin color characteristics; if so, the driver's vital signs information is determined in combination with the above physiological information.

[0047] In this embodiment, the above machine learning model is constructed based on LightGBM.

[0048] Furthermore, before the above-mentioned step S204, the method includes: inputting the above-mentioned facial features and the above-mentioned human feature point position information into a preset initial machine learning model, and outputting information gain; sorting the above-mentioned information gain to obtain sorted information gain; grouping the above-mentioned sorted information gain according to preset parameters to obtain multiple feature groups; training the above-mentioned initial machine learning model respectively through the target facial features and target human feature point position information corresponding to the above-mentioned multiple feature groups to obtain multiple trained initial machine learning models; using leave-one-out cross-validation to evaluate the performance of the above-mentioned multiple trained initial machine learning models to obtain multiple performance values; and determining the initial machine learning model whose performance value reaches a preset threshold as the above-mentioned pre-trained machine learning model.

[0049] Step S205: Determine whether the driver is in a critical illness state based on the operating status information and the vital sign information; if not, determine the driver's driving status score based on the limb stiffness score and posture stability score.

[0050] Step S206: Calculate the relationship between the driving status score and a preset score threshold.

[0051] Step S207: When the driving status score is less than the score threshold, a warning message is generated, the moving vehicle is controlled to stop within a preset lane line, and the double flash of the moving vehicle is turned on.

[0052] Step S208: Send the above-mentioned warning information and the above-mentioned operation status information to the designated terminal device.

[0053] In this embodiment, the designated terminal is a TSP platform; after the step of sending the warning information and the operating status information to the designated terminal device, the method further includes: sending the warning information and the operating status information to a preset medical system through the TSP platform.

[0054] Furthermore, after the step of determining the driving status score of the above-mentioned driver, the above-mentioned method includes: calculating the size relationship between the above-mentioned driving status score and a preset abnormal score range; when the above-mentioned driving status score is greater than the minimum value of the above-mentioned abnormal score range and less than the maximum value of the above-mentioned abnormal score range, determining the above-mentioned driving status score as the driver's abnormal score, and recording the current duration of the above-mentioned driver's abnormal score; when the above-mentioned current duration is greater than or equal to the preset duration, judging whether the above-mentioned driving vehicle is in an assisted driving state; if the above-mentioned driving vehicle is in an assisted driving state, stopping calling the preset assisted driving system, and controlling the above-mentioned driving vehicle to broadcast a preset prompt message.

[0055] Furthermore, if the above-mentioned vehicle is in an assisted driving state, then after the step of stopping calling the preset assisted driving system and controlling the above-mentioned vehicle to broadcast a preset prompt message, the above-mentioned method also includes: sending the above-mentioned driver abnormality score to the above-mentioned TSP platform, and through the above-mentioned TSP platform, according to the current duration of the above-mentioned driver abnormality score, counting the total duration of the above-mentioned driver abnormality score; when the above-mentioned total duration is greater than the preset abnormal threshold, generating the above-mentioned warning information and controlling the above-mentioned vehicle to stop within the above-mentioned lane line, and turning on the double flash of the above-mentioned vehicle; sending the above-mentioned warning information and the above-mentioned operating status information to the above-mentioned TSP platform; sending the above-mentioned warning information and the above-mentioned operating status information to the preset medical system through the above-mentioned TSP platform.

[0056] In one embodiment, the machine learning model includes: an upper limb stiffness scoring model, a neck stiffness scoring model and a posture stability scoring model; the steps of inputting the facial features and the position information of the human body feature points into a pre-trained machine learning model and outputting the limb stiffness score and the posture stability score include: inputting the position information of the human body feature points into the upper limb stiffness scoring model and outputting the upper limb stiffness score; inputting the facial features and the position information of the human body feature points into the neck stiffness scoring model and outputting the neck stiffness score; inputting the position information of the human body feature points into the posture stability scoring model and outputting the posture stability score; determining the limb stiffness score of the driver based on the upper limb stiffness score and the neck stiffness score, and outputting the limb stiffness score of the driver.

[0057] In this embodiment, the above-mentioned operating status information includes: the acceleration, speed, driving direction of the above-mentioned driving vehicle and the temperature and humidity information of the driving environment; the step of determining whether the above-mentioned driver is in a critical illness state based on the above-mentioned operating status information and the above-mentioned vital signs information includes: determining whether the above-mentioned driving vehicle is in a speed-changing driving state within a preset parameter range based on the above-mentioned acceleration, the above-mentioned speed and the above-mentioned driving direction; determining whether the above-mentioned driver is in a critical illness state based on the above-mentioned vital signs information, the temperature and humidity information of the driving environment and the above-mentioned speed-changing driving state.

[0058] Furthermore, the above-mentioned speed-changing driving state includes: an accelerating driving state and a decelerating driving state; the step of determining whether the above-mentioned driver is in a critical illness state based on the above-mentioned vital signs information, the above-mentioned driving environment temperature and humidity information, and the above-mentioned speed-changing driving state includes: determining whether the above-mentioned driver is in a critical illness state based on the above-mentioned vital signs information, the above-mentioned driving environment temperature and humidity information, the above-mentioned accelerating driving state, and the above-mentioned decelerating driving state.

[0059] The embodiment of the present invention provides a driver status detection method, comprising: obtaining image information of the driver through a vehicle-mounted image acquisition device; and obtaining physiological information of the driver through a wearable device; and obtaining running status information of a moving vehicle through a vehicle-mounted sensor; preprocessing the image information to obtain a preprocessed image; determining facial features, skin color features, and position information of human body feature points of the driver based on the preprocessed image; determining vital sign information of the driver based on the skin color features and the physiological information; and inputting the facial features and the position information of the human body feature points into a pre-trained machine learning model to output the driver's limb stiffness. The above machine learning model is pre-trained based on images with preset limb stiffness scores and preset posture stability scores; according to the above operating status information and the above vital signs information, determine whether the above driver is in a critical illness state; if not, determine the above driver's driving status score according to the above limb stiffness score and posture stability score; calculate the relationship between the above driving status score and the preset score threshold; when the above driving status score is less than the above score threshold, generate warning information and control the above vehicle to stop within the preset lane line, and turn on the double flash of the above vehicle; send the above warning information and the above operating status information to the designated terminal device. This method can accurately evaluate the driver's health status and driving status score by fusing images, physiology, vehicle operating status and machine learning technology from multiple sources of information, and perform corresponding safety operations when the driving status score is low, so as to further prevent traffic accidents.

[0060] Example 3

[0061] Based on the above embodiments, Figure 3 A schematic diagram of the structure of a driver status detection device provided in an embodiment of the present invention.

[0062] Depend on Figure 3 As can be seen, the device includes:

[0063] The data acquisition module 31 is used to acquire the image information of the driver through the vehicle-mounted image acquisition device; and to acquire the physiological information of the driver through the wearable device; and to acquire the running status information of the moving vehicle through the vehicle-mounted sensor.

[0064] The preprocessing module 32 is used to preprocess the above image information to obtain a preprocessed image.

[0065] The feature extraction module 33 is used to determine the facial features, skin color features and body feature point position information of the driver according to the pre-processed image.

[0066] The recognition module 34 is used to determine the vital sign information of the driver based on the above-mentioned skin color characteristics and the above-mentioned physiological information; and input the above-mentioned facial features and the above-mentioned body feature point position information into a pre-trained machine learning model, and output the driver's limb stiffness score and posture stability score; the above-mentioned machine learning model is pre-trained based on an image carrying a preset limb stiffness score and a preset posture stability score.

[0067] The driving state determination block 35 is used to determine whether the driver is in a critical illness state based on the operating state information and the vital sign information; if not, determine the driving state score of the driver based on the limb stiffness score and posture stability score.

[0068] The data acquisition module 31, the preprocessing module 32, the feature extraction module 33, the recognition module 34 and the driving state determination module 35 are connected in sequence.

[0069] The driver status detection device provided in the embodiment of the present invention has the same technical features as the driver status detection method provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the device described above can refer to the corresponding process in the above method embodiment, and will not be repeated here.

[0070] Example 4

[0071] This embodiment provides an electronic device, including a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the steps of the driver status detection method.

[0072] This embodiment provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of the driver status detection method are implemented.

[0073] See also Figure 4 A structural schematic diagram of an electronic device is shown, which includes: a memory 41 and a processor 42. The memory 41 stores a computer program that can be run on the processor 42. When the processor executes the computer program, the steps provided by the above-mentioned driver status detection method are implemented.

[0074] like Figure 4 As shown, the device further includes: a bus 43 and a communication interface 44, a processor 42, a communication interface 44 and a memory 41 are connected via the bus 43; the processor 42 is used to execute executable modules stored in the memory 41, such as computer programs.

[0075] The memory 41 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 44 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0076] The bus 43 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0077] Among them, the memory 41 is used to store the program, and the processor 42 executes the program after receiving the execution instruction. The method performed by the driver state detection device disclosed in any embodiment of the present invention can be applied to the processor 42, or implemented by the processor 42. The processor 42 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 42. The above processor 42 can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 41, and the processor 42 reads the information in the memory 41 and completes the steps of the above method in combination with its hardware.

[0078] Furthermore, an embodiment of the present invention also provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by the processor 42, the machine-executable instructions prompt the processor 42 to implement the above-mentioned driver status detection method.

[0079] The electronic device and computer-readable storage medium provided by the embodiments of the present invention have the same technical features, and therefore can solve the same technical problems and achieve the same technical effects.

[0080] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0081] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

Claims

1. A driver status detection method, characterized in that: include: Acquire the driver's image information through the vehicle-mounted image acquisition device; and obtaining physiological information of the driver through a wearable device; and, obtaining operating status information of the moving vehicle through on-board sensors; Preprocessing the image information to obtain a preprocessed image; Determining facial features, skin color features, and position information of body feature points of the driver based on the preprocessed image; Determining the driver's vital sign information according to the skin color characteristics and the physiological information; Furthermore, the facial features and the position information of the human body feature points are input into a pre-trained machine learning model, and the driver's limb stiffness score and posture stability score are output; the machine learning model is pre-trained based on images carrying preset limb stiffness scores and preset posture stability scores; Determine whether the driver is in a critical illness state based on the operating status information and the vital sign information; if not, determine the driver's driving status score based on the limb stiffness score and the posture stability score.

2. The driver status detection method according to claim 1, characterized in that: After the step of determining the driver's driving status score according to the limb stiffness score and the posture stability score, the method comprises: Calculating the relationship between the driving status score and a preset score threshold; When the driving status score is less than the score threshold, a warning message is generated, the moving vehicle is controlled to stop within a preset lane line, and the double flash of the moving vehicle is turned on; The warning information and the operating status information are sent to a designated terminal device.

3. The driver status detection method according to claim 2, characterized in that: The designated terminal is a TSP platform; After the step of sending the warning information and the operation status information to the designated terminal device, the method further includes: The warning information and the operating status information are sent to a preset medical system through the TSP platform.

4. The driver status detection method according to claim 3, characterized in that: After the step of determining the driving status score of the driver, the method comprises: Calculating the relationship between the driving status score and a preset abnormal score range; When the driving state score is greater than the minimum value of the abnormal score range and less than the maximum value of the abnormal score range, the driving state score is determined as the driver abnormal score, and the current duration of the driver abnormal score is recorded; When the current duration is greater than or equal to a preset duration, determining whether the driving vehicle is in an assisted driving state; If the driving vehicle is in an assisted driving state, the preset assisted driving system is stopped from being called, and the driving vehicle is controlled to broadcast a preset prompt message.

5. The driver status detection method according to claim 4, characterized in that: If the driving vehicle is in the assisted driving state, after the step of stopping calling the preset assisted driving system and controlling the driving vehicle to broadcast the preset prompt information, the method further includes: Sending the driver abnormality score to the TSP platform, and calculating the total duration of the driver abnormality score according to the current duration of the driver abnormality score through the TSP platform; When the total duration is greater than a preset abnormal threshold, the warning information is generated, the vehicle is controlled to stop within the lane line, and the double flash of the vehicle is turned on; Sending the warning information and the operation status information to the TSP platform; The warning information and the operating status information are sent to a preset medical system through the TSP platform.

6. The driver status detection method according to claim 1, characterized in that: The machine learning model includes: an upper limb stiffness scoring model, a neck stiffness scoring model and a posture stability scoring model; the steps of inputting the facial features and the position information of the human body feature points into the pre-trained machine learning model and outputting the limb stiffness score and the posture stability score include: Inputting the position information of the human body feature points into the upper limb stiffness scoring model, and outputting the upper limb stiffness score; Inputting the facial features and the position information of the human body feature points into the neck stiffness scoring model, and outputting the neck stiffness score; Inputting the position information of the human body feature points into the posture stability scoring model, and outputting a posture stability score; The limb stiffness score of the driver is determined according to the upper limb stiffness score and the neck stiffness score, and the limb stiffness score of the driver is output.

7. The driver status detection method according to claim 1, characterized in that: The running status information includes: the acceleration, speed, driving direction and driving environment temperature and humidity information of the driving vehicle; The step of determining whether the driver is in a critical disease state according to the operating state information and the vital sign information comprises: Determining whether the traveling vehicle is in a speed-changing traveling state within a preset parameter range according to the acceleration, the speed, and the traveling direction; Whether the driver is in a critical illness state is determined based on the vital sign information, the driving environment temperature and humidity information, and the speed change driving state.

8. The driver status detection method according to claim 7, characterized in that: The speed-changing driving state includes: an accelerating driving state and a decelerating driving state; and the step of determining whether the driver is in a critical illness state according to the vital sign information, the driving environment temperature and humidity information, and the speed-changing driving state includes: Whether the driver is in a critical illness state is determined based on the vital sign information, the driving environment temperature and humidity information, the accelerated driving state, and the decelerated driving state.

9. A driver status detection device, characterized in that: include: A data acquisition module, used to acquire the driver's image information through an on-board image acquisition device; and obtaining physiological information of the driver through a wearable device; and, obtaining operating status information of the moving vehicle through on-board sensors; A preprocessing module, used for preprocessing the image information to obtain a preprocessed image; A feature extraction module, used to determine the driver's facial features, skin color features and human body feature point position information based on the pre-processed image; an identification module, used to determine the vital sign information of the driver according to the skin color characteristics and the physiological information; Furthermore, the facial features and the position information of the human body feature points are input into a pre-trained machine learning model, and the driver's limb stiffness score and posture stability score are output; the machine learning model is pre-trained based on images carrying preset limb stiffness scores and preset posture stability scores; a driving state determination block, for determining whether the driver is in a critical disease state based on the operating state information and the vital sign information; If not, the driving status score of the driver is determined according to the limb stiffness score and the posture stability score.

10. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the method according to any one of claims 1 to 8.

Images