Driver state reminding method and system based on intelligent driving system and application

Abstract

The invention belongs to the technical field of intelligent driving, and provides a driver state reminding method and system based on an intelligent driving system and application. Judging whether the driver is in a distraction state based on the state data; if it is judged that the driver is in the distraction state, in-vehicle reminding is conducted on the driver through multi-sensory interaction; and if the driver still does not recover the control on the vehicle after the in-vehicle reminding is executed, taking over the vehicle to execute a preset safety strategy, and synchronously controlling the vehicle to perform out-vehicle warning. In-vehicle reminding is carried out on the distraction driver through multi-sensory interaction, in-vehicle warning is actively carried out under the condition that in-vehicle reminding is invalid, and dual safety guarantee is achieved.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to driver status reminder methods, systems and applications based on intelligent driving systems. Background Technology

[0002] Distracted driving is a key factor threatening road safety. Auditory interference, a highly insidious and far-reaching type of distraction, significantly reduces a driver's perception and decision-making abilities by consuming cognitive resources and diverting their attention, thereby prolonging reaction time and increasing collision risk. Globally, distracted driving has become one of the leading causes of traffic accidents. For example, a survey by the U.S. National Highway Traffic Safety Administration shows that up to 80% of traffic accidents are related to driver distraction, with the use of electronic devices such as mobile phones being a primary manifestation of distracted driving. Furthermore, in areas with mixed traffic characteristics, the complex composition of road participants (including motor vehicles, non-motorized vehicles, and pedestrians), chaotic traffic flow, and low rule compliance rates further amplify safety risks due to the combination of auditory interference and the complex environment, making the problem of distracted driving particularly prominent.

[0003] However, existing research focuses on single factors (such as visual distraction) or idealized road conditions, lacking in-depth exploration of the interaction between auditory interference and individual driver differences in mixed traffic scenarios.

[0004] To address the aforementioned issues, relevant technical personnel have optimized driver alert methods based on different types of distraction. For example, patent CN114132329B discloses a method and system for maintaining driver attention. This method includes: collecting driver information; determining whether the driver is distracted based on the driver information; further determining the type and degree of distraction when distraction is confirmed; and executing an alarm operation corresponding to the type and degree of distraction. This invention can effectively improve driver attention, prevent driver distraction, ensure driving safety, and reduce the probability of accidents. However, this invention focuses on in-vehicle alarm operations and the final safe stopping of the vehicle, lacking proactive warnings about the external environment and insufficient interaction with external traffic participants. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the present invention provides a driver status reminder method, system and application based on intelligent driving system. It provides in-vehicle reminders to distracted drivers through multi-sensory interaction, and actively provides external warnings when in-vehicle reminders are ineffective, thus achieving dual safety protection.

[0006] To achieve the above and related objectives, the present invention adopts the following technical solution:

[0007] The first aspect of this invention provides a driver status reminder method based on an intelligent driving system, comprising the following steps:

[0008] Step S100: Monitor the driver's status data in real time;

[0009] Step S200: Determine whether the driver is in a distracted state based on the status data;

[0010] In step S300, if it is determined that the driver is in a distracted state, an in-vehicle reminder is given to the driver through multi-sensory interaction;

[0011] In step S400, if the driver still fails to regain control of the vehicle after the in-vehicle reminder is executed, the vehicle is taken over to execute the preset safety strategy, and the vehicle is simultaneously controlled to issue external warnings.

[0012] Furthermore, in step S200, the distraction state includes at least an auditory distraction state and a visual distraction state. The auditory distraction state is when the driver's gaze is not directly forward and is continuously immersed in an auditory environment unrelated to driving, reaching a first time threshold. The visual distraction state is when the driver's gaze is not directly forward and is continuously focused on visual information unrelated to driving, reaching a second time threshold.

[0013] Furthermore, the first time threshold and / or the second time threshold are dynamically adjusted based on the vehicle's real-time speed.

[0014] Furthermore, in step S300, providing in-vehicle reminders to the driver through multi-sensory interaction includes: providing text and sound reminders through the vehicle's dashboard and / or head-up display, and providing vibration reminders by triggering the steering wheel or seat belt.

[0015] Furthermore, in step S400, the preset safety strategy includes controlling the vehicle to automatically pull over and park after taking over the vehicle.

[0016] Furthermore, in step S400, synchronously controlling the vehicle to provide external warnings includes synchronously controlling the vehicle's front and rear light modules to project warning information onto screens for external traffic participants.

[0017] A second aspect of the present invention provides a driver status reminder system based on an intelligent driving system, comprising:

[0018] The monitoring module is used to monitor the driver's status data in real time;

[0019] The judgment module is used to determine whether the driver is distracted based on status data.

[0020] The in-vehicle reminder module is used to remind the driver through multi-sensory interaction when it is determined that the driver is in a distracted state.

[0021] The external warning module is used to take over the vehicle and execute preset safety policies when the driver has not regained control of the vehicle after the in-vehicle reminder is executed, and simultaneously control the vehicle to issue external warnings.

[0022] Furthermore, it also includes a prompting and interaction module for providing text and sound reminders via the vehicle's dashboard and / or head-up display, and for providing vibration reminders by triggering the steering wheel or seat belt;

[0023] The vehicle control module is used to automatically take over the vehicle when the driver is unable to control it.

[0024] A third aspect of the present invention provides a computer-readable storage medium storing computer-readable instructions thereon, which, when executed by a computer's processor, cause the computer to execute the aforementioned driver status reminder method based on an intelligent driving system.

[0025] A fourth aspect of the present invention provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the driver status reminder method based on the intelligent driving system described above.

[0026] The beneficial technical effects of this invention are as follows:

[0027] This invention determines whether the driver is distracted based on driver status data and employs a multi-sensory interactive in-vehicle alert mechanism to remind the driver based on the type of distraction, ensuring the driver perceives danger immediately and takes appropriate action. Furthermore, if the invention confirms that the driver cannot regain control of the vehicle, it proactively takes over control and executes a safe strategy of pulling over to the side of the road, ensuring safety in emergency situations. Simultaneously, the invention also provides external warnings, forming a dual safety guarantee of in-vehicle alerts and external warnings, thereby effectively addressing the safety hazards posed by driver distraction to the driver and surrounding road users.

[0028] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0029] The accompanying drawings, incorporated in and forming part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without inventive effort. In the drawings:

[0030] Figure 1 This is a flowchart of the driver status reminder method based on the intelligent driving system in this application;

[0031] Figure 2 This is a framework diagram of the driver status reminder system based on the intelligent driving system in this application;

[0032] Figure 3 A schematic diagram of the structure of a computer system suitable for an embodiment of this application is shown. Detailed Implementation

[0033] Unless otherwise defined, all technical and / or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should be understood that certain features of the invention (described in the context of separate embodiments for clarity) may also be provided in a single embodiment. Conversely, multiple features of the invention (described in the context of a single embodiment for brevity) may also be provided separately or in any suitable combination or, where appropriate, in any other described embodiment of the invention. Certain features described in the context of various embodiments will not be considered essential features of those embodiments unless the embodiment is inoperable without those elements. The invention is further illustrated below by specific examples; however, it should be noted that the specific process conditions and results described in the embodiments of the invention are merely illustrative and should not be construed as limiting the scope of protection of the invention. All equivalent changes or modifications made in accordance with the spirit and essence of the invention should be covered within the scope of protection of the invention.

[0034] Please see Figure 1 The flowchart of the driver status reminder method based on the intelligent driving system in this application is described in detail below:

[0035] Step S100: Monitor the driver's status data in real time.

[0036] Specifically, this application can monitor the driver's status data in real time through an in-vehicle DMS camera. This status data can include the driver's head posture, facial orientation, gaze direction, eye movements, limb shape and position, etc. Furthermore, the aforementioned status data is transmitted to the vehicle control module via the DMS camera.

[0037] Step S200: Determine whether the driver is in a distracted state based on the status data.

[0038] Specifically, the distraction state of this application includes at least auditory distraction state and visual distraction state. The auditory distraction state is when the driver's gaze is not directly looking forward and is continuously immersed in a non-driving-related auditory environment for a first time threshold; the visual distraction state is when the driver's gaze is not directly looking forward and is continuously focused on non-driving-related visual information for a second time threshold.

[0039] Specifically, the method for determining that the driver's gaze is not directly forward can be to compare the monitored direction of the driver's gaze with the vehicle's direction of travel, calculate the gaze deviation angle, and if this angle exceeds a preset deviation threshold, then it is determined that the driver's gaze is not directly forward. Furthermore, the method for determining that the driver is continuously immersed in a non-driving-related auditory environment can be determined by using the status data of the in-vehicle infotainment system, such as audio being on or call status, and can also be combined with driver behavior such as wearing headphones for a comprehensive judgment. Furthermore, the method for determining that the driver is continuously focused on non-driving-related visual information can be to use a DMS camera to track the gaze point to determine whether the driver's gaze is focused on non-road areas for an extended period, such as areas other than the windshield, rearview mirror, and side windows.

[0040] Specifically, the first and / or second time thresholds of this application are dynamically adjusted based on the vehicle's real-time speed. For example, when the vehicle is traveling at high speeds, such as on highways, the safety risk is high. In this case, the first and / or second time thresholds will be shortened, such as from the initial 3 seconds to 2 seconds, to more quickly identify distractions. When the vehicle is traveling at low speeds or in congested traffic, the safety risk is relatively low. In this case, the first and / or second time thresholds will be appropriately extended, such as from the initial 3 seconds to 5 seconds, to reduce frequent interference with the driver and improve the user experience.

[0041] In step S300, if it is determined that the driver is in a distracted state, an in-vehicle reminder is given to the driver through multi-sensory interaction.

[0042] Specifically, this application provides text and sound alerts through the vehicle's dashboard and / or head-up display, and vibration alerts by triggering the steering wheel or seatbelt. More specifically, this application displays prominent warning text or graphics on the dashboard, projects warning text or graphics into the windshield's field of vision via the head-up display, emits alert sounds through the vehicle's audio system / speakers, and generates vibrations through the steering wheel and seatbelt, providing multi-sensory alerts through sight, hearing, and touch. More specifically, the above-mentioned multiple alert methods can be selectively triggered or triggered simultaneously.

[0043] In step S400, if the driver still fails to regain control of the vehicle after the in-vehicle reminder is executed, the vehicle is taken over to execute the preset safety strategy, and the vehicle is simultaneously controlled to issue external warnings.

[0044] Specifically, the preset safety strategy of this application includes controlling the vehicle to automatically pull over to the side of the road after taking over the vehicle. Simultaneously controlling the vehicle to provide external warnings includes simultaneously controlling the vehicle's front and rear light modules to project warning information to external road users.

[0045] Specifically, if the driver is adequately alerted through various means such as sight, hearing, and touch, but fails to demonstrate any effective intention or action to take over within a set time—for example, by holding the steering wheel, looking ahead, or applying steering or braking—the in-vehicle alert is deemed invalid, and the driver is deemed unable to control the vehicle independently. In this case, the vehicle control module automatically takes over the vehicle and, by comprehensively calculating the current road environment, traffic conditions, and the vehicle's own performance, plans a path for pulling over. Then, the vehicle control module controls the vehicle to decelerate, change lanes, and ultimately park the vehicle in a safe area on the side of the road.

[0046] Specifically, after the vehicle pulls over to the side of the road, the external lighting system can be used to proactively provide warnings outside the vehicle, such as controlling the front and rear light modules to turn on. Warning information can also be displayed on the screen, such as predefined warning messages about abnormalities inside the vehicle when not driving, in order to proactively inform surrounding vehicles and prevent secondary accidents.

[0047] Please see Figure 2 Here is a framework diagram of the driver status reminder system 200 based on the intelligent driving system of this application, including:

[0048] Monitoring module 210 is used to monitor the driver's status data in real time;

[0049] The judgment module 220 is used to determine whether the driver is in a distracted state based on the status data;

[0050] The in-vehicle reminder module 230 is used to remind the driver in the vehicle through multi-sensory interaction when it is determined that the driver is in a distracted state.

[0051] The external warning module 240 is used to take over the vehicle and execute preset safety strategies when the driver has not regained control of the vehicle after the in-vehicle reminder is executed, and simultaneously control the vehicle to issue external warnings.

[0052] Furthermore, it also includes a prompting and interaction module 250 for providing text and sound reminders via the vehicle's dashboard and / or head-up display, and for providing vibration reminders by triggering the steering wheel or seat belt;

[0053] The vehicle control module 260 is used to automatically take over the vehicle when the driver is unable to control it.

[0054] It should be noted that the driver status reminder system based on the intelligent driving system provided in the above embodiments and the driver status reminder method based on the intelligent driving system provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs operations have been described in detail in the method embodiments, and will not be repeated here. In practical applications, the driver status reminder system based on the intelligent driving system provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above, and this is not a limitation here.

[0055] Embodiments of this application also provide a computer device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the computer device to implement the driver status reminder method based on the intelligent driving system provided in the above embodiments.

[0056] Figure 3 A schematic diagram of the structure of a computer system suitable for an embodiment of this application is shown. It should be noted that... Figure 3 The computer system 300 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0057] like Figure 3 As shown, the computer system 300 includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage section 308 into a random access memory (RAM) 303, such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in the RAM 303. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304. The following components are connected to the I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (local area network) card, modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A driver 310 is also connected to the I / O interface 305 as needed. Removable media 311, such as disks, optical discs, magneto-optical discs, semiconductor memories, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0058] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer tool programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by central processing unit (CPU) 301, it performs various functions defined in the system of this application.

[0059] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, flash memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. Computer programs contained on computer-readable media can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0060] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0061] The units described in the embodiments of this application can be implemented by tools or by hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the unit itself.

[0062] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, causes the computer to perform the driver status reminder method based on the intelligent driving system described above. This computer-readable storage medium may be included in the computer device described in the above embodiments, or it may exist independently and not incorporated into the computer device.

[0063] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the driver status reminder method based on the intelligent driving system provided in the various embodiments above.

[0064] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A driver status reminder method based on an intelligent driving system, characterized in that, Includes the following steps: Step S100: Monitor the driver's status data in real time; Step S200: Determine whether the driver is in a distracted state based on the state data; Step S300: If it is determined that the driver is in the distracted state, then an in-vehicle reminder is given to the driver through multi-sensory interaction; In step S400, if the driver still fails to regain control of the vehicle after the in-vehicle reminder is executed, the vehicle is taken over to execute the preset safety strategy, and the vehicle is simultaneously controlled to issue external warnings.

2. The method according to claim 1, characterized in that, In step S200, the distraction state includes at least auditory distraction and visual distraction, wherein, The auditory distraction state is defined as the driver's eyes not looking directly ahead and being continuously immersed in an auditory environment unrelated to driving, reaching the first time threshold. The visual distraction state is defined as the driver's gaze not being directed forward and continuously focusing on non-driving-related visual information for a period of time until a second time threshold is reached.

3. The method according to claim 2, characterized in that, The first time threshold and / or the second time threshold are dynamically adjusted based on the real-time vehicle speed.

4. The method according to claim 1, characterized in that, In step S300, providing in-vehicle reminders to the driver through multi-sensory interaction includes: providing text and sound reminders through the vehicle's dashboard and / or head-up display, and providing vibration reminders by triggering the steering wheel or seat belt.

5. The method according to claim 1, characterized in that, In step S400, the preset safety strategy includes controlling the vehicle to automatically pull over and park after taking over the vehicle.

6. The method according to claim 5, characterized in that, In step S400, synchronously controlling the vehicle to provide external warnings includes synchronously controlling the vehicle's front and rear light modules to project warning information onto screens for external traffic participants.

7. A driver status reminder system based on an intelligent driving system, characterized in that, include: The monitoring module is used to monitor the driver's status data in real time; The judgment module is used to determine whether the driver is in a distracted state based on the state data; The in-vehicle reminder module is used to provide in-vehicle reminders to the driver through multi-sensory interaction when it is determined that the driver is in the distracted state. The external warning module is used to take over the vehicle and execute a preset safety strategy when the driver fails to regain control of the vehicle after the in-vehicle reminder is executed, and simultaneously control the vehicle to issue external warnings.

8. The system according to claim 7, characterized in that, It also includes a prompting and interaction module for providing text and sound reminders via the vehicle's dashboard and / or head-up display, and for providing vibration reminders by triggering the steering wheel or seat belt; The vehicle control module is used to automatically take over the vehicle when the driver is unable to control it.

9. A computer-readable storage medium, characterized in that, It stores computer-readable instructions, which, when executed by the computer's processor, cause the computer to perform the driver status reminder method based on the intelligent driving system as described in any one of claims 1 to 6.

10. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the steps of the driver status reminder method based on any one of claims 1 to 6.

Images