A method, device, equipment and storage medium for assisting parking warning

Abstract

The present invention relates to the field of automobile technology, and more specifically, to a method, device, equipment and storage medium for assisting parking warning. It includes: a driver status recognition module, an ultrasonic radar control module, and a holographic image control module; the present invention classifies driver fatigue into three states of awake, mild fatigue and severe fatigue based on multi-parameter fusion by recognizing eye features and mouth features; and has two working modes: a normal working mode and a radar self-starting mode, between which the mode switching can be realized according to the driver status or by a switch (priority: driver status>switch), in the mild fatigue and severe fatigue states, the radar module will switch to the radar self-starting mode; in different driving fatigue states, the frequency of the radar alarm sound will also be different; in addition, the present invention can display the environment around the car in real time.

Description

Technical Field

[0001] The present invention relates to the field of automobile technology, and in particular to a method, device, equipment and storage medium for assisting parking warning. Background Art

[0002] In narrow road conditions (such as parking lots and downtown areas), drivers are more likely to have scratches while driving, especially for drivers who are tired, causing unnecessary property losses or even casualties.

[0003] Most existing implementation solutions design the ultrasonic radar module and add a radar self-start function, that is, the radar turns on when it encounters an obstacle. The existing solution adds a radar self-start function, which can only be turned on and off manually; if you forget to turn on this function, it is easy to cause safety hazards. If you keep turning on this function, the radar will be repeatedly turned on and off in crowded sections, affecting the driving experience. Summary of the invention

[0004] The present invention provides a method, device, equipment and storage medium for auxiliary parking warning. By identifying eye features and mouth features, driver fatigue is divided into three states: awake, mild fatigue and severe fatigue based on multi-parameter fusion; and there are two working modes: normal working mode and radar self-starting mode. The two modes can be switched according to the driver's state or by switch (priority: driver state>switch). In the mild fatigue and severe fatigue states, the radar module will switch to the radar self-starting mode; in different driving fatigue states, the frequency of the radar alarm sound will also be different; in addition, the present invention can display the environment around the vehicle in real time, solving the above-mentioned problems existing in the existing auxiliary parking warning.

[0005] The technical solution of the present invention is described as follows in conjunction with the accompanying drawings:

[0006] In a first aspect, an embodiment of the present invention provides a method for assisting parking warning, comprising:

[0007] Step 1: Enable the driver status recognition module;

[0008] Step 2: The ultrasonic radar control module receives the driver fatigue state identified by the driver state recognition module, and starts the working mode according to the driver fatigue state;

[0009] Step 3: The holographic imaging control module executes the corresponding mode according to the working mode enabled by the ultrasonic radar control module.

[0010] Furthermore, in step one, the driver state recognition module divides the recognized driver fatigue state into three states: awake, mildly fatigued and severely fatigued through a deep learning method.

[0011] Furthermore, in step two, if the driver's state is identified as awake, the ultrasonic radar control module adjusts to the normal working mode after receiving the awake state signal; if the driver's state is identified as slightly fatigued, the ultrasonic radar control module adjusts to the first radar self-starting mode after receiving the mild fatigue state signal; if the driver's state is identified as severely fatigued, the ultrasonic radar control module adjusts to the second radar self-starting mode after receiving the severe fatigue state signal.

[0012] Further, in step three, when the ultrasonic radar control module turns on the first radar self-starting mode, it determines whether the vehicle speed is ≤15km / h; if the vehicle speed is greater than 15km / h, the radar stops working, and then determines whether the vehicle speed is ≤30km / h. If the vehicle speed is ≤30km / h, the holographic image control module turns on the first holographic image display mode for image display. If the vehicle speed is greater than 30km / h, the holographic image exits; if the vehicle speed is ≤15km / h, the first radar self-starting mode and the first holographic image display mode are started together; when the ultrasonic radar module turns on the second radar self-starting mode, it determines whether the vehicle speed is ≤15km / h; if the vehicle speed is greater than 15km / h, the radar stops working, and then determines whether the vehicle speed is ≤60km / h. If the vehicle speed is ≤60km / h, the holographic image control module turns on the second holographic image display mode for image display. If the vehicle speed is greater than 60km / h, the holographic image exits; if the vehicle speed is ≤15km / h, the second radar self-starting mode and the second holographic image display mode are started together.

[0013] Furthermore, the first radar self-starting mode will sound an alarm when an obstacle is detected, and the three detection distances correspond to three alarm sound levels according to close range, medium range and long range, and the alarm sound frequencies are: high, medium and low; the second radar self-starting mode will sound an alarm when an obstacle is detected, and the three detection distances correspond to three alarm sound levels according to close range, medium range and long range, and the alarm sound frequencies are: high, high and high.

[0014] Furthermore, the first holographic image display mode prominently displays close-range obstacles; and the second holographic image display mode prominently displays close-range and medium-range obstacles.

[0015] In a second aspect, an embodiment of the present invention further provides an auxiliary parking warning device, the device comprising:

[0016] The driver status recognition module is used to classify the driver's fatigue status into three states: awake, slightly fatigued, and severely fatigued by recognizing eye features and mouth features;

[0017] The ultrasonic radar control module is used to control the ultrasonic radar to be in two working modes: a normal working mode and a radar self-starting mode; wherein the radar self-starting mode includes a first radar self-starting mode and a second radar self-starting mode;

[0018] A holographic image control module, used for real-time display of the environment around the vehicle, including a first holographic image display mode and a second holographic image;

[0019] The driver status recognition module, ultrasonic radar control module and holographic image control module are connected via the vehicle-mounted CAN.

[0020] Furthermore, the normal working mode and the radar self-starting mode are switched according to the driver status or the switch implementation mode; the priority of the driver status implementation is greater than the switch implementation.

[0021] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, an assisted parking warning method as described in any one of the embodiments of the present invention is implemented.

[0022] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an assisted parking warning method as described in any one of the embodiments of the present invention.

[0023] The beneficial effects of the present invention are:

[0024] The present invention classifies driver fatigue into three states, namely, awake, mild fatigue and severe fatigue, based on multi-parameter fusion by identifying eye features and mouth features; and has two working modes: a normal working mode and a radar self-starting mode, between which the modes can be switched according to the driver's state or by a switch (priority: driver's state>switch). In mild fatigue and severe fatigue states, the radar module will switch to the radar self-starting mode; in different driving fatigue states, the frequency of the radar alarm sound will also be different; in addition, the present invention can display the environment around the vehicle in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a working flow chart of an auxiliary parking warning method in the present invention;

[0027] Figure 2 This is a working diagram of the first radar self-starting mode in the present invention;

[0028] Figure 3 The following is a flowchart of the second radar self-starting mode in the present invention;

[0029] Figure 4 It is a structural schematic diagram of an auxiliary parking warning device in the present invention;

[0030] Figure 5 The figure is a schematic diagram of the structure of an electronic device in the present invention. DETAILED DESCRIPTION

[0031] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0032] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0033] Embodiment 1

[0034] Figure 1 This is a flow chart of an auxiliary parking warning method provided in the first embodiment of the present invention. This embodiment is applicable to the case of auxiliary parking warning. The method can be executed by the auxiliary parking warning device in the embodiment of the present invention. The device can be implemented in software and / or hardware. Figure 1 As shown, the method specifically comprises the following steps:

[0035] There are two soft switches in the infotainment system settings, which respectively control the opening and closing of the driver status recognition module and the radar self-start module; there are two button switches in the cockpit, which respectively control the opening and closing of the radar and panoramic image.

[0036] Step 1: Enable the driver status recognition module;

[0037] The driver status recognition module supports multiple driver identity inputs and has its own driver status self-learning library. The driving habit library contains the driver's eye feature and mouth feature data sets. The system will classify the driver's status into three categories through deep learning methods: awake, mild fatigue and severe fatigue. After the driver status recognition module is started, if the user activates the system for the first time, the driver will be prompted to enter the identity (name or nickname). After the identity is entered, the system will self-learn the driver's driver for a period of time. If the user uses the system again, he can select his own name (or nickname) in the driver identity confirmation interface, and the system will implement the corresponding function based on the driver's driver status self-learning library.

[0038] See also Figure 2 and Figure 3 , step 2, the ultrasonic radar control module receives the driver fatigue state identified by the driver state recognition module, and starts the working mode according to the driver fatigue state;

[0039] The ultrasonic radar control module has two working modes to switch: normal working mode and radar self-starting mode. In normal working mode, the radar can be turned on and off by pressing the radar button switch; in radar self-starting mode, the radar automatically starts after the vehicle is IG ON. When the radar detects an obstacle, the radar display interface and the panoramic image display interface pop up and an alarm sound prompts. The radar self-starting mode is divided into the first radar self-starting mode and the second radar self-starting mode, and the alarm sound frequency is different between the two; the module requires a vehicle speed of ≤15km / h.

[0040] If the driver is identified as awake, the ultrasonic radar control module adjusts to the normal working mode after receiving the awake signal; if the driver is identified as slightly fatigued, the ultrasonic radar control module adjusts to the first radar self-starting mode after receiving the mild fatigue signal; if the driver is identified as severely fatigued, the ultrasonic radar control module adjusts to the second radar self-starting mode after receiving the severe fatigue signal.

[0041] See also Figure 2 and Figure 3 , step three, the holographic imaging control module executes the corresponding mode according to the working mode enabled by the ultrasonic radar control module.

[0042] The holographic image control module has two working modes to switch: normal working mode and image self-starting mode. In normal working mode, the image can be turned on and off only by pressing the image button switch; in image self-starting mode, the image is automatically displayed after receiving the driver's fatigue status or the radar detects an obstacle. The image self-starting mode is divided into the first holographic image display mode and the second radar self-starting mode, and the two have different speed requirements; the first holographic image display mode requires a speed of ≤30km / h, and the second radar self-starting mode requires a speed of ≤60km / h.

[0043] When the ultrasonic radar control module turns on the first radar self-starting mode, it determines whether the vehicle speed is ≤15km / h; if the vehicle speed is greater than 15km / h, the radar stops working, and then determines whether the vehicle speed is ≤30km / h. If the vehicle speed is ≤30km / h, the holographic image control module turns on the first holographic image display mode for image display. If the vehicle speed is greater than 30km / h, the holographic image exits; if the vehicle speed is ≤15km / h, the first radar self-starting mode and the first holographic image display mode are started together; when the ultrasonic radar module turns on the second radar self-starting mode, it determines whether the vehicle speed is ≤15km / h; if the vehicle speed is greater than 15km / h, the radar stops working, and then determines whether the vehicle speed is ≤60km / h. If the vehicle speed is ≤60km / h, the holographic image control module turns on the second holographic image display mode for image display. If the vehicle speed is greater than 60km / h, the holographic image exits; if the vehicle speed is ≤15km / h, the second radar self-starting mode and the second holographic image display mode are started together.

[0044] Turn on the driver status recognition module. If the driver is recognized to be awake, the ultrasonic radar control module will adjust to the normal working mode after receiving the signal. In this mode, the radar cannot be turned on automatically, but can be turned on manually through the radar button switch. After turning on, the front and rear radars correspond to three alarm sound levels according to the three detection distances (short distance, medium distance, and long distance), and the alarm sound frequencies are: high, medium, and low; if the driver is recognized to be slightly fatigued, the ultrasonic radar control module will adjust to the first radar self-starting mode after receiving the signal. In this mode, the radar is automatically turned on and an alarm is issued when an obstacle is detected. The three detection distances (short distance, medium distance, and long distance) correspond to three alarm sound levels, and the alarm sound frequencies are: high, medium, and low At the same time, the image control module receives the signal that the radar detects an obstacle and also displays the image, and displays close obstacles prominently (for example, the obstacle is partially enlarged and displayed); if the driver is identified as being in severe fatigue, the ultrasonic radar control module receives the signal and adjusts to the second radar self-start mode. In this mode, the radar is automatically turned on, and an alarm can be issued when an obstacle is detected. The three detection distances (close distance, medium distance, and long distance) correspond to three alarm sound levels, and the alarm sound frequencies are: high, high, and high. At the same time, the image control module receives the signal that the radar detects an obstacle and also displays the image, and displays close and medium distance obstacles prominently (for example, the obstacle is partially enlarged and displayed).

[0045] In addition, when the vehicle speed is greater than 15km / h and the radar module stops working, the panoramic image can continue to work according to the driver's fatigue status: when it is mild fatigue, the panoramic image can continue to work and display when the vehicle speed is ≤30km / h; when it is severe fatigue, the panoramic image can continue to work and display when the vehicle speed is ≤60km / h.

[0046] When the driver status recognition module is turned off, the ultrasonic radar control module has only two working modes, namely the normal working mode and the first radar self-start mode. The two modes can be switched through the radar self-start switch.

[0047] Among them, the short distance is 0-1.2 meters, the medium distance is 1.2-4.5 meters, and the long distance is 4.5-5 meters.

[0048] Embodiment 2

[0049] Figure 4 This is a schematic diagram of the structure of an auxiliary parking warning device provided in the second embodiment of the present invention. This embodiment can be applied to the case of auxiliary parking warning. The device can be implemented in software and / or hardware. The device can be integrated in any device that provides the function of auxiliary parking warning, such as Figure 4 As shown, the auxiliary parking warning device specifically includes:

[0050] The driver status recognition module is used to classify the driver's fatigue status into three states: awake, slightly fatigued, and severely fatigued by recognizing eye features and mouth features;

[0051] The ultrasonic radar control module is used to control the ultrasonic radar to be in two working modes: a normal working mode and a radar self-starting mode; wherein the radar self-starting mode includes a first radar self-starting mode and a second radar self-starting mode;

[0052] A holographic image control module, used for real-time display of the environment around the vehicle, including a first holographic image display mode and a second holographic image;

[0053] The normal working mode and the radar self-starting mode are switched according to the driver status or the switch; the priority of the driver status is greater than the switch implementation.

[0054] Embodiment 3

[0055] Figure 5 This is a schematic diagram of the structure of a computer device in Embodiment 3 of the present invention. Figure 5 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 5 The computer device 4 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0056] like Figure 5 As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0057] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0058] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0059] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0060] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0061] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable a user to interact with the computer device 12, and / or communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 22. In addition, the computer device 12 in this embodiment, the display 24 does not exist as an independent individual, but is embedded in the mirror surface, and when the display surface of the display 24 is not displayed, the display surface of the display 24 and the mirror surface are visually integrated. In addition, the computer device 12 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) through a network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the computer device 12 through a bus 18. It should be understood that although not shown in the figures, other hardware and / or software modules may be used in conjunction with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0062] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing an engine connecting rod buckling load testing method provided by an embodiment of the present invention.

[0063] Embodiment 4

[0064] Embodiment 4 of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, an auxiliary parking warning method provided in all the embodiments of the present application is implemented.

[0065] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0066] Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0067] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0068] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0069] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for assisting parking warning, characterized in that: include: Step 1: Enable the driver status recognition module; The driver status recognition module uses deep learning methods to classify the driver's fatigue status into three states: awake, mildly fatigued, and severely fatigued. Step 2: The ultrasonic radar control module receives the driver fatigue state identified by the driver state recognition module, and starts the working mode according to the driver fatigue state; Among them, if the driver is recognized as awake, the ultrasonic radar control module adjusts to the normal working mode after receiving the awake state signal; if the driver is recognized as slightly fatigued, the ultrasonic radar control module adjusts to the first radar self-starting mode after receiving the slightly fatigued state; if the driver is recognized as severely fatigued, the ultrasonic radar control module adjusts to the second radar self-starting mode after receiving the severely fatigued state; Step 3: The holographic imaging control module executes a corresponding mode according to the working mode enabled by the ultrasonic radar control module; Among them, when the ultrasonic radar control module turns on the first radar self-starting mode, it determines whether the vehicle speed is ≤15km / h; if the vehicle speed is greater than 15km / h, the radar stops working, and then determines whether the vehicle speed is ≤30km / h. If the vehicle speed is ≤30km / h, the holographic image control module turns on the first holographic image display mode for image display. If the vehicle speed is greater than 30km / h, the holographic image exits; if the vehicle speed is ≤15km / h, the first radar self-starting mode and the first holographic image display mode are started together; when the ultrasonic radar module turns on the second radar self-starting mode, it determines whether the vehicle speed is ≤15km / h; if the vehicle speed is greater than 15km / h, the radar stops working, and then determines whether the vehicle speed is ≤60km / h. If the vehicle speed is ≤60km / h, the holographic image control module turns on the second holographic image display mode for image display. If the vehicle speed is greater than 60km / h, the holographic image exits; if the vehicle speed is ≤15km / h, the second radar self-starting mode and the second holographic image display mode are started together; The first radar self-starting mode will sound an alarm when an obstacle is detected. The three detection distances correspond to three alarm sound levels according to close range, medium range and long range, and the alarm sound frequencies are: high, medium and low; the second radar self-starting mode will sound an alarm when an obstacle is detected. The three detection distances correspond to three alarm sound levels according to close range, medium range and long range, and the alarm sound frequencies are: high, medium and low; The first holographic image display mode prominently displays close obstacles; the second holographic image display mode prominently displays close and medium-distance obstacles.

2. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for assisting parking warning as claimed in claim 1 is implemented.

3. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for assisting parking warning as claimed in claim 1 is implemented.

Images