Driver fatigue detection and elimination control system for inland ship and ship

Abstract

The driver fatigue detection and elimination control system comprises a ship operation information acquisition module and a timer, the output end of a cockpit camera is connected to a face fatigue recognition module, and the face fatigue recognition module is connected with the module; the face fatigue recognition module recognizes the fatigue state based on face information, collected by a cockpit camera, of a driver and sends the fatigue state to the control module. The ship operation information acquisition module is used for acquiring operation information or instruction information of a ship, the output end of the ship operation information acquisition module is connected to the control module, the control module is connected with the timer, and the control module is connected to the cockpit camera and the face fatigue recognition module. And the control module is used for controlling whether the cockpit camera and the face fatigue recognition module are started or not. According to the scheme, conditions are set for starting the fatigue detection of the driver, so that the starting of the fatigue detection is more targeted, and energy is saved.

Description

Technical Field

[0001] The utility model relates to the field of ship safety monitoring, and particularly relates to a driver fatigue detection and elimination control system for inland river ships. Background Art

[0002] Driver fatigue detection can improve driving safety. The prior art often uses vision technology for fatigue detection. For example, a vision-based driver fatigue detection method and system with patent application number 201110136065.5 discloses a vision-based driver fatigue detection method and system, belonging to the field of computer technology. The invention includes an image acquisition part, which uses a wireless camera to collect the driver's facial image and transmits the image to the mobile phone through a local area network built by a wireless router; an eye detection part, which uses the adaboost algorithm for face detection and eye detection and judges the eye state by binarizing the eye area; a fatigue judgment part, which calculates the fatigue degree according to the opening and closing state of the eyes and in combination with the blink frequency according to the P80 measurement method of PERCLOS; a fatigue alarm part, when it is judged to be in a fatigue state, the mobile phone alarms with a ringtone and at the same time triggers an alarm to alarm.

[0003] As can be seen from the above prior art patent, the vision technology solutions commonly used in the prior art identify the face to obtain the fatigue state. Generally, a camera is used to collect face data and then a face recognition module is used to identify the fatigue data, and then an alarm is issued. However, in the prior art, there is no design for the activation of the fatigue driving module. As long as the ship starts, the camera and face recognition module for fatigue driving start to work. Its activation is not targeted. Moreover, due to the current electrification trend of ships, the long-term activation of the fatigue driving camera will waste electric energy and affect the endurance of electric ships. Therefore, the activation of the fatigue driving system in the prior art has defects and cannot start fatigue recognition in a targeted manner to save energy; moreover, in the prior art, after detecting fatigue driving, only an alarm is issued, and there is no design for eliminating fatigue. Summary of the Utility Model

[0004] The purpose of the utility model is to overcome the deficiencies of the prior art and provide a driver fatigue detection and elimination control system for inland river ships, which realizes targeted activation of the camera for fatigue recognition and saves energy consumption.

[0005] To achieve the above object, the technical solution adopted by the utility model is as follows: A driver fatigue detection and elimination control system for inland river ships, including a cockpit camera, a face fatigue recognition module, and a control module; the output end of the cockpit camera is connected to the face fatigue recognition module, and the face fatigue recognition module is connected to the module; the face fatigue recognition module recognizes the fatigue state based on the face information of the driver collected by the cockpit camera and sends it to the control module; the control system further includes a ship operation information collection module and a timer, the ship operation information collection module is used to collect the operation information or command information of the ship, and its output end is connected to the control module, the control module is connected to the timer, and the control module is respectively connected to the cockpit camera and the face fatigue recognition module, and is used to control the start and stop of the cockpit camera and the face fatigue recognition module.

[0006] The ship operation information collection module is used to collect or read the commands issued by the propulsion motor telegraph transmitter or the manual operation information of the driver.

[0007] The ship operation information collection module includes a communication interface module, and the control module is connected to the propulsion system, the internal communication system and / or the propulsion system of the ship through the communication interface module to obtain the operation information or command information of the ship.

[0008] The output end of the control module is connected to an alarm.

[0009] The alarm includes a speaker, and the speaker is a speaker with adjustable volume and frequency; the control module drives the alarm to emit an alarm sound with adjustable volume and frequency.

[0010] The control system further includes a whole ship alarm, and the output end of the control module is connected to the whole ship alarm, and is used to notify all crew members through the whole ship alarm after the alarm works for a set time.

[0011] The control system further includes a communication module, and the control module is connected to a shore alarm through the communication module, and the shore alarm is arranged in a ship management agency.

[0012] The control module is connected to a fatigue elimination module, and the fatigue elimination module is used to eliminate the fatigue state of the driver by means of sound and light stimulation.

[0013] The control module is connected to a marine keyboard and a trackball, and is used to input the operation information of the driver through the marine keyboard and the trackball; the control module is connected to the cockpit integrated information display screen, and is used to display a fatigue relief screen or display entertainment videos through the cockpit integrated information display screen in a fatigued state.

[0014] A ship, which includes the driver fatigue detection and elimination control system for inland river ships as described above.

[0015] The advantages of the present utility model are as follows: The activation of the driver fatigue detection is conditional, making the fatigue detection more targeted, avoiding energy waste caused by the constant operation of the fatigue detection camera and fatigue detection module, and saving energy; A multi-level alarm system is set up, which can give different alarm reminders to the driver, all crew members, and shore monitoring personnel, improving safety through multiple alarms; It can eliminate the driver's fatigue to a certain extent; It can verify the elimination of the driver's fatigue, and judge the elimination effect of the driver's fatigue according to the information input by the user keyboard or trackball input. Brief Description of the Drawings

[0016] The following briefly describes the content expressed in each drawing of the present invention specification and the marks in the drawings:

[0017] Figure 1 It is the structural principle diagram of the fatigue detection and elimination control system of the present utility model. Detailed Embodiment

[0018] The following further details the specific embodiments of the present invention by describing the optimal embodiments with reference to the drawings.

[0019] In view of the defects in the activation of the existing fatigue driving recognition system, this solution designs a fatigue detection and elimination control system for accurately and reliably activating fatigue recognition, and the main goals to be achieved include one of the following: 1. When or under what conditions to activate fatigue detection; 2. Alarm when the driver is fatigued; 3. Fatigue elimination plan when the driver is fatigued. The following details the goals to be achieved as follows:

[0020] As Figure 1 shown, a driver fatigue detection and elimination control system for inland river ships includes a cockpit camera, a face fatigue recognition module, a control module, a ship operation information collection module, and a timer;

[0021] The output end of the cockpit camera is connected to the face fatigue recognition module, and the face fatigue recognition module is connected to the module; after the cockpit camera starts to work, it collects the face video information of the driver in the cockpit and sends it to the connected face fatigue recognition module. The face fatigue recognition module identifies and judges whether the driver is in a fatigued state based on face recognition. The output end of the face fatigue recognition module is connected to the control module. The face fatigue recognition module identifies the fatigued state based on the face information of the driver collected by the cockpit camera and sends it to the control module. The control module controls operations such as issuing an alarm based on the fatigued state and performs the alarm operation after fatigue. Since the operation of the cockpit camera and the face fatigue recognition module consumes electrical energy, when used on ships such as electric ships, it will consume more energy because the continuous operation of the camera and the continuous operation of the face fatigue recognition module will consume some energy. Therefore, accurately and reliably starting the camera and the face fatigue recognition module is one of the considerations in this solution. Therefore, a ship operation information collection module and a timer are provided on this basis.

[0022] The ship operation information collection module is used to collect the operation information or command information of the ship. Its output end is connected to the control module. The control module is connected to the timer. The timer times the time of the operation information or command information of the ship collected by the ship operation information collection module. When no operation information or command information is collected for more than the set time threshold, it is judged that there may be fatigue driving. Therefore, at this time, it is necessary to start the work of the camera and the face fatigue recognition module. The control module is respectively connected to the cockpit camera and the face fatigue recognition module and is used to control whether the cockpit camera and the face fatigue recognition module start to work. The control module can control the start of the cockpit camera and the face fatigue recognition module in a trigger-driven manner or in a power supply control manner. Taking power supply control as an example, relays are respectively arranged in the power supply circuits of the cockpit camera and the face fatigue recognition module. The control module controls whether the power supply of the camera and the face fatigue recognition module is turned on by driving the relays, so as to realize whether the power supply of the camera and the face fatigue recognition module is turned on, and further control the purpose of their power-on work or power-off shutdown.

[0023] The ship operation information collection module is used to collect or read the instructions sent by the propulsion motor telegraph transmitter or the manual operation information of the driver. By collecting the instructions sent by the propulsion motor telegraph transmitter or the manual operation information of the driver, the operation status information of the driver is judged. If the instructions sent by the propulsion motor telegraph transmitter or the manual operation information of the driver are not detected for a long time, it means that the driver may be in a fatigued state at this time. Therefore, it is necessary to turn on the camera and the fatigue recognition module to check and identify this situation. In order to realize the collection and acquisition of data, this solution adopts the method of directly reading the data in the shipboard control system of the ship to obtain relevant data, that is, the ship operation information collection module includes a communication interface module, and the control module is connected to the propulsion system, the internal communication system and / or the propulsion system of the ship through the communication interface module to obtain the operation information or instruction information of the ship. The communication interface module adopts a communication interface that matches the ship propulsion system, the internal communication system and / or the propulsion system, and then reads the data in the propulsion system, the internal communication system and / or the propulsion system, so as to realize the purpose of directly reading the data in the propulsion system, the internal communication system and / or the propulsion system to obtain the operation information or instruction information of the ship. This method can achieve the purpose of saving the implementation cost of this solution, and the control module can be directly connected to the ship system through the communication interface.

[0024] In the above solution, if the operation information or instruction information of the ship is not read after reaching the set time, the camera and the face fatigue recognition module are turned on. If the control module receives the fatigue status information sent by the face fatigue recognition module, an alarm needs to be issued. The alarm is used to warn the driver on the one hand and also needs to issue a hierarchical alarm to the monitoring personnel on the whole ship, on shore, etc. By means of multi-level alarms, safety accidents caused by fatigue can be avoided. That is, this solution includes a cockpit alarm, a whole-ship alarm, and a shore alarm, which respectively achieve different levels of alarms.

[0025] The output end of the control module is connected to the cockpit alarm, and the speaker in the cockpit alarm is configured as a speaker with adjustable volume and frequency; the control module drives the alarm to emit an alarm sound with adjustable volume and frequency. That is, the control module drives the speaker to emit sounds with different volumes and frequencies through the speaker power circuit to achieve multiple purposes such as reminding the driver, eliminating fatigue, and issuing an alarm.

[0026] The whole-ship alarm is used to notify all crew members through the whole-ship alarm after the alarm has worked for the set time. The output end of the control module is connected to the whole-ship alarm, and the whole-ship alarm is set at other reasonable positions outside the cockpit to notify other people on the ship of the driver's fatigue status, such as in the lounge, the captain's cabin, etc.

[0027] The shore alarm is set within the ship management agency, which can be the ship shore company monitoring or the official monitoring room, such as the monitoring center of the ship operator itself or the monitoring centers of the coast guard and the maritime police station; the control module is connected to the shore alarm through the communication module, and the communication module accesses the shore alarm by means of wireless communication. Among them, the communication module can utilize the ship's in-house communication system to establish external network connections and alarm connections.

[0028] The above three alarm methods can be controlled differently according to the duration of fatigue state recognition. For example, within the time when fatigue recognition is lower than the set time, only the driver alarm is used to alarm the driver; when the fatigue duration exceeds a certain time and is less than the external alarm time, the whole ship is alarmed so that it can be processed by the ship's interior in a timely manner; if the duration is greater than the external alarm time, it means that the driver's fatigue cannot be processed internally or only the driver is driving fatigued on the ship alone, then an alarm is sent to the shore alarm to remind the shore management personnel to handle it in a timely manner. Through multiple alarm mechanisms, the occurrence of safety accidents caused by fatigue driving can be avoided or discovered and handled in a timely manner.

[0029] In this solution, a fatigue elimination module is provided, which is used to eliminate the driver's fatigue state by means of sound and light stimulation. After detecting and recognizing that the driver is in a fatigue state, the driver is stimulated to eliminate fatigue or wake up the driver. For example, the fatigue elimination module includes an audio module and a lighting module, which stimulate the driver by driving the audio module to emit stimulating music or sounds and driving the lighting module to emit dazzling and blinding lights, so that the driver can regain clarity, which has a certain effect on fatigue and drowsiness.

[0030] In a preferred embodiment of this application, in order to verify whether the driver's fatigue has been eliminated, this solution is provided with an input module and a display screen. The input module is implemented by a marine keyboard and a trackball mouse, and the display screen is implemented by a cockpit integrated information display screen, reducing costs through multiplexing. The control module is connected to the marine keyboard and the trackball, and is used to input the driver's operation information through the marine keyboard and the trackball; the control module is connected to the cockpit integrated information display screen, and is used to display the fatigue relief screen or display entertainment videos through the cockpit integrated information display screen in a fatigue state. After the driver is fatigued, the control module drives the alarm to sound, drives the fatigue elimination module to act on the driver at this time, and at the same time drives the display screen to display verification codes, pictures or questions and answers and other information. The user inputs the corresponding verification information through the keyboard or the trackball. When the information input by the user through the keyboard and the trackball is consistent or matches the information displayed on the display screen, it is determined that the fatigue has been eliminated. At this time, the control module can cancel the work of the alarm and the fatigue elimination module. For example, a digital verification code can be displayed on the display screen, and the user inputs the corresponding verification code through the keyboard. As long as the two are the same, it is determined that the driver has eliminated fatigue.

[0031] In this embodiment, the control module mainly realizes the driving control function of the alarm, the power supply control of the camera, etc., which is implemented by a microcontroller with data processing and control functions, including 51 series single-chip microcomputers and STM32 series single-chip microcomputers. According to the required on-chip resources and costs, one of the models of single-chip microcomputers can be selected and developed according to the above principles to realize the control module.

[0032] The present solution also provides a ship, in which the driver fatigue detection and elimination control system for inland ships in the above-mentioned embodiment is integrated.

[0033] The working principle of this solution mainly includes the following parts:

[0034] 1. Monitor the fatigue status of the ship driver: Receive information from the propulsion system and the internal communication system, such as instructions or manual operations issued by the propulsion motor telegraph transmitter. When such instructions or manual operations are present within ten minutes, the system considers that the driver is not in a fatigue state and the system is not turned on. When the signal is not received for ten minutes, the driver may be in a fatigue state, and the system automatically turns on and enters the fatigue status monitoring mode, that is, the control module drives the camera and the fatigue recognition module to work, and the cockpit camera module collects video stream data and transmits it to the fatigue status judgment module. The fatigue status judgment module determines whether the driver has entered a fatigue state.

[0035] 2. Eliminate the driver's fatigue: The fatigue state elimination module is combined with the cockpit comprehensive information display screen to display any verification information such as picture verification code, text verification code or mathematical verification code mixed verification information without affecting navigation to verify whether the driver has eliminated fatigue. After eliminating fatigue, it is convenient to cancel the alarm. The driver enters the answer through the marine keyboard and trackball to judge whether he has escaped from the fatigue state based on the input time and correctness. When eliminating fatigue, audio stimulation is used to improve the driver's attention by playing short music clips or sound prompts and displaying short entertainment videos on the comprehensive information display screen. The alarm can be issued directly or through the above verification information. The verification information appears up to 5 times. If it appears more than 5 times, the warning module is triggered, and an alarm is issued.

[0036] 3. Multi-level alarm: The warning module uses sound and light alarms and speakers to provide three-level warnings:

[0037] Stage 1: Mild warning, using sound and light alarms to alert the driver.

[0038] Stage 2: Moderate alert, increasing speaker volume and frequency to strengthen reminders.

[0039] Phase 3: Serious warning, further expanding the warning scope and notifying all crew members on board.

[0040] The long-time warning cannot be lifted, and the system accesses the external communication system for external communication alarm to notify the onshore management agency. Each level of alarm can be issued separately according to the fatigue duration.

[0041] The input module (keyboard and trackball) is connected to the control module through the USB interface and is used to receive the input operations of the driver. The internal communication system, external communication system, and propulsion system are connected to the control module through existing communication interfaces (such as CAN bus or serial interface) to obtain the system status information. The internal communication system refers to the control system inside the ship, the external communication system refers to the system for the ship to interact and control with the outside world, and the propulsion system refers to the power system of the ship, which is the power system for propelling the ship forward. Since this solution adopts the above solution and uses existing ship equipment, the installation of additional equipment is reduced, meeting the economic requirements. It automatically monitors and judges the fatigue state, effectively eliminating the driver's fatigue. The flexible three-level warning mechanism improves the effectiveness and applicability of the warning. It improves the accuracy and real-time performance of driver fatigue detection.

[0042] Obviously, the specific implementation of the present invention is not limited by the above methods. As long as various non-substantive improvements are made by adopting the method concept and technical solution of the present invention, they are all within the protection scope of the present invention.

Claims

1. A driver fatigue detection and elimination control system for inland vessels, comprising a cockpit camera, a face fatigue recognition module, and a control module; the output end of the cockpit camera is connected to the face fatigue recognition module, and the face fatigue recognition module is connected to the module; the face fatigue recognition module recognizes the fatigue state based on the driver's face information collected by the cockpit camera and sends it to the control module; it is characterized in that: The control system also includes a ship operation information acquisition module and a timer. The ship operation information acquisition module is used to collect the ship's operation information or command information, and its output end is connected to the control module. The control module is connected to the timer, and the control module is respectively connected to the cockpit camera and the face fatigue recognition module to control whether the cockpit camera and the face fatigue recognition module are started or not.

2. A driver fatigue detection and elimination control system for inland waterway vessels as claimed in claim 1, characterized in that: The ship operation information acquisition module is used to collect or read the instructions sent by the propulsion motor telegraph transmitter or the manual operation information of the driver.

3. A driver fatigue detection and elimination control system for inland waterway vessels as claimed in claim 2, characterized in that: The ship operation information acquisition module includes a communication interface module, and the control module is connected to the ship's propulsion system, internal communication system and / or propulsion system through the communication interface module to obtain the ship's operation information or instruction information.

4. A driver fatigue detection and elimination control system for inland vessels according to any one of claims 1 to 3, characterized in that: The output end of the control module is connected to an alarm.

5. A driver fatigue detection and elimination control system for inland waterway vessels as claimed in claim 4, characterized in that: The alarm device comprises a speaker, and the speaker is a speaker with adjustable volume and frequency; the control module drives the alarm device to emit an alarm sound with adjustable volume and frequency.

6. A driver fatigue detection and elimination control system for inland waterway vessels as claimed in claim 4, characterized in that: The control system also includes a whole ship alarm, and the output end of the control module is connected to the whole ship alarm, which is used to notify all ship personnel through the whole ship alarm after the alarm has worked for a set time.

7. A driver fatigue detection and elimination control system for inland waterway vessels as claimed in claim 4, characterized in that: The control system further comprises a communication module, and the control module is connected to an onshore alarm via the communication module, and the onshore alarm is arranged in a ship management agency.

8. A driver fatigue detection and elimination control system for inland vessels according to any one of claims 1 to 3, characterized in that: The control module is connected to a fatigue elimination module, and the fatigue elimination module is used to eliminate the driver's fatigue state through sound and light stimulation.

9. A driver fatigue detection and elimination control system for inland waterway vessels as claimed in claim 8, characterized in that: The control module is connected to a marine keyboard and a trackball, and is used to input the driver's operating information through the marine keyboard and the trackball; the control module is connected to a cockpit comprehensive information display screen, and is used to display fatigue-relieving pictures in a fatigue state or to display entertainment videos through the cockpit comprehensive information display screen.

10. A ship, characterized in that: The ship includes a driver fatigue detection and elimination control system for inland ships as described in any one of claims 1-9.

Images