Monitoring device, saddle-type vehicle, information terminal, monitoring method, and program

A monitoring device for straddle-type vehicles uses cameras and sensors to detect driver and vehicle abnormalities with a simple configuration, addressing the complexity and cost issues of existing technologies by providing accurate and timely alerts.

JP7579428B2Active Publication Date: 2024-11-07HONDA MOTOR CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2023508769
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2021-03-23
Filing Date
2022-02-14
Publication Date
2024-11-07
Estimated Expiration
2042-02-14

AI Technical Summary

Technical Problem

Existing technologies for straddle-type vehicles require complex processing to analyze accidents, increasing vehicle costs and complicating the detection of driver and vehicle abnormalities.

Method used

A monitoring device with simple configuration and processing that uses multiple cameras and sensors to detect driver and vehicle abnormalities, including falls, by analyzing images and sensor data to determine specific situations indicative of abnormalities.

Benefits of technology

Accurately detects driver and vehicle abnormalities with a simple setup, enabling timely notification and appropriate responses to potential hazards.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007579428000001
    Figure 0007579428000001
  • Figure 0007579428000002
    Figure 0007579428000002
  • Figure 0007579428000003
    Figure 0007579428000003
Patent Text Reader

Abstract

This monitoring device monitors an abnormality in a driver and / or in a vehicle, the monitoring device comprising an acquisition means for acquiring state information indicating the states of the driver and the vehicle, and a determination means for determining that the abnormality has occurred when a prescribed state is detected on the basis of the state information acquired by the acquisition means, and the state information including a first image obtained by a first imaging unit that is capable of imaging the driver driving the vehicle.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a monitoring technique for monitoring an abnormality in a driver and / or a saddle-type vehicle. [Background technology]

[0002] Patent Document 1 proposes a technique for analyzing the circumstances of an accident based on video or audio before and after the accident transmitted from a drive recorder mounted on a vehicle, and transmitting the analysis results to an emergency rescue center. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-123814 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology described in Patent Document 1, in order to analyze the accident situation, advanced (complex) processing is required for the video and audio captured by the drive recorder. However, in the case of a saddle-type vehicle, for example, from the viewpoint of suppressing an increase in vehicle costs, it is desirable to accurately detect abnormalities (such as a fall) of the driver and / or the vehicle with a simple configuration and processing.

[0005] Therefore, an object of the present invention is to provide an advantageous technique for accurately detecting an abnormality in a driver and / or a vehicle with a simple configuration and processing. [Means for solving the problem]

[0006] In order to achieve the above object, a monitoring device according to one aspect of the present invention is mounted on a saddle riding type vehicle and monitors a driver of the saddle riding type vehicle and / or an abnormality in the saddle riding type vehicle, and includes an acquisition means for acquiring situation information indicating a situation of the driver and the saddle riding type vehicle, and a determination means for determining that the abnormality has occurred when a predetermined situation is detected based on the situation information acquired by the acquisition means, and the situation information includes a first image acquired by a first imaging unit provided in the saddle riding type vehicle and capable of capturing an image of the driver seated in a seat of the saddle riding type vehicle, and an image of the surroundings of the saddle riding type vehicle. and a second image obtained by a second photographing unit provided in the saddle-riding vehicle and capable of photographing the surrounding area, the predetermined situation includes a first situation in which the driver is not seated in the seat and a second situation in which the driver is lying on the road around the saddle-riding vehicle, and when the determination means detects the first situation based on the first image obtained by the acquisition means and detects the second situation based on the second image obtained by the acquisition means, it determines that the abnormality is that the saddle-riding vehicle has rolled over while the driver is driving the saddle-riding vehicle. Effect of the Invention

[0007] According to the present invention, for example, it is possible to provide an advantageous technique for accurately detecting an abnormality in a driver and / or a vehicle with a simple configuration and processing.

[0008] Other features and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings, in which the same reference numerals are used to designate the same or similar components throughout the drawings. [Brief description of the drawings]

[0009] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention. [Figure 1] A diagram showing an example of an information terminal mounted on a saddle-type vehicle. [Diagram 2] FIG. 1 is a block diagram showing an example of the configuration of a system including a monitoring device. [Diagram 3] A diagram showing an example of the arrangement of the photographing unit (camera) in a vehicle. [Figure 4] A flowchart showing a monitoring process for monitoring an abnormality while a vehicle is being driven. [Figure 5A] FIG. 1 is a diagram showing an example of an image obtained by an imaging unit; [Figure 5B] FIG. 1 is a diagram showing an example of an image obtained by an imaging unit; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The present invention is not limited to the following embodiment, and includes changes and modifications of the configuration within the scope of the present invention. Furthermore, not all of the combinations of features described in the present embodiment are necessarily essential to the present invention. Note that the same components are given the same reference numbers and their description will be omitted.

[0011] First Embodiment An example of application of the monitoring device according to the present invention to a saddle-type vehicle will be described below. The monitoring device according to the present invention is a device for monitoring an abnormality (such as a fall) of a driver and / or a vehicle and detecting the abnormality. In the following embodiment, a motorcycle will be described as an example of a saddle-type vehicle, but the monitoring device according to the present invention can also be applied to other types of saddle-type vehicles such as three-wheeled vehicles and vehicles other than saddle-type vehicles such as four-wheeled vehicles. The monitoring device may be configured as an in-vehicle device provided in a vehicle, but in this embodiment, it can be configured as an information terminal (mobile terminal). An information terminal is, for example, a device having various functions such as a communication function and a calling function, such as a smartphone or tablet terminal that has become widely used in recent years, and can be configured to be detachable from a saddle-type vehicle V by a dedicated holding mechanism HM, as shown in FIG. 1. FIG. 1 shows an example in which an information terminal 20 as a mobile terminal is mounted on a saddle-type vehicle V using a dedicated holding mechanism HM.

[0012] 2 is a block diagram showing a configuration example of a system 100 including a monitoring device of this embodiment. The system 100 of this embodiment may include an in-vehicle device 10 mounted on a vehicle V and an information terminal 20 owned by a driver (user) of the vehicle V. In this embodiment, the information terminal 20 may be understood as a monitoring device that monitors abnormalities of the driver and / or the vehicle V. The in-vehicle device 10 and the information terminal 20 may be connected to each other so as to be able to communicate with each other using a communication method such as Bluetooth (registered trademark) or Wi-Fi (registered trademark), but may also be connected to each other so as to be able to communicate with each other via the Internet.

[0013] First, a configuration example of an in-vehicle device 10 mounted on a vehicle V will be described. The in-vehicle device 10 may include, for example, a processing unit 11, a storage unit 12, an image capturing unit 13, a status detection sensor 14, and a communication unit 15, which are communicatively connected to each other via a system bus 16. The processing unit 11 is, for example, an ECU (Electronic Control Unit), and may include a processor represented by a CPU, a storage device such as a semiconductor memory, an interface with an external device, and the like. The storage unit 12 stores programs executed by the processor, data used by the processor for processing, and the like, and the processing unit 11 can read out the programs stored in the storage unit 12 into a storage device such as a memory and execute them.

[0014] The photographing unit 13 includes, for example, a camera, and is provided on the vehicle V to photograph a part of the vehicle V (driver) and / or the surroundings of the vehicle V to obtain an image (video). For example, the photographing unit 13 may perform photographing so as to repeatedly (for example, periodically) obtain still images, or may perform photographing so as to obtain a moving image. The photographing unit 13 of this embodiment may include a plurality of cameras 13a to 13f arranged on the vehicle V as shown in FIG. 3. FIG. 3 is a diagram of the vehicle V as a saddle-type vehicle viewed from above, and shows an example of the arrangement of the photographing unit 13 (cameras 13a to 13f) on the vehicle V. In the figure, the fan-shaped dashed lines indicate the photographing direction (photographing range) of each camera, with "FR" indicating the front of the vehicle V and "RR" indicating the rear of the vehicle, respectively.

[0015] In the example shown in FIG. 3, the cameras 13a to 13b are cameras (first photographing units) arranged on the vehicle V so as to be able to photograph the driver who is driving the vehicle V. The camera 13a is provided on the vehicle V so as to obtain an image of the rear of the vehicle V, and photographs the front of the driver who is seated on the seat 32. The camera 13b is provided on the vehicle V so as to obtain an image of the front of the vehicle V, and photographs the back of the driver who is seated on the seat 32. In the case of this embodiment, the cameras 13a to 13b can be arranged so as to be able to photograph the handlebars 31 (left and right grips), the seat 32 (driver's seat), and the ignition key 33. The cameras 13c to 13f are cameras (second photographing units) arranged on the vehicle V so as to be able to photograph the surroundings of the vehicle V. The camera 13c can photograph the front of the vehicle V, the camera 13d can photograph the rear of the vehicle V, the camera 13e can photograph the left side of the vehicle V, and the camera 13f can photograph the right side of the vehicle V.

[0016] The status detection sensor 14 is a sensor that detects various statuses of the vehicle V. For example, the status detection sensor 14 may include a position sensor (GPS sensor) that detects the current position of the vehicle V, a speed sensor that detects the speed of the vehicle V, an acceleration sensor that detects the acceleration of the vehicle V, and an attitude sensor (tilt sensor) that detects the attitude (tilt) of the vehicle V. The status detection sensor 14 may also include a sensor that detects the ignition state (on state or off state) of the vehicle V, a sensor that detects the position of a side stand to allow the vehicle V to stand on its own when stopped, and the like.

[0017] The communication unit 15 communicates with the information terminal 20 using a communication method such as Bluetooth (registered trademark) or Wi-Fi (registered trademark). Specifically, the communication unit 15 can have a function as a transmitter that transmits information to the information terminal 20 and a function as a receiver that receives information from the information terminal 20.

[0018] Next, a configuration of the information terminal 20 configured as a monitoring device will be described. As described above, the information terminal 20 is a device (apparatus) that can transmit and receive information to and from the in-vehicle device 10 of the vehicle V, such as a smartphone or a tablet terminal, and can include a processing unit 21, a storage unit 22, a notification unit 23, an input / output unit 24, and a communication unit 25 that are communicably connected to each other via a system bus 26.

[0019] The processing unit 21 includes a processor represented by a CPU, a storage device such as a semiconductor memory, an interface with an external device, and the like. The storage unit 22 stores programs executed by the processor and data used by the processor for processing, and the processing unit 21 can read out the programs stored in the storage unit 22 into a storage device such as a memory and execute them. In the case of the present embodiment, the storage unit 22 stores an application program (hereinafter, sometimes referred to as a monitoring program) for monitoring an abnormality (e.g., a rollover) of the driver and / or the vehicle V, and the processing unit 21 can read out the monitoring program stored in the storage unit 22 into a storage device such as a memory and execute it. In other words, the processing unit 21 can be a computer that executes the monitoring program.

[0020] The processing unit 21 of this embodiment may be provided with an acquisition unit 21a and a judgment unit 21b. The acquisition unit 21a acquires situation information indicating the situation of the driver and the situation of the vehicle V from the photographing unit 13 and / or the state detection sensor 14. The situation information is information indicating the current situation of the driver and the vehicle V, and may include an image (first image) obtained by photographing the driver while driving with the cameras 13a to 13b (first photographing unit), and / or an image (second image) obtained by photographing the surroundings of the vehicle V with the cameras 13c to 13f (second photographing unit). The situation information may also include at least one of information on the position, speed, acceleration, and attitude of the vehicle V detected by various sensors of the state detection sensor 14. The situation information may also include information on the ignition state of the vehicle V and information on the position of the side stand. The judgment unit 21b judges whether or not an abnormality (for example, an abnormality during driving) has occurred in the driver and / or the vehicle V based on the situation information acquired by the acquisition unit 21a. For example, when a predetermined situation is detected based on the situation information acquired by the acquisition unit 21a, the determination unit 21b can determine that the abnormality has occurred. The predetermined situation may be understood as a determination item used to determine (estimate) the occurrence of an abnormality (fall), and the details thereof will be described later.

[0021] The notification unit 23 notifies the occurrence of an abnormality. For example, if the information terminal 20 has a speaker, the notification unit 23 can notify the occurrence of an abnormality by outputting a sound from the speaker. The notification unit 23 may also communicate with the in-vehicle device 10 of the vehicle V via the communication unit 25 and notify the occurrence of an abnormality by outputting a sound from a speaker mounted on the vehicle V or by turning on or blinking the lighting devices (headlights, taillights) or hazard lights of the vehicle V. The notification unit 23 may notify the occurrence of an abnormality by contacting an external organization (for example, a fire department, a police station, an insurance company, etc.) about the occurrence of the abnormality using a telephone function or an email function mounted on the information terminal 20.

[0022] The input / output unit 24 outputs information to the driver (user) and receives information input from the driver. For example, the input / output unit 24 may be a unit including a display unit (display) and an input unit (buttons), or may be a touch panel display. The communication unit 25 communicates with the in-vehicle device 10 of the vehicle V using a communication method such as Bluetooth (registered trademark) or Wi-Fi (registered trademark). Specifically, the communication unit 25 may have a function as a transmitter that transmits information to the in-vehicle device 10 and a function as a receiver that receives information from the in-vehicle device 10.

[0023] [Monitoring process] Next, a flow of processing (monitoring processing) performed by the processing unit 21 of the information terminal 20 when the monitoring program is executed will be described. FIG. 4 is a flowchart showing the monitoring processing performed by the processing unit 21. As described above, the processing unit 21 of this embodiment acquires situation information obtained by the photographing unit 13 and / or the state detection sensor 14, and judges that an abnormality has occurred when a predetermined situation is detected based on the situation information. The predetermined situation may be configured by a predetermined combination of multiple types of situations detected from the situation information. That is, the processing unit 21 can judge that an abnormality has occurred when a predetermined combination of multiple types of situations is detected from the situation information. Each of the multiple situations may be understood as a judgment item used to judge (estimate) the occurrence of an abnormality (fall). Note that this flowchart starts when a user (e.g., a driver) starts (launches) the monitoring program, and can be repeatedly executed unless the user stops the monitoring program. That is, this flowchart can be started anew from step S11 even after it has ended. During execution of this flowchart, in the in-vehicle device 10 of the vehicle V, the photographing unit 13 photographs the vehicle and the status detection sensor 14 detect the vehicle.

[0024] In step S11, the processing unit 21 judges whether or not the driver is driving the vehicle V. Specifically, the processing unit 21 (acquisition unit 21a) acquires an image (first image) from the cameras 13a to 13b of the photographing unit 13. Then, the processing unit 21 (determination unit 21b) performs a known image processing technique on the image (first image) acquired from the cameras 13a to 13b, and judges, based on the image, whether or not the driver is gripping the steering wheel 31 (at least one of the left and right grips) and / or whether or not the driver is seated in the seat 32 (driver's seat) of the vehicle V. When a state in which the driver is gripping the steering wheel 31 and / or a state in which the driver is seated in the seat 32 of the vehicle V is detected, it is judged that the driver is driving the vehicle V, and step S11 is repeated. On the other hand, if the images from the cameras 13a to 13b detect a situation in which the driver is not gripping the steering wheel 31 and / or the driver is not seated in the seat 32 of the vehicle V, it is determined that the driver is not driving the vehicle V (first situation) and the process proceeds to step S12.

[0025] In step S12, the processing unit 21 judges whether or not a driver has been detected around the vehicle. Specifically, the processing unit 21 (acquisition unit 21a) acquires an image (second image) from the cameras 13c to 13f of the photographing unit 13 by the acquisition unit 21a. Then, the processing unit 21 (determination unit 21b) performs a known image processing technique on the image (second image) acquired from the cameras 13c to 13f, and judges whether or not a driver has been detected around the vehicle based on the image. As an example, the processing unit 21 can determine whether or not a driver is captured in the image acquired from the cameras 13c to 13f in this step S12 by extracting and storing characteristic points of the driver, such as the color and pattern of the helmet and clothes, based on the images acquired from the cameras 13a to 13b while driving the vehicle V. If the driver is captured in the image acquired from the cameras 13c to 13f, it is judged that the driver has been detected around the vehicle V, and the process proceeds to step S13. On the other hand, if the driver is not captured in the images acquired from the cameras 13c to 13f, it is determined that the driver has not been detected around the vehicle V, and the process proceeds to step S17.

[0026] In step S13, the processing unit 21 judges whether or not the driver is lying on the road in the image (second image) acquired in step S12. Specifically, the processing unit 21 (judgment unit 21b) judges whether or not a situation in which the driver is lying on the road (second situation) has been detected based on the image acquired in step S12 by performing a known image processing technique on the image. If a situation in which the driver is lying on the road (second situation) has been detected, the process proceeds to step S14, where the processing unit 21 judges that an abnormality has occurred while the driver is driving the vehicle V and that the driver has been thrown from the vehicle V. Then, the process proceeds to step S15, where the processing unit 21 notifies the occurrence of the abnormality by the notification unit 23. As described above, the notification mode by the notification unit 23 includes output of sound from the speaker of the information terminal 20 or the vehicle V, lighting or blinking of the lighting devices or hazard lights of the vehicle V, and contacting an external organization of the occurrence of the abnormality by using a telephone function or an email function.

[0027] On the other hand, in step S13, if a situation in which the driver is standing or walking on the road is detected, but a situation in which the driver is lying on the road (second situation) is not detected, the process proceeds to step S16, where the processing unit 21 (determination unit 21b) determines that the driver is safe and ends the process. In this case, since there is a possibility that the driver himself will report the occurrence of an abnormality, or that no abnormality has occurred, in this embodiment, the process ends without reporting the occurrence of an abnormality.

[0028] In step S17, which is reached when it is determined in step S12 that the driver has not been detected around the vehicle V, the processing unit 21 determines whether the ignition of the vehicle V is on (third situation). Specifically, the processing unit 21 (acquisition unit 21a) acquires an image (first image) from the cameras 13a to 13b of the photographing unit 13. As an example, FIG. 5A shows an image 41 acquired by the camera 13b of the photographing unit 13. The image 41 shown in FIG. 5A includes images of the steering wheel 31, the seat 32, and the ignition key 33. The processing unit 21 (determination unit 21b) then performs a known image processing technique on the image acquired from the cameras 13a to 13b (image 41 in FIG. 5A) to detect from the image whether the position (rotation position) of the ignition key 33 is in the on position or the off position, and determines whether the ignition of the vehicle V is on. If it is determined that the ignition of the vehicle V is off, the process proceeds to step S18, where the processing unit 21 (determination unit 21b) determines that the driver has normally dismounted from the vehicle V (i.e., no abnormality has occurred) and ends the process. On the other hand, if it is determined that the ignition of the vehicle V is on (third situation), the process proceeds to step S19.

[0029] Here, in step S17 of the present embodiment, the processing unit 21 determines whether or not the ignition of the vehicle V is on based on the images acquired from the cameras 13a to 13b, but the present invention is not limited to this. When an ignition detection sensor that detects the ignition state of the vehicle V is provided in the vehicle V as one of the state detection sensors 14, the processing unit 21 may determine whether or not the ignition of the vehicle V is on based on ignition state information acquired from the ignition detection sensor. For example, when a sensor that detects the position (rotation position) of the ignition key 33 is used as the ignition detection sensor, the processing unit 21 may determine whether or not the ignition of the vehicle V is on based on state information indicating the position of the ignition key 33 acquired from the sensor. In addition, if the vehicle V is an electric vehicle (EV), the processing unit 21 may use a sensor that detects the state (remaining charge or output amount) of the battery installed in the vehicle V as an ignition detection sensor, and determine whether or not the ignition of the vehicle V is on based on status information indicating the state of the battery obtained from the sensor.

[0030] In step S19, the processing unit 21 judges whether or not the vehicle V is in a state of overturning (fourth state). Specifically, the processing unit 21 (acquisition unit 21a) acquires an image (second image) acquired from the cameras 13c to 13f of the photographing unit 13. Then, the processing unit 21 (determination unit 21b) performs a known image processing technique on the image (second image) acquired from the cameras 13c to 13f, thereby judging whether or not the vehicle V is in a state of overturning based on the image. As an example, FIG. 5B shows an image 42 acquired by the camera 13c of the photographing unit 13 when the vehicle V has overturned. The image 42 shown in FIG. 5B includes an image of another vehicle OV. The processing unit 21 extracts feature points of the object (object) in the image 42 as a known image processing technique to identify what structure the object is (another vehicle OV in FIG. 5B), and calculates the degree of inclination (amount of inclination) of the object (other vehicle OV) in the image 42 relative to the inclination of the object (other vehicle OV) that can be obtained by the camera 13c under normal circumstances. If the calculated amount of inclination is equal to or greater than a threshold, it can be determined that the vehicle V is in an overturned state. If it is determined that the vehicle V is not in an overturned state, the process proceeds to step S20, and the processing unit 21 (determination unit 21b) determines that the driver forgot to turn off the ignition when leaving the vehicle V (i.e., no abnormality has occurred), and ends the process. For example, if the ignition is on for a predetermined time after it is determined that the vehicle V is not in an overturned state, the processing unit 21 can determine that the driver forgot to turn off the ignition when leaving the vehicle V. On the other hand, if it is determined that the vehicle V is in an overturned state (fourth situation), the processing proceeds to step S14, and after the processing unit 21 (judgment unit 21b) determines that an abnormality has occurred, the processing unit 21 notifies the occurrence of the abnormality via the notification unit 23 in step S15.

[0031] Here, in step S19 of this embodiment, the processing unit 21 determines whether or not the vehicle V is in a tipped-over state based on the images acquired from the cameras 13c to 13f, but the present invention is not limited to this. For example, if a stand detection sensor that detects the state (position) of the side stand of the vehicle V is provided on the vehicle V as one of the state detection sensors 14, the processing unit 21 may determine whether or not the vehicle V is in a tipped-over state based on the state information of the side stand acquired from the stand detection sensor. Also, if an attitude sensor (tilt sensor) that detects the attitude (tilt) of the vehicle V is provided on the vehicle V as one of the state detection sensors 14, the processing unit 21 may determine whether or not the vehicle V is in a tipped-over state based on the state information (attitude information of the vehicle V) acquired from the attitude sensor.

[0032] As described above, the monitoring device (information terminal 20) of this embodiment uses images (first images) obtained by the cameras 13a-13b (first imaging unit) of the imaging unit 13 capable of imaging the driver who is driving the vehicle V to monitor (detect, determine) abnormalities in the driver and / or the vehicle V. This makes it possible to accurately detect the abnormality with a simple configuration and processing.

[0033] <Second embodiment> In the above embodiment, the processing unit 21 (the determination unit 21b) may determine the type of abnormality based on the situation information acquired by the acquisition unit 21a from the photographing unit 13 and / or the state detection sensor 14, and may change the notification mode by the notification unit 23 according to the type of abnormality. For example, in the flowchart of FIG. 4, the type (mode) of the abnormality may be different between a case where the occurrence of an abnormality is determined by detecting a situation in which the driver is lying on the road in step S13 (hereinafter referred to as the first case) and a case where the occurrence of an abnormality is determined by detecting a situation in which the vehicle V is in a rollover state in step S19 (hereinafter referred to as the second case). Specifically, the abnormality in the first case is that the driver is lying on the road. Therefore, as an example, a notification mode may be taken in which a voice to request an ambulance is output to the surroundings of the vehicle V, or a call is made to the fire department to request an ambulance. On the other hand, the abnormality in the second case is that fuel may be leaking from the vehicle V. Therefore, a notification mode may be adopted in which a voice message is output to those around the vehicle V requesting that the vehicle V be moved away, or a call is made to the fire department to request a fire engine. In the second case, a voice message may be output to those around the vehicle V requesting that the ignition of the vehicle V be turned off.

[0034] <Third embodiment> In the above embodiment, an example has been described in which the information terminal 20 functions as a monitoring device that monitors abnormalities in the driver and / or the vehicle V. However, the in-vehicle device 10 may function as a monitoring device. In this case, the functions of the acquisition unit 21a and the determination unit 21b included in the processing unit 21 of the information terminal 20 may be provided in the processing unit 11 of the in-vehicle device 10. In addition, the function of the notification unit 23 of the information terminal 20 may also be provided in the in-vehicle device 10.

[0035] <Other embodiments> Furthermore, a program for realizing one or more functions described in the above embodiments can be supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device can read and execute the program. The present invention can also be realized in such an embodiment.

[0036] <Summary of the embodiment> 1. The monitoring device of the above embodiment A monitoring device (e.g., 20) for monitoring an abnormality of a driver and / or a vehicle (e.g., V), An acquisition means (e.g., 21a) for acquiring situation information indicating the situation of the driver and the vehicle; A determination means (e.g., 21b) that determines that the abnormality has occurred when a predetermined situation is detected based on the situation information acquired by the acquisition means; Equipped with The situation information includes a first image obtained by a first imaging unit (eg, 13a to 13b) capable of imaging the driver who is driving the vehicle. According to this embodiment, the occurrence of an abnormality in the driver and / or vehicle can be accurately detected (determined) with a simple configuration and processing.

[0037] 2. In the above embodiment, the predetermined situation includes a first situation in which the driver is not driving the vehicle; The determination means is capable of detecting the first situation based on the first image. According to this embodiment, by simply using the first image obtained by the first imaging unit capable of imaging the driver, the first situation in which the driver is not driving the vehicle can be accurately detected as a judgment item for determining whether an abnormality has occurred.

[0038] 3. In the above embodiment, The first situation includes a situation in which the driver is not gripping a steering wheel (eg, 31) and / or a situation in which the driver is not seated in a driver's seat (eg, 32) of the vehicle. According to this embodiment, by simply detecting the driver's seating status and / or the driver's grip on the steering wheel using the first image, the first situation in which the driver is not driving the vehicle can be accurately detected as a judgment item for determining whether an abnormality has occurred.

[0039] 4. In the above embodiment, The situation information includes a second image obtained by a second photographing unit (e.g., 13c to 13f) capable of photographing the surroundings of the vehicle, The predetermined situation further includes a second situation in which the driver is lying down; The determination means is capable of detecting the second situation based on the second image, and determines that the abnormality has occurred when the second situation is detected in addition to the first situation. According to this embodiment, by detecting a combination of multiple types of situations, abnormalities in the driver and / or vehicle can be accurately detected, and the driver's situation can also be detected based on the second image, making it possible to take appropriate measures (processing, notification) according to the driver's situation.

[0040] 5. In the above embodiment, The determining means determines that the driver is safe when the driver is not lying down in the second image. According to this embodiment, it is possible to appropriately take measures (processing, notification) according to the state of the driver detected based on the second image.

[0041] 6. In the above embodiment, The predetermined condition further includes a third condition in which an ignition of the vehicle is on; The determination means is capable of detecting the third situation based on the situation information. According to this embodiment, the vehicle situation can be detected with high accuracy, and appropriate measures (processing, notification) can be taken according to the situation.

[0042] 7. In the above embodiment, The judgment means is capable of detecting the third situation based on information indicating at least one of an image obtained by a camera (e.g., 13a to 13b) capable of photographing an ignition key (e.g., 33), the position of the ignition key (e.g., 33), and the state of a battery installed in the vehicle. According to this embodiment, the third situation in which the ignition of the vehicle is on can be detected with high accuracy.

[0043] 8. In the above embodiment, The determining means determines that the driver has exited the vehicle when the ignition is off. According to this embodiment, it is possible to accurately determine whether or not the driver has appropriately exited the vehicle, based on a combination of a plurality of types of detected situations.

[0044] 9. In the above embodiment, The predetermined situation further includes a fourth situation in which the vehicle is in a rollover state, The determining means determines that the abnormality has occurred when detecting the third situation and the fourth situation in addition to the first situation. According to this embodiment, the occurrence of an abnormality in the driver and / or the vehicle can be detected with high accuracy based on a combination of a plurality of types of detected situations.

[0045] 10. In the above embodiment, The determining means determines that the driver has forgotten to turn off the ignition if the vehicle is not in an overturned state. According to this embodiment, it is possible to recognize that the driver has forgotten to turn off the vehicle ignition, based on a combination of a plurality of types of detected situations.

[0046] 11. In the above embodiment, The system further includes a notification means (e.g., 23) for notifying the user of the abnormality when the determination means determines that the abnormality has occurred, The determining means determines the type of the abnormality based on the situation information acquired by the acquiring means, and changes the manner of notification by the notifying means depending on the type of the abnormality. According to this embodiment, the occurrence of an abnormality can be appropriately notified depending on the type of abnormality that has occurred in the driver and / or vehicle, making it possible to appropriately deal with (process) the abnormality.

[0047] 12. In the above embodiment, The vehicle is a saddle-type vehicle. According to this embodiment, it is possible to accurately detect the occurrence of an abnormality in a saddle type vehicle.

[0048] The invention is not limited to the above-described embodiment, and various modifications and variations are possible within the scope of the gist of the invention.

[0049] This application claims priority based on Japanese patent application No. 2021-049129, filed on March 23, 2021, the entire contents of which are incorporated herein by reference. [Explanation of symbols]

[0050] V: vehicle, 10: in-vehicle device, 13: image capture unit, 14: status detection sensor, 20: information terminal (monitoring device), 21: processing unit, 21a: acquisition unit, 21b: determination unit, 23: notification unit

Claims

1. A monitoring device that is mounted on a saddle-riding type vehicle and monitors an abnormality of a driver of the saddle-riding type vehicle and / or an abnormality of the saddle-riding type vehicle, an acquisition unit for acquiring situation information indicating a situation of the driver and the saddle riding type vehicle; a determination means for determining that the abnormality has occurred when a predetermined situation is detected based on the situation information acquired by the acquisition means; Equipped with the situation information includes a first image that is obtained by a first image capturing unit that is capable of capturing an image of the driver seated on a seat of the saddle riding type vehicle and that is provided in the saddle riding type vehicle, and a second image that is obtained by a second image capturing unit that is capable of capturing an image of the surroundings of the saddle riding type vehicle and that is provided in the saddle riding type vehicle, the predetermined situation includes a first situation in which the driver is not seated in the seat and a second situation in which the driver is lying on a road around the saddle riding type vehicle, The monitoring device is characterized in that, when the judgment means detects the first situation based on the first image acquired by the acquisition means and detects the second situation based on the second image acquired by the acquisition means, the judgment means judges that the abnormality is that the saddle-type vehicle has rolled over while the driver is driving the saddle-type vehicle.

2. 2. The monitoring device according to claim 1, wherein the first situation includes a situation in which the driver is not gripping a steering wheel.

3. The monitoring device according to claim 1 or 2, characterized in that the judgment means judges that the driver is safe when the first situation is detected based on the first image and the second situation is not detected based on the second image.

4. the predetermined situation further includes a third situation in which an ignition of the saddle riding type vehicle is on, and a fourth situation in which the saddle riding type vehicle is in a toppled state, The monitoring device according to any one of claims 1 to 3, characterized in that when the judgment means detects the third situation and the fourth situation based on the situation information acquired by the acquisition means instead of the second situation, it judges that a fall of the saddle-type vehicle has occurred while the driver is driving the saddle-type vehicle.

5. 5. The monitoring device according to claim 4, wherein the status information includes information indicating at least one of an image obtained by a camera capable of photographing an ignition key, a position of the ignition key, and a state of a battery mounted in the saddle-type vehicle.

6. 6. The monitoring device according to claim 4 or 5, wherein the determination means determines that, when the ignition is in an off position, the saddle-type vehicle has not fallen over while the driver is driving the saddle-type vehicle, and that the driver has dismounted from the saddle-type vehicle.

7. 7. The monitoring device according to claim 4, wherein the determination means determines that the driver has forgotten to turn off the ignition when the determination means detects the first situation and the third situation but does not detect the fourth situation.

8. a notification means for notifying an abnormality when the determination means determines that the abnormality has occurred, The monitoring device according to any one of claims 1 to 7, characterized in that the judgment means determines the type of the abnormality based on the situation information acquired by the acquisition means, and changes the notification mode by the notification means depending on the type of the abnormality.

9. 9. The monitoring device according to claim 1, wherein the first image capturing unit and the second image capturing unit are provided on the saddle-type vehicle.

10. A saddle-type vehicle comprising the monitoring device according to any one of claims 1 to 9.

11. An information terminal communicably connected to a saddle-type vehicle, 10. An information terminal comprising the monitoring device according to claim 1, and configured to acquire the situation information from the saddle-type vehicle.

12. 1. A monitoring method for monitoring an abnormality in a driver of a saddle-type vehicle and / or the saddle-type vehicle, comprising: an acquisition step of acquiring situation information indicating a situation of the driver and the saddle riding type vehicle; a determination step of determining that the abnormality has occurred when a predetermined situation is detected based on the situation information acquired in the acquisition step; Including, the situation information includes a first image that is obtained by a first image capturing unit that is capable of capturing an image of the driver seated on a seat of the saddle riding type vehicle and that is provided in the saddle riding type vehicle, and a second image that is obtained by a second image capturing unit that is capable of capturing an image of the surroundings of the saddle riding type vehicle and that is provided in the saddle riding type vehicle, the predetermined situation includes a first situation in which the driver is not seated in the seat and a second situation in which the driver is lying on a road around the saddle riding type vehicle, The monitoring method is characterized in that, in the judgment process, the first situation is detected based on the first image acquired in the acquisition process, and if the second situation is detected based on the second image acquired in the acquisition process, it is determined that the abnormality is that the saddle-type vehicle has rolled over while the driver is driving the saddle-type vehicle.

13. A program for causing a computer to execute each step of the monitoring method according to claim 12.

Citation Information

Patent Citations

  • Two-wheeler accident detecting device and notifying device

    JP2001030971A

  • Vehicle accident prevention system, computer program, method for controlling vehicle accident prevention, and power supply control device

    JP2010033409A

  • Collision avoidance support device

    JP2010073027A

  • Processor for traffic accident information

    JP2011123814A

  • Method and device for detecting fall of driver of two-wheeled vehicle

    JP2015107798A