Pedal misapplication prevention device

Abstract

To provide a pedal step-down mistake suppression device which suppresses pedal step-down mistake and does not disturb the concentration on driving.SOLUTION: A suppression device 1 comprises: a photographed image information acquisition unit 2 which acquires photographed image information from a foot camera 8 that photographs an accelerator pedal, a brake pedal, and a foot of a driver; a sign detection unit 3 which detects a sign of step-down mistake; and a photographed image information output unit 4 which outputs foot image information in such a mode that the driver can visually recognize the foot image information when the sign detection unit 3 detects the sign of step-down mistake. The suppression device 1 further comprises: a foot position information generation unit 6 which generates foot position information in the pedal operation of the driver based on the foot image information; and a distribution information generation unit 7 which generates distribution information of the foot position information. The sign detection unit 3 detects the sign of step-down mistake based on the foot position information and distribution information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a pedal misapplication prevention device. [Background technology]

[0002] Patent Document 1 (Utility Model Registration No. 3227177) discloses a technology to prevent accidents caused by accidentally stepping on the accelerator pedal instead of the brake pedal. Specifically, a display device is installed to display images of the accelerator pedal, the brake pedal, and the driver's feet. Summary of the Invention [Problem to be solved by the invention]

[0003] However, the configuration of Patent Document 1 has a problem in that the display device constantly displays images, making it difficult to concentrate on driving.

[0004] An object of the present disclosure is to provide a pedal erroneous application prevention device that prevents pedal erroneous application while taking into consideration not to interfere with concentration on driving. [Means for solving the problem]

[0005] According to the present disclosure, there is provided a pedal misapplication prevention device comprising: an image information acquisition unit that acquires image information from an imaging unit that images an accelerator pedal, a brake pedal, and a driver's feet; an indication detection unit that detects indications of pedal misapplication by the driver; and an image information output unit that, when the indication detection unit detects the indication, outputs the image information in a manner that allows the driver to view the image information. Effect of the Invention

[0006] According to the present disclosure, a pedal misapplication prevention device is provided that prevents pedal misapplication while being mindful of not interfering with the driver's concentration on driving. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a functional block diagram of a suppression device (first embodiment). [Diagram 2] 1 is a control flow of the suppression device (first embodiment). [Diagram 3] FIG. 11 is a diagram showing an example of an output from a captured image information output unit (first embodiment). [Figure 4] FIG. 11 is a diagram showing an example of an output from a captured image information output unit (first embodiment). [Diagram 5] FIG. 11 is a diagram showing an example of an output from a captured image information output unit (first embodiment). [Figure 6] FIG. 11 is a functional block diagram of a suppression device (second embodiment). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] (First embodiment) A first embodiment of the present disclosure will be described below with reference to Fig. 1 to Fig. 5. The suppression device 1 is a specific example of a pedal erroneous depression suppression device. The suppression device 1 suppresses a vehicle driver from erroneously depressing the accelerator pedal when he or she intends to depress the brake pedal.

[0009] The suppression device 1 includes a CPU1a (Central Processing Unit) as a central processing unit, a readable and writable RAM1b (Random Access Memory), and a read-only ROM1c (Read Only Memory). The CPU1a reads and executes a control program stored in the ROM1c, causing hardware such as the CPU1a to function as various functional units.

[0010] The various functional sections include an image information acquisition section 2, a sign detection section 3, an image information output section 4, a pedal operation detection section 5, a foot position information generation section 6, and a distribution information generation section .

[0011] Furthermore, a foot camera 8 and a display 9 are connected to the restraint device 1 .

[0012] The foot camera 8 is a specific example of an imaging unit. The foot camera 8 captures the accelerator pedal, brake pedal, and driver's feet of the vehicle. The driver's feet are typically the right foot of the driver involved in pedal operation, and are the part from the ankle forward. In the embodiment, the operation of the right foot is described, but the invention can also be applied to brake pedal operation by the left foot. The foot camera 8 is typically attached facing downward around the dashboard so that the accelerator pedal, brake pedal, and driver's feet can be simultaneously captured. However, instead of this, a camera mounted on a drive recorder may be used as the foot camera 8. The camera mounted on a drive recorder is typically provided on the windshield or dashboard, and captures the front of the vehicle. Therefore, when a camera mounted on a drive recorder is used as the foot camera 8, multiple mirrors may be installed in the vehicle interior so that the accelerator pedal, brake pedal, and driver's feet are within the angle of view of the camera mounted on the drive recorder, and mirror reflection may be used. Similarly, a camera for a driver monitoring system or a camera for capturing images inside the vehicle interior may be used.

[0013] The display 9 is a HUD (Head-Up Display) that displays an image as a virtual image, or a projector that displays an image as a real image. The display 9 may be a display mounted on a navigation device, a display mounted on an instrument panel, or a display unit of a portable terminal owned by a passenger that is used via wireless communication or the like.

[0014] The captured image information acquiring unit 2 acquires captured image information from the foot camera 8. Hereinafter, the captured image information acquired by the captured image information acquiring unit 2 from the foot camera 8 is also referred to as foot image information.

[0015] The sign detection unit 3 detects signs of pedal misapplication by the driver. "Pedal misapplication by the driver" means that the driver mistakenly presses the accelerator pedal while intending to press the brake pedal. Hereinafter, "signs of pedal misapplication by the driver" will be simply referred to as pedal misapplication signs.

[0016] When the sign detection unit 3 detects a sign of mispeduction, the captured image information output unit 4 outputs the feet image information in a manner that allows the driver to visually recognize the feet image information. The captured image information output unit 4 typically outputs the feet image information to a display 9. Fig. 3 shows a specific example of a manner in which the captured image information output unit 4 outputs the feet image information.

[0017] Returning to FIG. 1, the pedal operation detection unit 5 detects the pedal operation by the driver via an in-vehicle network (not shown) or the like. Here, the "pedal operation" includes the driver's operation of depressing the brake pedal with his / her foot and the driver's operation of depressing the accelerator pedal with his / her foot. The pedal operation detection unit 5 typically detects the driver's operation of depressing the brake pedal with his / her foot using a mechanical switch or a brake pedal position sensor mounted on the brake pedal. Also, the pedal operation detection unit 5 typically detects the driver's operation of depressing the accelerator pedal with his / her foot using an accelerator position sensor mounted on the accelerator pedal.

[0018] The foot position information generating unit 6 generates foot position information when the driver operates the pedal based on the foot image information. Specifically, the foot position information generating unit 6 detects the driver's feet using machine learning object detection techniques such as Faster R-CNN (Region Based Convolutional Neural Networks), YOLO (You only look once), and SSD (Single Shot Multibox Detector) based on the foot image information acquired by the captured image information acquiring unit 2 when the pedal operation detecting unit 5 detects the pedal operation, generates foot position information when the driver operates the pedal, and stores the generated foot position information in the RAM 1b. The foot position information is coordinate information corresponding to the position of the foot in the foot image information. The foot position information is typically coordinate information of a specific reference point of a bounding box generated when the foot position information generating unit 6 detects the driver's feet using the object detection technique. The specific reference point is typically the upper left corner of the bounding box. The foot position information generating unit 6 generates accelerator foot position information, which is foot position information when the driver operates the accelerator pedal, and brake foot position information, which is foot position information when the driver operates the brake pedal.

[0019] The distribution information generator 7 generates distribution information of the foot position information and stores the generated distribution information in the RAM 1b. The distribution information includes accelerator distribution information, which is distribution information of the accelerator foot position information, and brake distribution information, which is distribution information of the brake foot position information.

[0020] The accelerator distribution information includes the average and standard deviation of the accelerator foot position information. When the accelerator foot position information is coordinate information composed of ordinate values ​​and abscissa values, the accelerator distribution information includes the average and standard deviation of the ordinate values ​​and the average and standard deviation of the abscissa values.

[0021] Similarly, the brake distribution information includes the mean and standard deviation of the brake foot position information. If the brake foot position information is coordinate information consisting of ordinate and abscissa values, the brake distribution information includes the mean and standard deviation of the ordinate values ​​and the mean and standard deviation of the abscissa values.

[0022] The distribution information generating unit 7 stores the generated accelerator distribution information and brake distribution information in the RAM 1b.

[0023] Then, the sign detection unit 3 detects a sign of a misstep based on the distribution information.

[0024] Specifically, when the pedal operation detection unit 5 detects the operation of the accelerator pedal by the driver, the sign detection unit 3 detects a sign of misapplication based on the accelerator foot position information generated by the foot position information generation unit 6 and the accelerator distribution information stored in RAM 1b. For example, the sign detection unit 3 detects a sign of misapplication when the abscissa value of the accelerator foot position information deviates from the average of the abscissa values ​​of the accelerator distribution information by more than twice the standard deviation.

[0025] Similarly, when the pedal operation detection unit 5 detects the operation of the brake pedal by the driver, the sign detection unit 3 detects a sign of misapplication based on the brake foot position information generated by the foot position information generation unit 6 and the brake distribution information stored in RAM 1b. For example, the sign detection unit 3 detects a sign of misapplication when the abscissa value of the brake foot position information deviates from the average of the abscissa values ​​of the brake distribution information by more than twice the standard deviation.

[0026] In short, the sign detection unit 3 detects a sign of misapplication when the position of the driver's foot when the driver operates the pedal is significantly different from the usual position. The position of the driver's foot and the degree of deviation may be determined and updated using a known method such as machine learning. Also, for example, if the ordinate value of the foot position is significantly different from the average of the ordinate values ​​of the distribution information of the accelerator or brake, it is possible that the driver is not familiar with the vehicle and may miss pressing the pedal, and such a case may also be detected as a sign of misapplication.

[0027] Next, the operation of the suppression device 1 will be described with reference to FIG.

[0028] S100: First, the captured image information acquisition unit 2 acquires foot image information from the foot camera 8.

[0029] S110: Next, the pedal operation detection unit 5 detects the pedal operation by the driver.

[0030] S120: Next, the foot position information generating unit 6 generates foot position information when the driver operates the pedal based on the foot image information, and stores the generated foot position information in the RAM 1b.

[0031] S130: Next, the suppression device 1 determines whether the foot position information generating unit 6 has repeated the generation of foot position information a predetermined number of times. If NO in step S130, the suppression device 1 returns the process to step S100. If YES in step S130, the suppression device 1 advances the process to step S140.

[0032] S140: Next, the distribution information generating unit 7 generates distribution information of the foot position information and stores the generated distribution information in the RAM 1b. The generation of the distribution information of the foot position information is not limited to the above, and the pedal operation of the driver may be continuously detected and periodically updated, or the driver may be identified using a publicly known person authentication technology (not shown) and information may be stored and used for each driver.

[0033] S150: Next, the captured image information acquisition unit 2 acquires foot image information from the foot camera 8 .

[0034] S160: Next, the pedal operation detection unit 5 detects the pedal operation by the driver.

[0035] S170: Next, the foot position information generating unit 6 generates foot position information when the driver operates the pedal based on the foot image information, and stores the generated foot position information in the RAM 1b.

[0036] S180: Next, the sign detection unit 3 detects a sign of misstepping based on the distribution information. If a sign of misstepping is detected, the sign detection unit 3 proceeds to step S190. On the other hand, if the sign detection unit 3 does not detect a sign of misstepping, the sign detection unit 3 returns the process to step S150.

[0037] S190: Next, the captured image information output unit 4 outputs the feet image information to the display 9 as shown in FIG.

[0038] S200: Next, the suppression device 1 determines whether the accelerator pedal is operated. If the answer is YES in step S200, the suppression device 1 advances the process to step S210. On the other hand, if the answer is NO in step S200, the suppression device 1 advances the process to step S230.

[0039] S210: Next, the suppression device 1 determines whether the brake pedal is operated. If the result of step S210 is YES, the captured image information output unit 4 stops outputting the foot image information (S220). On the other hand, if the result of step S210 is NO, the suppression device 1 repeats step S210.

[0040] S230: Next, the suppression device 1 determines whether a predetermined time has elapsed since the determination in step S200. If the result in step S230 is YES, the captured image information output unit 4 stops outputting the foot image information (S220). On the other hand, if the result in step S230 is NO, the suppression device 1 repeats step S230.

[0041] The first embodiment has been described above, and the above embodiment has the following features.

[0042] That is, the suppression device 1 (pedal misapplication suppression device) includes an image information acquisition unit 2 that acquires image information from a foot camera 8 (imaging unit) that captures images of the accelerator pedal, the brake pedal, and the driver's feet, a sign detection unit 3 that detects signs of misapplication, and an image information output unit 4 that outputs foot image information in a manner that allows the driver to visually recognize the foot image information when the sign detection unit 3 detects signs of misapplication. With the above configuration, a suppression device 1 is realized that suppresses pedal misapplication while taking care not to interfere with concentration on driving.

[0043] The suppression device 1 further includes a foot position information generating unit 6 that generates foot position information when the driver operates the pedal based on the foot image information, and a distribution information generating unit 7 that generates distribution information of the foot position information. The sign detecting unit 3 detects a sign of mispeduction based on the foot position information and the distribution information. That is, for example, when the driver twists his / her body to directly check the rear with his / her eyes, when there is a foreign object at the driver's feet that prevents the driver from operating the pedal as usual, or when the driver's clothes are disheveled that prevents the driver from operating the pedal as usual, the driver operates the pedal with a foot position different from usual, so there is a high possibility that the driver will mispeduce the pedal. Therefore, according to the above configuration, for example, when the driver operates the pedal with a foot position different from usual, it is possible to detect a sign of mispeduction.

[0044] In step S160, the pedal operation detection unit 5 detects the pedal operation by the driver. Specifically, the pedal operation detection unit 5 detects the operation of the accelerator pedal by the driver and the operation of the brake pedal by the driver. However, instead of this, the pedal operation detection unit 5 may detect only the operation of the accelerator pedal by the driver in step S160.

[0045] Furthermore, the captured image information output unit 4 outputs the foot image information until the driver changes his / her foot from the accelerator pedal to the brake pedal (steps S200, S210, S220). According to the above configuration, the foot image information continues to be output until the pedal misapplication is resolved, so that the pedal misapplication can be effectively suppressed.

[0046] The above first embodiment can be modified as follows.

[0047] That is, the captured image information output unit 4 may output the foot image information after processing the foot image information so that the entire foot image has a reddish hue by doubling the R value and halving the G and B values ​​of the RGB component information of the foot image information. This makes it possible to effectively alert the driver.

[0048] Also, as shown in FIG. 4, the captured image information output unit 4 may output the foot image information after image processing the foot image information so that the driver's right foot is emphasized. Specifically, the captured image information output unit 4 may image process the foot image information so that a colored transparent figure 4a is displayed superimposed on the position of the right foot that operates the pedal. Also, the captured image information output unit 4 may image process the foot image information so as to relatively increase the luminance of the area of ​​the right foot that operates the pedal. Also, the captured image information output unit 4 may image process the foot image information so as to display a message, symbol, figure, etc. indicating that a sign of pedal misapplication has been detected around the area of ​​the right foot that operates the pedal by superimposing it. This makes it possible to effectively alert the driver.

[0049] 5, the captured image information output unit 4 may output the foot image information after image processing the foot image information so that a figure 4b is displayed to prompt the driver to shift from the accelerator pedal to the brake pedal when the driver is operating the accelerator pedal. This makes it possible to effectively call the driver's attention. The captured image information output unit 4 may output the foot image information so that the figure 4b is displayed in a flashing manner.

[0050] Second embodiment Next, a second embodiment of the present disclosure will be described with reference to FIG.

[0051] As shown in Fig. 6, a front camera 10 is connected to the suppression device 1. The front camera 10 is a camera that captures an image in front of the vehicle. The captured image information acquisition unit 2 acquires front image information as captured image information from the front camera 10. The front camera 10 is not limited to this, and may be a combination of a front camera and a rear camera that capture images in the traveling direction of the vehicle, or may be composed of multiple cameras facing the front, rear, left and right around the vehicle, or a camera that captures images of the entire circumference using a fisheye lens or the like.

[0052] In this embodiment, the sign detection unit 3 detects signs of misstepping when the driver does not press the accelerator pedal a predetermined time after the vehicle driven by the driver is stopped as the lead vehicle at an intersection where traffic lights are installed and the traffic light state changes from a no-go state that prohibits the vehicle from starting to a no-go state that allows the vehicle to start.

[0053] Specifically, the sign detection unit 3 detects that the vehicle is stopped as the leading vehicle at an intersection where a traffic light is installed based on a known image recognition method from the forward image information. The sign detection unit 3 also detects that the indication state of the traffic light has switched from a no-go state that prohibits the vehicle from starting to a proceeding allowance state that allows the vehicle to start, based on the forward image information. Then, when the vehicle driven by the driver is stopped as the leading vehicle at an intersection where a traffic light is installed, and the driver does not press the accelerator pedal even after a predetermined time has elapsed since the indication state of the traffic light switched from a no-go state that prohibits the vehicle from starting to a proceeding allowance state that allows the vehicle to start, the sign detection unit 3 detects a misstep sign when the driver does not press the accelerator pedal.

[0054] That is, for example, when the timing of an operation that should be performed is delayed, such as when the driver drops something at his / her feet or is confused about the operation of the shift lever and does not press the accelerator pedal for a predetermined time, the risk of misstepping increases. The predetermined time is, for example, about several seconds, but it is not limited to this and may be a time set by learning the tendency of the time it takes to operate the pedal from the driver's daily operation timing and surrounding conditions. Therefore, with the above configuration, the risk of misstepping can be effectively reduced.

[0055] The above-described embodiments may be implemented in combination with each other, and may be modified, for example, as follows.

[0056] For example, the sign detection unit 3 may detect a sign of misstepping when it detects that the clothing around the feet is loosened to a predetermined extent based on the foot image information. "The clothing around the feet is loosened to a predetermined extent" refers to a state in which the excess part of the shoelace is not fastened, the shoelace is untied, the hook-and-loop fastener of the shoe is not fastened, and other states in which the part protruding from the outline of the shoe body is extensive (for example, several centimeters or more). In addition, "The clothing around the feet is loosened to a predetermined extent" refers to a state in which the clothing such as pants or a skirt is spread out to a predetermined extent or more around the pedal, the outline of the shoe body cannot be recognized, and the shoe body cannot be detected by the object detection technology, for example.

[0057] Furthermore, the sign detection unit 3 may quantitatively measure the driver's fatigue level and driving posture, and detect a sign of pedal misapplication based on the measurement results.

[0058] Also, the sign detection unit 3 may detect the position of a floor mat of the driver's seat based on the foot image information, and detect a sign of a misstep when the position of the floor mat is not within a predetermined range.

[0059] Furthermore, the sign detection unit 3 may detect a sign of misapplication when it detects that there is a foreign object at the feet of the driver's seat based on the feet image information.

[0060] Furthermore, the sign detection unit 3 may detect a sign of misapplication when the distance from the vehicle to an obstacle ahead when the vehicle starts moving forward is equal to or less than a predetermined value.

[0061] Furthermore, the sign detection unit 3 may detect a sign of misapplication when the distance from the vehicle to an obstacle behind the vehicle when the vehicle starts moving backward is equal to or shorter than a predetermined value.

[0062] Furthermore, the sign detection unit 3 may be configured to detect a sign of mispeduction when the vehicle starts moving forward while parked facing forward.

[0063] Furthermore, the sign detection unit 3 may be configured to detect a sign of mispeduction when the vehicle starts moving backwards while parked facing backwards.

[0064] In addition, the captured image information output unit 4 may detect the accelerator pedal, brake pedal, and the driver's feet based on the foot image information, and output the foot image information after image processing so that an accelerator pedal figure is displayed at the accelerator pedal position, a brake pedal figure is displayed at the brake pedal position, and foot figures are displayed at the driver's foot positions.

[0065] The suppression device 1 may further include an alarm sound output unit that outputs an alarm sound when the sign detection unit 3 detects a sign of misstepping.

[0066] The suppression device 1 may further include a vibration unit that vibrates the steering wheel or the driver's seat when the sign detection unit 3 detects a sign of mispeduction. The vibration unit includes a vibration motor and a vibration control unit. When the sign detection unit 3 detects a sign of mispeduction, the vibration control unit drives the vibration motor to vibrate the steering wheel or the driver's seat.

[0067] In the above example, the program can be stored and provided to the computer using various types of non-transitory computer readable media. The non-transitory computer readable media includes various types of tangible storage media. Examples of the non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives) and magneto-optical recording media (e.g., magneto-optical disks). Examples of the non-transitory computer readable media further include CD-ROM (Read Only Memory), CD-R, CD-R / W, and semiconductor memory (e.g., mask ROM. Examples of the non-transitory computer readable media further include PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, and RAM (random access memory). The program may also be provided to the computer by various types of transitory computer readable media. Examples of the transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire or an optical fiber, or via a wireless communication path.

[0068] (Appendix 1) an image information acquiring step of acquiring image information from an imaging unit that captures images of an accelerator pedal, a brake pedal, and a driver's feet; a sign detection step of detecting a sign of pedal misapplication by the driver; an image information output step of outputting the image information in a manner that allows the driver to visually recognize the image information when the sign is detected in the sign detection step; Including, A method for preventing pedal misapplication.

[0069] (Appendix 2) A program that causes a computer to execute the pedal misapplication prevention method described in appendix 1. [Explanation of symbols]

[0070] 1 Suppressor 2. Image information acquisition unit 3. Predictor Detection Unit 4. Image information output section 4a Colored transparent figures 4b Shapes 5 Pedal operation detector 6 Foot position information generation section 7 Distribution information generation section 8. Foot Camera 9. Display 10. Front Camera

Claims

1. an image information acquiring unit that acquires image information from an imaging unit that captures images of an accelerator pedal, a brake pedal, and a driver's feet; a foot position information generating unit that generates foot position information, which is a position of the foot of the driver when operating a pedal, within the captured image information; a distribution information generating unit that generates distribution information including an average value of the foot position information; a sign detection unit that detects a sign of pedal erroneous operation by the driver based on the foot position information and an average value of the distribution information; an image information output unit that outputs the image information in a manner that allows the driver to visually recognize the image information when the sign detection unit detects the sign; Equipped with Pedal misapplication prevention device.

2. An image information acquisition unit that acquires image information from an imaging unit that images an accelerator pedal, a brake pedal, and a driver's feet; a foot position information generating unit that generates foot position information, which is a position of the foot of the driver when operating a pedal, within the captured image information; a distribution information generating unit that generates distribution information including an average value of the foot position information; a sign detection unit that detects a sign of pedal erroneous operation by the driver based on the foot position information and an average value of the distribution information; at least one of a warning sound output unit that outputs a warning sound when the sign detection unit detects the sign, and a vibration unit that vibrates a predetermined location when the sign detection unit detects the sign; Equipped with Pedal misapplication prevention device.

3. An image information acquisition unit that acquires image information from an imaging unit that images an accelerator pedal, a brake pedal, and a driver's feet; a sign detection unit that detects a sign of pedal misapplication by the driver; an image information output unit that outputs the image information in a manner that allows the driver to visually recognize the image information when the sign detection unit detects the sign; Equipped with the sign detection unit detects this as a sign of pedal misapplication when the driver does not depress the accelerator pedal even after a predetermined time has elapsed since the indication state of the traffic light changes from a no-go state that prohibits the vehicle from starting to a go-permit state that permits the vehicle to start, while the vehicle driven by the driver is stopped as a leading vehicle at an intersection where a traffic light is installed. Pedal misapplication prevention device.

4. the captured image information output unit outputs the captured image information so as to emphasize a foot position of the driver. The pedal misapplication prevention device according to claim 1 or 3.

5. the captured image information output unit outputs the captured image information until the driver changes his / her foot position from the accelerator pedal to the brake pedal. The pedal misapplication prevention device according to claim 1 or 3.

Images