In-vehicle device and method for controlling in-vehicle device

Abstract

To provide an on-vehicle device that can sense wrong operation by a driver to an accelerator with good accuracy and notify of the wrong operation, and a control method of an on-vehicle device.SOLUTION: An on-vehicle device according to one aspect of an embodiment comprises a control part. The control part detects motion of a foot part of a driver on the basis of an image of a vehicle interior having a condition of the vehicle interior of a vehicle photographed. The control part determines whether or not the vehicle is in a vehicle start- prohibited situation where the vehicle is prohibited from starting, on the basis of situation information showing a situation of the vehicle. The control part determines whether or not the vehicle is moving, on the basis of movement information showing a movement state of the vehicle. When determining that the detected motion of the foot part is accelerator motion operating the accelerator and that the vehicle is in the vehicle start-prohibited situation and when determining that the vehicle is moving, the control part senses that the motion of the foot part is a wrong operation to the accelerator and notifies of the wrong operation.SELECTED DRAWING: Figure 1A

Description

[Technical Field]

[0001] The present invention relates to an in-vehicle device and a control method for an in-vehicle device. [Background technology]

[0002] Conventionally, a technique for estimating the position of the toes of a person seated in a seat inside a vehicle has been proposed (see, for example, Patent Document 1). In the conventional technique, the position of the toes is estimated from an image of the interior of the vehicle captured by an interior camera of the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-068059 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, there has been a demand for detecting and notifying an erroneous operation of the accelerator by a driver (for example, pressing the wrong pedal). However, conventional technologies have room for improvement in terms of accurately detecting and notifying an erroneous operation of the accelerator by a driver.

[0005] The present invention has been made in view of the above, and has an object to provide an in-vehicle device and a control method for an in-vehicle device that can accurately detect and notify an erroneous operation of an accelerator by a driver. [Means for solving the problem]

[0006] In order to solve the above problems and achieve the object, the present invention provides an in-vehicle device having a control unit. The control unit detects the movement of the driver's feet based on an interior image captured of the interior of the vehicle. The control unit determines whether the vehicle is in a start prohibition situation in which starting of the vehicle is prohibited based on situation information indicating the situation of the vehicle. The control unit determines whether the vehicle is moving based on movement information indicating the movement state of the vehicle. If the detected foot movement is an accelerator movement to operate an accelerator, and it is determined that the vehicle is in the start prohibition situation and that the vehicle is moving, the control unit detects and notifies that the foot movement is an erroneous operation of the accelerator. [Effects of the Invention]

[0007] According to the present invention, an erroneous accelerator operation by a driver can be accurately detected and notified. [Brief explanation of the drawings]

[0008] [Figure 1A] FIG. 1A is a diagram illustrating an overview of a control method according to an embodiment. [Figure 1B] FIG. 1B is a diagram showing an overview of a control method according to an embodiment. [Figure 1C] FIG. 1C is a diagram illustrating an overview of a control method according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of a vehicle system including an on-board device. [Figure 3] FIG. 3 is a diagram illustrating an example of accelerator operation information. [Figure 4] FIG. 4 is a diagram showing an example of notification of an erroneous accelerator operation. [Figure 5] FIG. 5 is a flowchart showing a processing procedure executed by the in-vehicle device. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] The following describes in detail an embodiment of an in-vehicle device and a control method for an in-vehicle device disclosed in the present application with reference to the accompanying drawings. Note that the present invention is not limited to the following embodiments.

[0010] <Outline of control method for in-vehicle device> First, an outline of a control method for an in-vehicle device according to an embodiment will be described below with reference to Figures 1A to 1C. Figures 1A to 1C are diagrams showing an outline of a control method according to an embodiment.

[0011] 1A, the in-vehicle device 10 is mounted on a vehicle C such as an automobile. The in-vehicle device 10 may be, for example, a drive recorder.

[0012] The in-vehicle device 10 includes an in-vehicle camera 11. The in-vehicle camera 11 captures an image of an interior C1 of the vehicle C. The in-vehicle device 10 may also include an exterior camera 12 (see FIG. 2) that captures an image of an exterior of the vehicle C.

[0013] The in-vehicle device 10 detects the movement of the foot Da of the driver D based on the in-vehicle image captured by the in-vehicle camera 11 (step S1). For example, the in-vehicle device 10 analyzes the in-vehicle image to detect the foot Da (more specifically, the thigh Da1) of the driver D, and detects the movement of the detected foot Da (thigh Da1).

[0014] Here, the in-vehicle image will be described with reference to Fig. 1B. Fig. 1B shows an in-vehicle image A captured by an in-vehicle camera 11, and more specifically, an in-vehicle image A showing the interior C1 of a vehicle C.

[0015] 1B, the in-vehicle image A includes the feet Da of the driver D of the vehicle, and more specifically, the thighs Da1 of the driver D. In the in-vehicle device 10 according to this embodiment, the movement of the feet Da is detected, thereby detecting the accelerator operation of the driver D operating the accelerator 100 (see FIG. 1A).

[0016] More specifically, the position of the foot Da of the driver D is different when the driver D is operating the brake and when the driver D is operating the accelerator 100. That is, when the foot Da (for example, the thigh Da1) shifts from a state in which the foot Da is operating (pressing) the brake to a state in which the foot Da is operating (pressing) the accelerator 100, the position of the foot Da moves from the brake side to the accelerator 100 side.

[0017] It is assumed here that the brake is located on the left side as viewed from the driver D (i.e., the right side in FIG. 1B), and the accelerator 100 is located on the right side as viewed from the driver D (i.e., the left side in FIG. 1B). In addition, in FIG. 1B, the foot Da (thigh Da1) in the state where the brake is being operated is shown by a solid line, and the foot Da (thigh Da1) in the state where the accelerator 100 is being operated is shown by a dashed line.

[0018] Therefore, in the in-vehicle device 10 according to this embodiment, when a movement of the foot Da such that the foot Da moves from a position for braking to a position for accelerator operation is detected based on the in-vehicle image A, such a movement of the foot Da is detected as an accelerator operation. Note that even if the foot Da has not moved from the position for braking, the detection of an accelerator operation by simply detecting a state in which the foot Da is in the position for accelerator operation is also included.

[0019] Continuing with the explanation of FIG. 1A, the in-vehicle device 10 determines whether or not the vehicle C is in a start prohibition situation (step S2). A start prohibition situation is a situation in which the start of the vehicle C is prohibited. As an example, a start prohibition situation is a situation in which the display state of a traffic light present in the traveling direction of the vehicle C is a display state instructing the vehicle C to stop (e.g., a red light), or a situation in which the distance from the vehicle C to an object present around the vehicle C (e.g., a vehicle ahead or a wall) is relatively short, but these are merely examples and are not limited thereto. The determination of a start prohibition situation is made based on, for example, an outside-vehicle image captured by the outside-vehicle camera 12 (see FIG. 2), which will be described later.

[0020] Then, when the in-vehicle device 10 determines that the detected movement of the foot Da is an accelerator operation and that the vehicle C is in a start-prohibited state (step S3), it detects that the detected movement of the foot Da is an erroneous operation of the accelerator 100 (step S4). That is, since the driver D is operating the accelerator despite the vehicle C being in a state where it is not permitted to start (a start-prohibited state), the in-vehicle device 10 detects the movement of the foot Da as an erroneous operation of the accelerator 100.

[0021] In this way, in this embodiment, by using the movement of the driver D's foot Da detected based on the in-vehicle image A and the situation of the vehicle C (whether or not a starting prohibition situation exists), it is possible to accurately detect the driver D's erroneous operation of the accelerator 100.

[0022] Next, when the in-vehicle device 10 detects that the movement of the foot Da is an erroneous operation of the accelerator, it notifies the driver of the erroneous operation of the accelerator (step S5). In this way, in this embodiment, by notifying the driver of the erroneous operation of the accelerator, it is possible to make the driver D aware of the erroneous operation of the accelerator 100. The notification of the erroneous operation of the accelerator is made, for example, via the output unit 14 (see FIG. 2) such as a speaker or a display of the in-vehicle device 10, but is not limited to this.

[0023] Furthermore, the in-vehicle device 10 according to this embodiment may further detect and notify an erroneous operation of the accelerator 100 depending on the moving state of the vehicle C. For example, the in-vehicle device 10 may detect and notify an erroneous operation of the accelerator 100 when the vehicle C is moving (more precisely, when the vehicle C starts to move (starts moving)). This allows for accurate detection and notification of an erroneous operation of the accelerator 100 by the driver D.

[0024] Specifically, the in-vehicle device 10 determines whether the vehicle C is moving based on movement information indicating the movement state of the vehicle C. As the movement information, for example, the in-vehicle image A can be used.

[0025] Here, the in-vehicle image A used as movement information will be described with reference to Fig. 1C. As shown in Fig. 1C, the in-vehicle image A may include an exterior view of the vehicle C through the window 111 of the driver's seat 110 or the window 121 of the passenger seat 120. Therefore, the in-vehicle device 10 determines whether the vehicle C is moving based on the exterior view of the vehicle C in the in-vehicle image A.

[0026] For example, outside the vehicle C, there are various objects that become feature points when the in-vehicle image A is analyzed, such as the roadway or guardrails, signs, lanes, sidewalks, and advertisements provided along the roadway. Therefore, the in-vehicle device 10 can analyze the in-vehicle image A (for example, by optical flow analysis) to detect feature points of objects present outside the vehicle C, and determine that the vehicle C is moving when such feature points move. Note that, although the above example shows the use of the in-vehicle image A as movement information, the present invention is not limited to this, and other objects may also be used, such as an image outside the vehicle or a signal transmitted from the vehicle C (for example, a signal indicating the vehicle speed or the operation amount of the accelerator 100).

[0027] Then, when the in-vehicle device 10 determines that the detected movement of the foot Da is an accelerator operation, that the vehicle C is in a start-prohibited state, and that the vehicle C is moving, it detects and notifies that the movement of the foot Da is an erroneous operation of the accelerator 100. That is, even though the vehicle C is in a state where it should not start (a start-prohibited state), the driver D is operating the accelerator, and furthermore the vehicle C is moving (has started moving), so the in-vehicle device 10 detects and notifies that the movement of the foot Da is an erroneous operation of the accelerator 100.

[0028] In this way, the in-vehicle device 10 can accurately detect and notify the driver D of any erroneous operation of the accelerator 100 by using the movement of the driver D's foot Da detected based on the in-vehicle image A and the status of the vehicle C (whether or not a starting prohibition situation exists and whether or not the vehicle C is moving).

[0029] Furthermore, as described above, the in-vehicle device 10 can detect an erroneous operation of the accelerator 100 using the in-vehicle image A from the in-vehicle camera 11, and therefore can be easily attached to a completed vehicle C after completion, for example.

[0030] <Vehicle system configuration> Next, the configuration of a vehicle system including the on-vehicle device 10 according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of a vehicle system 1 including the on-vehicle device 10. Note that in the block diagram of Fig. 2, only the components necessary for explaining the features of the present embodiment are shown as functional blocks, and descriptions of general components are omitted.

[0031] In other words, each component shown in the block diagram of Figure 2 is a functional concept and does not necessarily have to be physically configured as shown. For example, the specific form of distribution and integration of each functional block is not limited to that shown, and all or part of it can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0032] As shown in Fig. 2, the vehicle system 1 includes an on-board device 10 and a vehicle sensor group 40. The vehicle sensor group 40 is mounted on the vehicle and outputs vehicle information related to the vehicle. Specifically, the vehicle sensor group 40 includes various sensors that output signals indicating various states (conditions) related to the vehicle. For example, the vehicle sensor group 40 includes a vehicle speed sensor 41, an accelerator sensor 42, a brake sensor 43, a travel distance sensor 44, and a shift sensor 45.

[0033] The vehicle speed sensor 41 outputs a signal indicating the vehicle speed. The accelerator sensor 42 outputs a signal indicating the amount of operation of the accelerator (more precisely, the accelerator pedal). The brake sensor 43 outputs a signal indicating the amount of operation of the brake (more precisely, the brake pedal). The travel distance sensor 44 outputs a signal indicating the starting distance of the vehicle. The shift sensor 45 outputs a signal indicating the position of the shift lever. Each signal from the vehicle sensor group 40, including the above-mentioned vehicle speed sensor 41, is transmitted to the in-vehicle device 10.

[0034] The vehicle sensor group 40 is not limited to the above-described various sensors, and may include other types of sensors, such as a steering angle sensor, in addition to or instead of the various sensors. In the above description, the signals of the vehicle sensor group 40 are transmitted to the in-vehicle device 10, but this is not limiting, and for example, the signals of the vehicle sensor group 40 may not be transmitted to the in-vehicle device 10.

[0035] The in-vehicle device 10 includes an in-vehicle camera 11, an outside-vehicle camera 12, an acceleration sensor 13, an output unit 14, a control unit 20, and a storage unit 30.

[0036] The in-vehicle camera 11 is installed at an appropriate position on the vehicle and captures, for example, images of the interior of the vehicle. The exterior camera 12 is installed at an appropriate position on the vehicle and captures, for example, images of the exterior of the vehicle. In other words, the exterior camera 12 captures images of the periphery of the vehicle, such as the area in front of the vehicle. The in-vehicle camera 11 outputs the captured images of the interior of the vehicle to the control unit 20, and the exterior camera 12 outputs the captured images of the exterior of the vehicle to the control unit 20.

[0037] The in-vehicle camera 11 and the out-vehicle camera 12 are, for example, cameras equipped with a lens and an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), but are not limited to this.

[0038] The acceleration sensor 13 detects the acceleration acting on the vehicle and outputs a signal indicating the detected acceleration to the control unit 20. The output unit 14 outputs various information, such as a notification when an erroneous accelerator operation by the driver is detected. For example, the output unit 14 includes an audio output unit such as a speaker and a display unit such as a display, and outputs various information, such as notifications, to the driver of the vehicle.

[0039] The control unit 20 includes an acquisition unit 21, a detection unit 22, a determination unit 23, a detection unit 24, and a notification unit 25, and includes, for example, a computer having a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), data flash, input / output ports, etc., and various circuits.

[0040] The CPU of the computer functions as the acquisition unit 21, detection unit 22, determination unit 23, sensing unit 24, and notification unit 25 of the control unit 20, for example, by reading and executing a program stored in the ROM.

[0041] In addition, at least one or all of the acquisition unit 21, detection unit 22, judgment unit 23, detection unit 24 and notification unit 25 of the control unit 20 can be configured using hardware such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0042] The storage unit 30 is configured with a storage device such as a non-volatile memory, a data flash, or a hard disk drive. The storage unit 30 can store accelerator operation information 31, information on various programs, etc. The in-vehicle device 10 may also acquire the above-mentioned programs and various information via another computer or portable recording medium connected via a wired or wireless network.

[0043] The accelerator operation information 31 is information relating to the accelerator operation of the driver. For example, the accelerator operation information 31 includes information that serves as a criterion for determining that the movement of the driver's foot is an accelerator operation. Here, the accelerator operation information 31 will be explained using FIG. 3. FIG. 3 is a diagram showing an example of the accelerator operation information 31.

[0044] As shown in FIG. 3, accelerator operation information 31 includes items such as "position when braking" and "position when accelerating," and each item is associated with each other.

[0045] "Position during braking" is information indicating the position of the driver's feet when braking, more specifically, the position of the driver's thighs. The position of the driver's feet Da (thighs Da1) when braking is indicated by the solid line in Fig. 1B.

[0046] The "position when braking" in the accelerator operation information 31 described above may include, for example, information indicating the position of the driver's feet when it is estimated that the brakes are being applied. That is, as one example, the driver may press a brake setting button (not shown) while applying the brakes, and the position of the driver's feet when the brake setting button is pressed may be set (stored) as the "position when braking." As another example, an instruction to apply the brakes may be output to the driver via the output unit 14 or the like, and the position of the driver's feet at that time may be set (stored) as the "position when braking." Note that the above-described method of setting the "position when braking" is merely an example and is not limited thereto, and any method may be used.

[0047] "Position when accelerating" is information indicating the position of the driver's feet when the accelerator is being operated, and more specifically, the position of the driver's thighs. The position of the driver's feet Da (thighs Da1) when the accelerator is being operated is the position indicated by the dashed line in FIG. 1B.

[0048] The "position when accelerating" in the accelerator operation information 31 described above may include, for example, information indicating the position of the driver's feet when it is estimated that the accelerator is being operated. That is, as one example, the driver may press an accelerator setting button (not shown) while operating the accelerator, and the position of the driver's feet when the accelerator setting button is pressed may be set (stored) as the "position when accelerating." As another example, an instruction to operate the accelerator may be output to the driver via the output unit 14 or the like, and the position of the driver's feet at that time may be set (stored) as the "position when accelerating." Note that the above-described method of setting the "position when accelerating" is merely an example and is not limited thereto, and any method may be used.

[0049] The accelerator operation information 31 described above is stored in advance in the storage unit 30, but is not limited to this. For example, while driving a vehicle, learning may be performed using a machine learning model regarding the tendency of foot (thigh) position when accelerating or braking, and accelerator operation information 31 according to the learning results may be stored in the storage unit 30.

[0050] In the example shown in Fig. 3, the position when braking is "F1" and the position when accelerating is "G1." Note that in the example shown in Fig. 3, for convenience, the position when braking is abstractly described as "F1" and the position when accelerating is "G1," but specific information is stored in "F1" and "G1."

[0051] Returning to the explanation of FIG. 2, the acquisition unit 21 of the control unit 20 acquires various images, information, signals, etc. For example, the acquisition unit 21 acquires an interior image of the vehicle from the interior camera 11. Furthermore, if the interior image includes an exterior view of the vehicle, the acquisition unit 21 acquires the interior image as movement information indicating the vehicle's movement state. That is, as described above, the interior image may include an exterior view of the vehicle through the window 111 of the driver's seat 110 or the window 121 of the passenger seat 120 (see FIG. 1C ), and in such a case, the acquisition unit 21 acquires the interior image of the vehicle as movement information. The acquisition unit 21 outputs the acquired interior image of the vehicle to the detection unit 22, the determination unit 23, etc.

[0052] The acquisition unit 21 also acquires vehicle exterior images from the vehicle exterior camera 12. The acquisition unit 21 also acquires the vehicle exterior images as situation information indicating the vehicle situation. This situation information (vehicle exterior images) is information used in a process of determining whether the vehicle is in a start prohibition situation, which will be described later.

[0053] The acquisition unit 21 may also acquire vehicle exterior images as movement information. That is, for example, when the vehicle interior images do not include the above-described exterior of the vehicle, the acquisition unit 21 may acquire vehicle exterior images as movement information instead of or in addition to the vehicle interior images. The acquisition unit 21 outputs the acquired vehicle exterior images to the detection unit 22, the determination unit 23, etc.

[0054] Furthermore, the acquisition unit 21 may acquire, as movement information, signals of the vehicle sensor group 40, which are signals transmitted from the vehicle. That is, for example, when the interior image of the vehicle does not include the state of the exterior of the vehicle as described above, the acquisition unit 21 may acquire, as movement information, signals of the vehicle sensor group 40 instead of or in addition to the interior image of the vehicle. The acquisition unit 21 outputs the acquired signals of the vehicle sensor group 40 to the detection unit 22, the determination unit 23, etc.

[0055] Furthermore, the acquisition unit 21 may store the acquired vehicle interior images and vehicle exterior images as a vehicle operation record in the storage unit 30. Furthermore, when the acquisition unit 21 detects sudden braking, sudden acceleration, abrupt steering, an impact due to a collision accident, or the like of the vehicle based on the output of the acceleration sensor 13, the acquisition unit 21 may perform processing to prohibit overwriting of data of vehicle interior images and vehicle exterior images for a predetermined period before and after the detection.

[0056] The detection unit 22 detects the movement of the driver's feet based on the vehicle interior image. For example, the detection unit 22 analyzes the vehicle interior image to detect the driver's feet (more specifically, the thighs (thighs)). The detection unit 22 then detects the movement of the detected feet (thighs). Note that in the above description, the detection unit 22 detects the thighs of the driver's feet, but this is not limiting, and the detection unit 22 may detect, for example, the knees or lower legs in addition to or instead of the thighs.

[0057] Furthermore, based on the detected foot (thigh) movement and accelerator operation information 31, detection unit 22 determines whether the movement of the driver's foot is an accelerator operation. For example, if the foot detected from the vehicle interior image is a movement of moving from a position during braking in accelerator operation information 31 to a position during accelerator operation, detection unit 22 determines that the detected foot (thigh) movement is an accelerator operation, that is, detects the foot (thigh) movement as an accelerator operation. Conversely, if the foot detected from the vehicle interior image is not a movement of moving from a position during braking in accelerator operation information 31 to a position during accelerator operation, detection unit 22 determines that the detected foot movement is not an accelerator operation, that is, does not detect the foot movement as an accelerator operation. Detection unit 22 outputs information indicating the detection result to detection unit 24.

[0058] The determination unit 23 performs processing such as determining whether the vehicle is in a start prohibition situation. For example, the determination unit 23 determines whether the vehicle is in a start prohibition situation based on situation information (exterior image) of the vehicle. For example, the determination unit 23 analyzes the exterior image, which is situation information, and detects the display state of a traffic light present in the traveling direction of the vehicle. Then, when the display state of the detected traffic light is a display state instructing the vehicle to stop (e.g., a red light), the determination unit 23 determines that the vehicle is in a situation where it is not permitted to start, i.e., a start prohibition situation. Conversely, when the display state of the detected traffic light is a display state instructing the vehicle to proceed (e.g., a green light), the determination unit 23 determines that the vehicle is in a situation where it is permitted to start (a start-permitted situation), i.e., the start prohibition situation is not present. The determination unit 23 outputs information indicating the determination result to the detection unit 24.

[0059] In this way, in this embodiment, whether or not a starting prohibition situation exists is determined based on an image outside the vehicle, so that even with a simple configuration that uses an image outside the vehicle, it is possible to accurately determine whether or not a starting prohibition situation exists.

[0060] In addition, in this embodiment, the display state of the traffic light is detected based on the image outside the vehicle to determine whether or not a start-prohibited situation exists, so that it is possible to more accurately determine whether or not a start-prohibited situation exists.

[0061] In the above description, whether or not a start prohibition situation exists is determined based on the display state of a traffic light obtained based on the vehicle's situation information. However, this is not limiting. For example, the determination unit 23 analyzes an outside-vehicle image, which is situation information, and calculates the distance from the vehicle to an object present around the vehicle. The object here may be, for example, a vehicle ahead or a wall, but is not limited to these. The determination unit 23 then determines whether or not a start prohibition situation exists based on the calculated distance. For example, if the calculated distance is equal to or less than a predetermined value, the determination unit 23 determines that the vehicle is in a start prohibition situation. Note that the above-mentioned predetermined value is set to, for example, a value that estimates that the vehicle may come into contact with an object if the vehicle starts, but is not limited to this and can be set to any value. Conversely, if the calculated distance is greater than the predetermined value, the determination unit 23 determines that the vehicle is in a situation where it is permitted to start (a start-permitted situation), i.e., the start prohibition situation does not exist.

[0062] In this way, in this embodiment, the distance to objects around the vehicle is calculated based on the outside vehicle image, and whether or not a starting prohibition situation exists is determined based on the calculated distance, thereby making it possible to more accurately determine whether or not a starting prohibition situation exists.

[0063] Furthermore, the determination unit 23 performs a process of determining whether the vehicle is moving. For example, the determination unit 23 determines whether the vehicle is moving based on movement information indicating the moving state of the vehicle. As described above, the movement information may include an image inside the vehicle, an image outside the vehicle, signals from the vehicle sensor group 40, and the like.

[0064] For example, when the interior image of the vehicle includes the exterior of the vehicle, the determination unit 23 analyzes the interior image (for example, optical flow analysis) to detect feature points of objects (for example, guardrails, signs, etc.) present outside the vehicle, and determines that the vehicle is moving when such feature points move. Conversely, when the feature points in the interior image of the vehicle do not move, the determination unit 23 determines that the vehicle is not moving, i.e., is stopped. The determination unit 23 outputs information indicating the determination result to the detection unit 24.

[0065] In this way, in this embodiment, whether or not the vehicle is moving is determined based on the in-vehicle image, so that even with a simple configuration that uses the in-vehicle image, it is possible to accurately determine whether or not the vehicle is moving.

[0066] Furthermore, when the interior image does not include the exterior of the vehicle, the determination unit 23 analyzes the exterior image (for example, optical flow analysis) to detect feature points of objects (for example, guardrails, signs, etc.) that exist outside the vehicle, and determines that the vehicle is moving when such feature points move. Conversely, when the feature points in the exterior image do not move, the determination unit 23 determines that the vehicle is not moving, i.e., is stopped.

[0067] In this way, in this embodiment, whether or not the vehicle is moving is determined based on the outside-of-vehicle image, so that even with a simple configuration that uses the outside-of-vehicle image, it is possible to accurately determine whether or not the vehicle is moving.

[0068] The determination unit 23 may determine whether the vehicle is moving based on signals from the vehicle sensor group 40, instead of or in addition to the interior image or exterior image. As one example, the determination unit 23 determines that the vehicle is moving when a vehicle speed signal indicating that the vehicle is moving is transmitted from the vehicle speed sensor 41 of the vehicle sensor group 40. As another example, the determination unit 23 determines that the vehicle is moving when an accelerator signal indicating that the vehicle is moving is transmitted from the accelerator sensor 42. As another example, the determination unit 23 determines that the vehicle is moving when a travel distance signal indicating that the vehicle is moving is transmitted from the travel distance sensor 44.

[0069] In this way, in this embodiment, whether or not the vehicle is moving is determined based on the signals of the vehicle sensor group 40, in other words, based on the signals transmitted from the vehicle. Therefore, even with a simple configuration that uses the signals of the vehicle sensor group 40, it is possible to accurately determine whether or not the vehicle is moving.

[0070] Based on the information indicating the detection result by the detection unit 22 and the determination result by the determination unit 23, the detection unit 24 detects that the movement of the driver's foot is an erroneous operation of the accelerator.

[0071] For example, when it is determined that the detected foot (e.g., thigh) movement is an accelerator movement to operate the accelerator and the vehicle is in a start-prohibited situation, the detection unit 24 detects that the foot (thigh) movement is an erroneous accelerator operation. That is, since the driver is operating the accelerator despite the vehicle being in a start-prohibited situation in which the vehicle should not start, the detection unit 24 detects the foot (thigh) movement as an erroneous accelerator operation. The detection unit 24 outputs information indicating the detection result to the notification unit 25.

[0072] In this way, in this embodiment, by using the movement of the driver's feet (thighs) detected based on the in-vehicle image and the determination result of whether the vehicle is in a starting-prohibited situation, it is possible to accurately detect the driver's erroneous operation of the accelerator.

[0073] The detection unit 24 may further detect an erroneous operation of the accelerator depending on the moving state of the vehicle. For example, when the detected movement of the foot (e.g., thigh) is an accelerator operation, the detection unit 24 determines that the vehicle is in a start-prohibited state, and the vehicle is moving, the detection unit 24 detects that the foot movement is an erroneous operation of the accelerator. In other words, even though the vehicle is in a start-prohibited state in which the vehicle should not start, the driver is operating the accelerator, and the vehicle is moving (starting to move), so the detection unit 24 detects the movement of the foot (thigh) as an erroneous operation of the accelerator.

[0074] In this way, in this embodiment, by using the movement of the driver's feet (thighs) detected based on the in-vehicle image, the determination result of whether the vehicle is in a starting-prohibited situation or not, and the determination result of whether the vehicle is moving or not, it is possible to accurately detect the driver's erroneous operation of the accelerator.

[0075] The notification unit 25 notifies the driver of an erroneous operation of the accelerator. For example, when the notification unit 25 detects that the movement of the foot is an erroneous operation of the accelerator, the notification unit 25 notifies the driver of the erroneous operation. Specifically, the notification unit 25 notifies the driver via the output unit 14.

[0076] Here, an example of notification when the output unit 14 is a display unit such as a display will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of notification of an erroneous operation of the accelerator. As shown in Fig. 4, when the output unit 14 is a display unit, the notification unit 25 displays information indicating that the movement of the driver's foot is an erroneous operation of the accelerator in a display field 200. Furthermore, although not shown, when the output unit 14 is an audio output unit such as a speaker, the notification unit 25 outputs a sound, a buzzer, or the like from the audio output unit indicating that the movement of the driver's foot is an erroneous operation of the accelerator.

[0077] In this way, in this embodiment, by notifying the driver of the erroneous operation of the accelerator, it is possible to make the driver aware of the erroneous operation of the accelerator.

[0078] Continuing with the description of FIG. 2, the notification unit 25 may prohibit notification of an erroneous operation depending on the shift state of the vehicle. That is, the notification unit 25 may detect the shift state of the vehicle, and prohibit notification of an erroneous operation if the detected shift state is other than drive. In more detail, the notification unit 25 may prohibit notification of an erroneous operation if the detected shift state is other than drive, that is, if the shift state is a shift state in which the vehicle is not traveling forward (e.g., neutral, park, etc.). Conversely, the notification unit 25 may notify of an erroneous operation only when the shift state of the vehicle is drive.

[0079] As a result, in this embodiment, when the shift state is other than drive and the vehicle is not moving forward, notification of an erroneous accelerator operation can be prevented; in other words, notifications that are unnecessary for the driver can be prevented from being made.

[0080] The detection of the shift state of the vehicle is performed, for example, based on an interior image of the vehicle. That is, for example, if the interior image of the vehicle includes an image of a shift lever, the notification unit 25 analyzes the interior image of the vehicle to detect the position of the shift lever, and detects the shift state of the vehicle based on the detected position.

[0081] In the above, the notification unit 25 detects the shift state of the vehicle based on an image of the interior of the vehicle, but this is not limited to this, and the notification unit 25 may detect the shift state of the vehicle based on, for example, a signal output from the shift sensor 45 of the vehicle sensor group 40.

[0082] <Control processing of in-vehicle devices> Next, an example of a specific processing procedure in the in-vehicle device 10 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the processing procedure executed by the in-vehicle device 10.

[0083] 5, the control unit 20 of the in-vehicle device 10 stores accelerator operation information 31 in the storage unit 30 (step S10). For example, the control unit 20 stores in advance, as accelerator operation information 31, information such as "braking position" indicating the position of the driver's feet when braking, and information such as "accelerating position" indicating the position of the driver's feet when operating the accelerator.

[0084] Then, the control unit 20 executes the accelerator erroneous operation detection process from step S11 onwards. For example, the control unit 20 acquires an inside-vehicle image from the inside-vehicle camera 11 and an outside-vehicle image from the outside-vehicle camera 12 (step S11).

[0085] Next, the control unit 20 detects the driver's foot movement based on the in-vehicle image (step S12). Subsequently, the control unit 20 determines whether the detected driver's foot movement is an accelerator operation based on the accelerator operation information 31 and the like (step S13).

[0086] If the control unit 20 determines that the driver's foot movement is not an accelerator operation (step S13, No), it skips the subsequent processing. On the other hand, if the control unit 20 determines that the driver's foot movement is an accelerator operation (step S13, Yes), it determines whether the vehicle is in a start-prohibited state based on the outside-of-vehicle image or the like (step S14).

[0087] If the control unit 20 determines that the vehicle is not in a start-prohibited state (step S14, No), it skips the subsequent processing. On the other hand, if the control unit 20 determines that the vehicle is in a start-prohibited state (step S14, Yes), it determines whether the vehicle is moving based on the interior image or the like (step S15).

[0088] If the control unit 20 determines that the vehicle is not moving (step S15, No), it skips the subsequent processing. On the other hand, if the control unit 20 determines that the vehicle is moving (step S15, Yes), it detects that the foot movement is an erroneous operation of the accelerator and notifies the user (step S16).

[0089] As described above, the in-vehicle device 10 according to the embodiment includes the control unit 20. The control unit 20 detects the driver's foot movement based on an interior image of the vehicle interior. The control unit 20 determines whether the vehicle is in a start prohibition situation in which starting of the vehicle is prohibited based on situation information indicating the vehicle's situation. The control unit 20 determines whether the vehicle is moving based on movement information indicating the vehicle's movement state. If the detected foot movement is an accelerator operation to operate the accelerator, and it is determined that the vehicle is in a start prohibition situation and that the vehicle is moving, the control unit 20 detects and notifies that the foot movement is an erroneous operation of the accelerator. This makes it possible to accurately detect and notify an erroneous operation of the accelerator by the driver.

[0090] Further advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents. [Explanation of symbols]

[0091] 10 Onboard equipment 11 In-car camera 12. Exterior camera 20 Control Unit

Claims

1. An in-vehicle device having a control unit, The control unit Detecting the movement of the driver's feet based on an interior image of the vehicle; determining whether or not a start prohibition situation exists in which starting of the vehicle is prohibited based on situation information indicating the situation of the vehicle; If the interior image includes an exterior view of the vehicle, determining whether the vehicle is moving based on the exterior view of the vehicle in the interior image; If the detected foot movement is an accelerator operation to operate an accelerator, and it is determined that the vehicle is in the start prohibition situation and that the vehicle is moving, the system detects and notifies that the foot movement is an erroneous operation of the accelerator. In-vehicle device.

2. The control unit If the interior image does not include an exterior of the vehicle, it is determined whether the vehicle is moving based on an exterior image capturing an exterior of the vehicle. The in-vehicle device according to claim 1 .

3. The control unit If the interior image does not include an exterior view of the vehicle, it is determined whether the vehicle is moving based on a signal transmitted from the vehicle. The in-vehicle device according to claim 1 or 2.

4. The control unit detecting a thigh movement of the driver based on the in-vehicle image; If the detected thigh movement is the accelerator operation, and it is determined that the vehicle is in the start prohibition state and that the vehicle is moving, it is detected that the thigh movement is an erroneous operation of the accelerator. The in-vehicle device according to any one of claims 1 to 3.

5. The control unit an outside image of the vehicle captured as the situation information, and determining whether the vehicle is in the start prohibition situation based on the acquired outside image of the vehicle; The in-vehicle device according to any one of claims 1 to 4.

6. A method for controlling an in-vehicle device, comprising: Detecting the movement of the driver's feet based on an interior image of the vehicle; determining whether or not a start prohibition situation exists in which starting of the vehicle is prohibited based on situation information indicating the situation of the vehicle; If the interior image includes an exterior view of the vehicle, determining whether the vehicle is moving based on the exterior view of the vehicle in the interior image; If the detected foot movement is an accelerator operation to operate an accelerator, and it is determined that the vehicle is in the start prohibition situation and that the vehicle is moving, the system detects and notifies that the foot movement is an erroneous operation of the accelerator. A method for controlling an in-vehicle device.

Images