Inspection device, inspection method, and inspection program

Abstract

To prevent the erroneous execution of a vehicle equipment inspection function.SOLUTION: An inspection device which executes an inspection function necessary to inspect equipment of a vehicle 100 comprises: an authentication section 331 which determines whether or not a predetermined authentication condition is met based on a detection result from a detection device mounted on the vehicle; a recognition section 332 which recognizes a gesture of a passenger in the vehicle from an image captured by an in-vehicle camera 11 imaging an interior of the vehicle; and an inspection section 333 which executes the inspection function corresponding to the gesture of the passenger recognized through the recognition section when the authentication section determines that the authentication condition is met.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to an inspection device, an inspection method, and an inspection program. [Background technology]

[0002] It has been known for some time that a driver's gesture contained in an image of the interior of a vehicle captured by an imaging unit that captures the interior of the vehicle is identified, and if the identified gesture matches a gesture that has been stored in advance, the image of the interior of the vehicle is displayed on a display (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-126357 Summary of the Invention [Problem to be solved by the invention]

[0004] However, vehicle equipment needs to be inspected at any time. To reduce the burden on the inspection worker, it is conceivable that the inspection function for inspecting such vehicle equipment can be started based on the above-described gesture of the user inside the vehicle.

[0005] However, if the inspection function is started by a gesture or the like made by the vehicle occupant, there is a possibility that the occupant may unintentionally make a gesture to start the inspection function, resulting in an erroneous operation such as the inspection function being executed by mistake.

[0006] In view of the above-mentioned problems, an object of the present disclosure is to prevent the inspection function of vehicle equipment from being erroneously executed. [Means for solving the problem]

[0007] The gist of the present disclosure is as follows.

[0008] (1) An inspection device that performs inspection functions necessary for inspecting vehicle equipment, an authentication unit that determines whether a predetermined authentication condition is satisfied based on a detection result by a detection device provided in the vehicle; a recognition unit that recognizes a gesture of an occupant of the vehicle included in an image captured by an in-vehicle camera that captures an image of the interior of the vehicle; an inspection unit that, when the authentication unit determines that the authentication condition is satisfied, executes the inspection function corresponding to the gesture of the occupant recognized by the recognition unit. (2) The inspection device according to (1) above, wherein the authentication conditions do not include conditions regarding the gestures of the occupant. (3) Further comprising a control unit for controlling operations of the vehicle other than the inspection function, The inspection device described in (1) or (2) above, wherein the control unit does not control the operation of the vehicle based on gestures of the vehicle occupants contained in the images captured by the in-vehicle camera. (4) The detection device includes a positioning sensor that detects the vehicle's own position, The inspection device according to any one of (1) to (3) above, wherein the authentication condition includes that the self-position detected by the positioning sensor is located within a predetermined designated area. (5) The detection device includes an in-vehicle camera that captures an image of the interior of the vehicle, The inspection device according to any one of (1) to (4) above, wherein the authentication condition includes that a predetermined image is included in the image captured by the in-vehicle camera. (6) An inspection device according to any one of (1) to (5) above, wherein the authentication unit, the recognition unit, and the inspection unit are configured to operate even when the ignition switch of the vehicle is off. (7) An inspection method for performing an inspection function required for inspecting a vehicle's equipment, comprising: determining whether a predetermined authentication condition is satisfied based on a detection result by a detection device provided in the vehicle; Recognizing a gesture of an occupant of the vehicle included in an image captured by an in-vehicle camera that captures an image of the interior of the vehicle; and when it is determined that the authentication condition is satisfied, executing the inspection function corresponding to the recognized gesture of the occupant. (8) An inspection program that performs inspection functions necessary for inspecting vehicle equipment, determining whether a predetermined authentication condition is satisfied based on a detection result by a detection device provided in the vehicle; Recognizing a gesture of an occupant of the vehicle included in an image captured by an in-vehicle camera that captures an image of the interior of the vehicle; and executing the inspection function corresponding to the recognized gesture of the occupant when it is determined that the authentication condition is satisfied. [Effects of the Invention]

[0009] According to the present disclosure, the inspection function of the vehicle's equipment is prevented from being executed erroneously. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic block diagram showing the configuration of the inspection system. [Figure 2] FIG. 2 is a schematic side view partially showing the interior of a vehicle equipped with the inspection system. [Figure 3] FIG. 3 is a functional block diagram of the processor of the ECU. [Figure 4] FIG. 4 is a diagram showing the state of the vehicle when the optical axis of the exterior camera is adjusted. [Figure 5] FIG. 5 is a flowchart showing the flow of authentication processing for determining whether or not authentication conditions for executing the inspection function are satisfied. [Figure 6] FIG. 6 is a flowchart showing the flow of the inspection process for executing the inspection function. [Figure 7]FIG. 7 is a schematic side view partially showing the interior of the vehicle around the in-vehicle camera. [Figure 8] FIG. 8 is a schematic side view partially showing the interior of the vehicle around the in-vehicle camera. [Figure 9] FIG. 9 is a flowchart similar to FIG. 5, showing the flow of authentication processing for determining whether the authentication conditions for executing the inspection function are satisfied. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, the embodiments will be described in detail with reference to the drawings. In the following description, like components are designated by like reference numerals.

[0012] First embodiment <Inspection system configuration> The configuration of an inspection system 1 equipped with an inspection device according to a first embodiment will be described with reference to Figs. 1 to 3. The inspection device performs inspection functions necessary for inspecting equipment on a vehicle. Fig. 1 is a schematic block diagram showing the configuration of the inspection system 1. Fig. 2 is a schematic side view partially showing the interior of a vehicle 100 equipped with the inspection system 1.

[0013] The inspection system 1 is mounted on a vehicle 100 and performs functions necessary for inspecting equipment of the vehicle 100. As shown in Fig. 2, the vehicle 100 includes a steering wheel 102 attached via a steering column 101, and a driver's seat 103 on which the driver sits.

[0014] 1, in this embodiment, the inspection system 1 includes an in-vehicle camera 11, an outside-vehicle camera 12, a distance measurement sensor 13, a positioning sensor 14, a human-machine interface (HMI) 15, a vehicle actuator 21, and an electronic control unit (hereinafter referred to as "ECU") 30. The in-vehicle camera 11, the outside-vehicle camera 12, the distance measurement sensor 13, the positioning sensor 14, the HMI 15, and the ECU 30 are communicatively connected, for example, via an in-vehicle network 25. The in-vehicle network 25 is, for example, a network that complies with a standard such as CAN (Controller Area Network). The ECU 30 is also connected to the vehicle actuator 21 via a signal line.

[0015] The in-vehicle camera 11 is a device that captures images of the interior of the vehicle and is an example of a detection device provided in the vehicle 100. In this embodiment, the in-vehicle camera 11 is installed facing the space above the driver's seat so as to capture an image of an occupant sitting in the driver's seat of the vehicle 100. In particular, in this embodiment, the in-vehicle camera 11 is installed so as to be able to capture an image of the face and part of the upper body of the occupant sitting in the driver's seat. The in-vehicle camera 11 is provided, for example, on the top of the steering column 101 as shown in FIG. 2 . Note that the in-vehicle camera 11 may be provided in a position other than the top of the steering column 101 as long as it can capture an image of an occupant inside the vehicle 100. For example, the in-vehicle camera 11 may be provided on the steering wheel 102, a room mirror, a meter panel, a meter hood, a ceiling, or the like of the vehicle 100.

[0016] The in-vehicle camera 11 includes a camera and a floodlight. For example, the camera is a CMOS (complementary metal-oxide semiconductor) camera or a CCD (charge-coupled device) camera, and the floodlight is an LED (light-emitting diode). Furthermore, the floodlight is preferably a near-infrared LED so that the face of the occupant can be captured without causing discomfort to the occupant even in low-light conditions, such as at night, and the camera is preferably capable of detecting near-infrared light. For example, the floodlights are two near-infrared LEDs arranged on both sides of the camera. The camera may also be provided with a filter such as a visible light cut filter. The in-vehicle camera 11 captures an image of the driver's seat area of the vehicle 100 at predetermined imaging intervals and generates an image of that area. Each time an image is generated, the in-vehicle camera 11 outputs the generated image to the ECU 30 via the in-vehicle network 25.

[0017] The exterior camera 12 is a device that captures images of the vehicle's surroundings and is an example of a detection device provided on the vehicle 100. In this embodiment, the exterior camera 12 captures images of the area ahead of the vehicle 100. The exterior camera 12 is, for example, a CMOS camera or a CCD camera sensitive to visible light. In this embodiment, the exterior camera 12 is mounted, for example, inside the vehicle 100 so as to face the front of the vehicle 100. The exterior camera 12 captures images of the area ahead of the vehicle 100 at predetermined imaging intervals and generates images of the area ahead. Each time an image is generated, the exterior camera 12 outputs the generated image to the ECU 30 via the in-vehicle network 25. The exterior camera 12 may be a monocular camera or a stereo camera. If a stereo camera is used as the exterior camera 12, the exterior camera 12 also functions as the distance sensor 13. The vehicle 100 may be provided with multiple exterior cameras with different imaging directions or focal lengths.

[0018] The ranging sensor 13 is a device that measures the distance to an object present around the vehicle 100, and is an example of a detection device provided in the vehicle 100. In particular, in this embodiment, the ranging sensor 13 measures the distance to an object present in front of the vehicle 100. Furthermore, in this embodiment, the ranging sensor 13 can also measure the direction of an object present around the vehicle 100. The ranging sensor 13 is, for example, a radar such as a millimeter-wave radar, a LIDAR, or an ultrasonic sensor. In this embodiment, the ranging sensor 13 measures the distance to an object present in front of the vehicle. The ranging sensor 13 outputs the measurement results of the distance to the surrounding object to the ECU 30 via the in-vehicle network 25 at predetermined intervals.

[0019] The positioning sensor 14 is a device that measures the self-position of the vehicle 100, and is an example of a detection device provided in the vehicle 100. The positioning sensor 14 is, for example, a GNSS receiver. The GNSS receiver receives GNSS signals from a plurality of GNSS satellites and measures the self-position of the vehicle 100 based on the received GNSS signals. The positioning sensor 14 outputs self-position information of the vehicle 100 to the ECU 30 via the in-vehicle network 25 at predetermined intervals. Note that the positioning sensor 14 may be a receiver that complies with other satellite positioning systems as long as it can measure the self-position of the vehicle 100.

[0020] The HMI 15 is an interface for inputting and outputting information between the occupant and the inspection system 1. The HMI 15 includes an information providing device for providing various information to the occupant, and an input device for the occupant to perform input operations.

[0021] Specifically, the HMI 15 includes a display 16 as an information providing device for displaying text information or image information. The display 16 is an example of a display device that displays images. The display 16 is a display device of any type, such as a liquid crystal display or an organic light-emitting diode (OLED) display. The display 16 is disposed so that at least the occupants can view the screen. Therefore, the display 16 is disposed, for example, on an instrument panel, meter panel, or the like of the vehicle 100. The display 16 receives an image signal from the ECU 30 via the in-vehicle network 25 and displays an image in accordance with this image signal. Note that the vehicle 100 may include, as an information providing device, another type of display device, such as a head-up display, instead of or in addition to the display 16.

[0022] The HMI 15 also has a speaker 17 as an information providing device. The speaker 17 is an example of a device that outputs audio. The speaker 17 receives an audio signal from the ECU 30 via the in-vehicle network 25 and outputs audio in accordance with this audio signal. Note that the HMI 15 may also include devices (e.g., a vibration device) other than the display 16 and the speaker 17 as an information providing device that provides various types of information to the occupant.

[0023] Additionally, the HMI 15 has a touch panel 18 as an input device. The touch panel 18 is an example of a device that is touched by a passenger to perform input. When an occupant performs an operation by touching the touch panel 18, the touch panel 18 outputs an operation signal to the ECU 30 via the in-vehicle network 25. Note that the HMI 15 may also have devices (e.g., buttons, switches, etc.) other than the touch panel 18 as an input device for the occupant to perform input operations.

[0024] The vehicle actuators 21 are devices used to cause the vehicle 100 to perform various operations. The vehicle actuators 21 include devices used to drive the vehicle 100. Specifically, the vehicle actuators 21 include, for example, a drive actuator that controls an internal combustion engine or an electric motor for driving the vehicle 100, a braking actuator that controls brakes that brake the vehicle 100, and a steering actuator that controls steering of the vehicle 100. The vehicle actuators 21 control the acceleration, braking, and steering of the vehicle 100 in accordance with control signals transmitted from the ECU 30 via signal lines.

[0025] The vehicle actuators 21 also include devices used for operations other than driving the vehicle 100. Specifically, the vehicle actuators 21 include devices used for operations other than driving, braking, and steering the vehicle 100, such as headlights that illuminate the road ahead of the vehicle 100, glass motors that move the window glass of the vehicle 100 up and down, and fans and compressors for air conditioning the vehicle 100. Note that the vehicle actuators 21 may also include devices used for operations other than the above-mentioned operations.

[0026] <Configuration of inspection equipment> The ECU 30 functions as an inspection device that executes functions necessary for inspecting equipment of the vehicle 100. In this embodiment, the ECU 30 executes functions necessary for inspecting the exterior camera 12 and the distance measurement sensor 13. The ECU 30 also controls the vehicle actuators 21. Therefore, the ECU 30 controls the driving, braking, and steering of the vehicle 100, and the turning on and off of headlights, etc. As shown in FIG. 1 , the ECU 30 has a communication interface 31, a storage unit 32, and a processor 33.

[0027] The communication interface 31 is a circuit for connecting the ECU 30 to the in-vehicle network 25 .

[0028] The storage unit 32 stores data. The storage unit 32 includes, for example, at least one of a volatile semiconductor memory, a non-volatile semiconductor memory, a hard disk drive (HDD), and a solid state drive (SSD). The storage unit 32 stores computer programs executed by the processor 33 of the ECU 30. The storage unit 32 also stores data used in the computer programs executed by the processor 33, such as data transmitted from the in-vehicle camera 11.

[0029] The processor 33 has one or more central processing units (CPUs) and their peripheral circuits. The processor 33 may further have other arithmetic circuits such as a logic operation unit or a numerical operation unit. The processor 33 executes a computer program stored in the storage unit 32.

[0030] 3 is a functional block diagram of the processor 33 of the ECU 30. As shown in FIG. 3, the processor 33 includes an authentication unit 331, a recognition unit 332, an inspection unit 333, and a control unit 334.

[0031] The authentication unit 331 determines whether or not predetermined authentication conditions are met based on the detection results from a detection device provided in the vehicle 100. In particular, in this embodiment, the authentication unit 331 determines whether or not the authentication conditions are met without using equipment outside the vehicle 100 (for example, an inspection PC, etc.). Therefore, no equipment outside the vehicle 100 is required to establish the authentication conditions. In this embodiment, the authentication unit 331 receives information about the vehicle's own position detected by the positioning sensor 14 from the positioning sensor 14. Then, the authentication unit 331 outputs the determination result as to whether or not the authentication conditions are met to the inspection unit 333.

[0032] In this embodiment, the authentication condition in the authentication unit 331 includes that the self-position detected by the positioning sensor 14 is located within a predetermined designated area. In particular, in this embodiment, the authentication condition is met when the detected self-position is located within the predetermined designated area, and the authentication condition is not met when the detected self-position is not located within the predetermined designated area.

[0033] The designated area is registered in advance by the owner of vehicle 100 or the dealer of vehicle 100. For example, the designated area may be an area where a dealer that performs maintenance on vehicle 100 is located, or an area where the home of the owner of vehicle 100 is located. Therefore, when a dealer that performs maintenance on vehicle 100 is registered as a designated area, authentication unit 331 determines that the authentication condition is satisfied if the vehicle's own position detected by positioning sensor 14 is within the area where the dealer is located. On the other hand, when the vehicle's own position detected by positioning sensor 14 is outside the area where the dealer is located, for example, on a road, authentication unit 331 determines that the authentication condition is not satisfied.

[0034] The recognition unit 332 recognizes gestures of an occupant of the vehicle 100 included in an image captured by the in-vehicle camera 11. In particular, in this embodiment, the recognition unit 332 recognizes gestures of an occupant sitting in the driver's seat of the vehicle 100 by performing image processing on image information captured by the in-vehicle camera 11. Information on the image captured by the in-vehicle camera 11 is input to the recognition unit 332. Then, when the recognition unit 332 recognizes a gesture of an occupant sitting in the driver's seat of the vehicle 100, the recognition unit 332 outputs information on the recognized gesture to the inspection unit 333.

[0035] Specifically, in this embodiment, the recognition unit 332 first inputs an image acquired from the in-vehicle camera 11 into a classifier that recognizes hands and their shapes, and recognizes the positions and shapes of the hands contained in the image. For example, a convolutional neural network (CNN) that receives an image of an occupant as input and outputs the positions and shapes of the hands is used as such a classifier. However, the recognition unit 332 may detect the positions and shapes of the occupant's hands using any method other than the method using such a classifier.

[0036] Additionally, the recognition unit 332 recognizes the gesture of the occupant of the vehicle 100 based on whether or not the changes in the hand position and hand shape recognized over time match a pre-registered gesture. For example, consider a case where changing the hand from a five-fingered closed state (a so-called fist state) to a five-fingered open state (a so-called paper state) is pre-registered as a gesture. In this case, when the recognition unit 332 recognizes the hand of an occupant sitting in the driver's seat with five fingers closed and then immediately recognizes the hand with five fingers open, the recognition unit 332 recognizes that the occupant of the vehicle 100 has performed the same gesture as the pre-registered gesture.

[0037] Specifically, in this embodiment, the recognition unit 332 inputs the recognized changes in hand position and hand shape over time into a classifier for gesture recognition. When the input changes in hand position and hand shape over time match a pre-registered gesture, the classifier outputs a signal indicating that a gesture corresponding to the input gesture has been performed. For example, a convolutional neural network (CNN) is used as the classifier, which receives input changes in hand position and hand shape over time and outputs a signal indicating that a gesture corresponding to the input changes has been performed. However, the recognition unit 332 may detect that a corresponding gesture has been performed using any method other than the method using the classifier.

[0038] In this embodiment, the recognition unit 332 is configured to recognize a gesture made by the hand of an occupant sitting in the driver's seat. However, the recognition unit 332 may be configured to recognize a gesture made by an occupant other than the occupant sitting in the driver's seat, or may be configured to recognize a gesture made by a part of the occupant other than the hand (for example, the head or the upper body).

[0039] When the authentication unit 331 determines that the authentication conditions are satisfied, the inspection unit 333 executes an inspection function corresponding to the gesture of the occupant recognized by the recognition unit 332. The inspection unit 333 receives an input of the determination result of whether the authentication conditions are satisfied from the authentication unit 331, and also receives input of information on the recognized gesture from the recognition unit 332. Then, the inspection unit 333 outputs a control signal for a device corresponding to the inspection function to be executed.

[0040] In this embodiment, a corresponding inspection function is set for each gesture action performed by the occupant of the vehicle 100. Therefore, when the recognition unit 332 recognizes a certain gesture (first gesture), an inspection function (first inspection function) corresponding to that gesture is executed. Then, when the recognition unit 332 recognizes another gesture (second gesture) different from the first gesture, an inspection function (second inspection function different from the first inspection function) corresponding to the other gesture is executed. This allows the occupant of the vehicle 100 to execute a plurality of different inspection functions depending on the gesture.

[0041] Further, the inspection function means a function necessary for inspecting the vehicle 100. Therefore, the inspection function includes not only a function for actually inspecting the equipment of the vehicle 100, but also a function that needs to be performed on the equipment of the vehicle 100 when inspecting the vehicle 100, such as a function for preparing to inspect the equipment of the vehicle 100.

[0042] Among these, the function of actually inspecting the equipment of the vehicle 100 includes, for example, a function for adjusting the optical axis of the exterior camera 12. The optical axis adjustment of the exterior camera 12 is a function for adjusting the imaging direction of the exterior camera 12, for example, when the front camera to which the exterior camera 12 is attached is replaced. Below, the optical axis adjustment function of the exterior camera 12 will be briefly explained with reference to FIG. 4. FIG. 4 is a diagram showing the state of the vehicle 100 when the optical axis adjustment of the exterior camera 12 is performed. In particular, FIG. 4(A) is a diagram showing the vehicle 100 and the target 110 as viewed from the side when the optical axis adjustment of the exterior camera 12 is performed, and FIG. 4(B) is a diagram showing the target 110 as viewed from the front (the vehicle side of FIG. 4(A)).

[0043] As shown in Fig. 4, when adjusting the optical axis of the exterior camera 12, a target 110 is placed in front of the vehicle 100. As shown in Fig. 4(B), the target 110 has a rod-shaped member 111 extending vertically and a target plate 112 placed on top of the rod-shaped member 111. In the example shown in Fig. 4(B), three predetermined figures lined up in the horizontal direction are drawn on the target plate 112. The target 110 is placed on the center line extending in the longitudinal direction of the vehicle 100, with the side of the target plate 112 on which the figures are drawn facing the vehicle 100.

[0044] 4(A), when the optical axis adjustment function of the exterior camera 12 is executed with the target 110 appropriately positioned in front of the vehicle 100, the optical axis of the exterior camera 12 is automatically adjusted. This allows the ECU 30 to appropriately recognize the capturing direction of the image captured by the exterior camera 12.

[0045] The function of adjusting the optical axis of the exterior camera 12 has been described as an example of a function for actually inspecting equipment on the vehicle 100. However, the function for actually inspecting equipment on the vehicle 100 may also include a function for inspecting other equipment. For example, similar to adjusting the optical axis of the exterior camera 12, the vehicle may also include a function for adjusting the optical axis of the distance measuring sensor 13 (e.g., millimeter wave radar, LIDAR, ultrasonic sensor).

[0046] Furthermore, functions that need to be performed on the equipment of vehicle 100 when inspecting vehicle 100 include, for example, putting vehicle 100 into a test mode. For example, if vehicle 100 is a hybrid vehicle (HEV) equipped with an internal combustion engine, the internal combustion engine automatically stops depending on the state of charge of the battery. Therefore, when inspecting the ignition timing of the internal combustion engine, the function that automatically stops the internal combustion engine must be disabled. For this reason, in the test mode for inspecting the ignition timing, the function that automatically stops the internal combustion engine is disabled.

[0047] Furthermore, when inspecting the speedometer of the vehicle 100 using a chassis dynamometer, if the traction control is activated, the tires of the vehicle 100 may not operate properly. Therefore, when inspecting the speedometer of a vehicle 100 that has a traction control function, the traction control must be deactivated. For this reason, the traction control is deactivated in the test mode for inspecting the speedometer.

[0048] Note that, as examples of functions that need to be performed on the equipment of vehicle 100 when inspecting vehicle 100, setting the operating state of vehicle 100 to a test mode for checking ignition timing and a test mode for checking the speedometer have been given. However, functions that need to be performed on the equipment of vehicle 100 when inspecting vehicle 100 may also include functions for performing other operations on the equipment of vehicle 100.

[0049] In this embodiment, the inspection unit 333 performs such an inspection function in accordance with the gesture recognized by the recognition unit 332. For example, when the recognition unit 332 recognizes a first gesture, the inspection unit 333 adjusts the optical axis of the exterior camera 12. When the recognition unit 332 recognizes a second gesture different from the first gesture, the inspection unit 333 sets the operating state of the vehicle 100 to a test mode for ignition timing inspection. Furthermore, when the recognition unit 332 recognizes a third gesture different from the first and second gestures, the inspection unit 333 sets the operating state of the vehicle 100 to a test mode for speedometer inspection.

[0050] The control unit 334 controls the operations of the vehicle 100 other than the inspection function executed by the inspection unit 333. The control unit 334 receives as input images captured by the exterior camera 12, distance information measured by the distance measurement sensor 13, self-position information measured by the positioning sensor 14, and input information input by the occupant via an input device (such as the touch panel 18) of the HMI 15. The control unit 334 also outputs a control signal for a device corresponding to the operation to be executed.

[0051] In this embodiment, the control unit 334 controls the operation of the vehicle 100. Therefore, for example, when the vehicle 100 is an autonomous vehicle, the control unit 334 controls the vehicle actuators 21 based on the image input from the exterior camera 12, the distance information input from the distance measurement sensor 13, and the vehicle's own position information input from the positioning sensor 14. In particular, the control unit 334 controls the drive actuator, the braking actuator, and the steering actuator so that the vehicle 100 travels according to a travel route generated based on the input information.

[0052] The control unit 334 also controls operations of the vehicle 100 other than driving. Therefore, the control unit 334 controls, for example, turning on and off headlights that illuminate the road ahead of the vehicle 100 and opening and closing windows of the vehicle 100. The control unit 334 also controls an air conditioning fan and compressor so that the temperature inside the vehicle 100 reaches a target temperature. In addition, the control unit 334 may control operations of the vehicle 100 other than the operations described above.

[0053] However, in this embodiment, the control unit 334 does not use the output of the recognition unit 332. Therefore, the control unit 334 does not control the operation of the vehicle 100 based on the gesture of the occupant recognized by the recognition unit 332. In other words, the control unit 334 controls the operation of the vehicle 100 based on parameters other than the gesture of the occupant recognized by the recognition unit 332 (such as an image captured by the outside camera 12).

[0054] In the inspection device configured in this manner, when the recognition unit 332 recognizes a gesture of an occupant of the vehicle 100, an inspection function corresponding to the recognized gesture of the occupant is executed. Therefore, according to the inspection device of this embodiment, the occupant can execute the inspection function without touching the equipment of the vehicle 100, thereby reducing the possibility of the vehicle 100 being scratched or soiled.

[0055] Furthermore, if the inspection device is configured to be able to execute an inspection function corresponding to a gesture regardless of whether or not the authentication conditions are satisfied, for example, if an occupant unintentionally performs a gesture corresponding to the inspection function, there is a possibility that the inspection function will be executed erroneously. In contrast, in the inspection device of this embodiment, the inspection function corresponding to the gesture is executed when the authentication unit 331 determines that the authentication conditions are satisfied. In this way, the inspection function corresponding to the gesture is executed only when the authentication conditions are satisfied, thereby reducing the possibility of the inspection function being executed erroneously.

[0056] In particular, in this embodiment, the gestures of the occupant of the vehicle 100 recognized by the recognition unit 332 are not used to control operations other than the inspection function of the vehicle 100. This prevents the occupant of the vehicle 100 from unintentionally executing operations other than the inspection function of the vehicle 100.

[0057] In this embodiment, the authentication condition is satisfied when the vehicle 100 is located within a designated area such as a garage at home, a dealership, etc. Therefore, the authentication condition is satisfied without the occupant having to perform any special operation, thereby reducing the operational burden on the occupant.

[0058] Note that, when controlling operations of the vehicle 100 other than the inspection function, the control unit 334 may use the gestures of the occupant of the vehicle 100 recognized by the recognition unit 332. In this case, the control unit 334 may use the gestures of the occupant of the vehicle 100 only when the authentication unit 331 determines that some authentication condition is satisfied.

[0059] <Flow of inspection control using inspection equipment>

[0060] Next, the flow of inspection control by the inspection device will be described with reference to Figures 5 and 6. Figure 5 is a flowchart showing the flow of authentication processing for determining whether authentication conditions for executing the inspection function are satisfied. The illustrated authentication processing is executed by the processor 33 of the ECU 30 at regular time intervals.

[0061] Note that the authentication process is executed as long as the accessory switch of the vehicle 100 is on, even if the ignition switch of the vehicle 100 is off. Therefore, the authentication unit 331 of the processor 33 is configured to operate as long as the accessory switch of the vehicle 100 is on, even if the ignition switch of the vehicle 100 is off. Here, the accessory switch is a switch that, when turned on, does not supply power to devices necessary for driving and steering the vehicle 100, but does supply power to devices not directly related to driving and steering the vehicle 100, such as the HMI 15. On the other hand, the ignition switch is a switch that, when turned on, supplies power not only to devices not directly related to driving and steering the vehicle 100, such as the HMI 15, but also to devices necessary for driving and steering the vehicle 100.

[0062] 5, when the authentication process starts, the authentication unit 331 acquires current self-location information measured by the positioning sensor 14 (step S11). The authentication unit 331 may acquire the self-location information directly from the positioning sensor 14, or may acquire the self-location information stored in the storage unit 32.

[0063] Next, the authentication unit 331 determines whether the current location of the vehicle 100 is located within a pre-registered designated area based on the acquired self-location information (step S12). The designated area is registered in advance by the owner of the vehicle 100 or the dealer of the vehicle 100, and is stored in the storage unit 32. Therefore, the authentication unit 331 determines whether the current location of the vehicle is located within the designated area stored in the storage unit 32.

[0064] If it is determined in step S12 that the position of the vehicle 100 is not within the specified area, for example, if the vehicle 100 is traveling on an external road, the authentication unit 331 protects the inspection function (step S13). Therefore, in this case, the protection of the inspection function is not released, and the inspection function is not executed even if the occupant of the vehicle 100 makes any gesture.

[0065] On the other hand, if it is determined in step S12 that the position of vehicle 100 is within the designated area, for example, if vehicle 100 is located in a garage at home or in a dealership, authentication unit 331 cancels protection of the inspection function (step S14). As a result, in the inspection process described below, the inspection function is executed in response to a gesture by the occupant of vehicle 100.

[0066] In addition, if the vehicle 100 moves outside the designated area after the protection of the inspection function is released in step S14, in the next authentication process, it is determined in step S12 that the current location of the vehicle 100 is not located within the previously registered designated area, and as a result, the inspection function is protected again.

[0067] 6 is a flowchart showing the flow of an inspection process for executing the inspection function. The illustrated inspection process is executed by the processor 33 of the ECU 30 at regular time intervals. Like the authentication process, the inspection process is also executed when the accessory switch of the vehicle 100 is on, even if the ignition switch of the vehicle 100 is off. Therefore, the recognition unit 332 and the inspection unit 333 of the processor 33 are configured to operate even if the ignition switch of the vehicle 100 is off, as long as the accessory switch of the vehicle 100 is on. Therefore, in this embodiment, inspection control can be performed even when the vehicle 100 is not in a state where it can run (when the ignition switch is off).

[0068] 6, when the inspection process starts, the inspection unit 333 determines whether protection of the inspection function has been released (step S21). Specifically, the inspection unit 333 determines that protection of the inspection function has been released when protection of the inspection function has been released in step S14 of the authentication process. If it is determined in step S21 that protection of the inspection function has not been released, the inspection process ends without executing the inspection function.

[0069] If it is determined in step S21 that the protection of the inspection function has been released, the recognition unit 332 determines whether the in-vehicle camera 11 is operating normally (step S22). Whether the in-vehicle camera 11 is operating normally can be determined by any method. For example, if the ECU 30 does not receive an image signal from the in-vehicle camera 11, if there is no change in the image signal for a long period of time, or if the ECU 30 receives an image signal that represents an entirely monochromatic image, the recognition unit 332 determines that the in-vehicle camera 11 is not operating normally. On the other hand, if the ECU 30 receives an image signal from the in-vehicle camera 11 that changes over time, the recognition unit 332 determines that the in-vehicle camera 11 is operating normally. If it is determined in step S22 that the in-vehicle camera 11 is not operating normally, the inspection process ends without executing the inspection function.

[0070] If it is determined in step S22 that the in-vehicle camera 11 is operating normally, the recognition unit 332 determines whether or not it has recognized a predetermined gesture (step S23). Specifically, the recognition unit 332 determines whether or not the changes in the hand position and hand shape recognized over time based on the images from the in-vehicle camera 11 match a pre-registered gesture. If it is determined in step S23 that the recognition unit 332 has not recognized a predetermined gesture, for example, if it is determined that the changes in the hand position and hand shape recognized over time do not match a pre-registered gesture, the inspection process ends without executing the inspection function.

[0071] On the other hand, if it is determined in step S23 that the recognition unit 332 has recognized a predetermined gesture, for example, if it is determined that the changes in the hand position and hand shape recognized over time match a pre-registered gesture movement, the inspection unit 333 executes an inspection function corresponding to the recognized gesture (step S24).

[0072] Second embodiment Next, an inspection device according to a second embodiment will be described with reference to Figures 7 to 9. The configuration and control of the inspection device according to the second embodiment are basically the same as the configuration and control of the inspection device according to the first embodiment. The following description will focus on the parts that are different from the inspection device according to the first embodiment.

[0073] In the first embodiment, the authentication conditions in the authentication unit 331 include the fact that the self-position of the vehicle 100 detected by the positioning sensor 14 is located within a predetermined designated area. However, in the present embodiment, the authentication conditions include conditions related to things other than the self-position of the vehicle 100 in addition to the conditions related to the self-position of the vehicle 100 or instead of the conditions related to the self-position of the vehicle 100.

[0074] In this embodiment, the authentication conditions for authentication unit 331 include that a predetermined authentication image is included in the image captured by in-vehicle camera 11. For example, the authentication conditions for authentication unit 331 include that a pre-registered authentication pattern (e.g., a two-dimensional barcode) is included in the image captured by in-vehicle camera 11. Alternatively, the authentication conditions for authentication unit 331 include that a pre-registered facial image of a person is included in the image captured by in-vehicle camera 11.

[0075] Specifically, the authentication unit 331 inputs the image acquired from the in-vehicle camera 11 into a classifier that recognizes the authentication pattern, and recognizes the authentication pattern. As such a classifier, a convolutional neural network (CNN) that recognizes the authentication pattern contained in an input image is used. For example, as shown in FIG. 7, when a piece of paper X on which an authentication pattern is drawn is imaged by the in-vehicle camera 11, such a classifier identifies the authentication pattern drawn on the piece of paper X. However, the recognition unit 332 may detect the authentication pattern by any method other than the method using such a classifier.

[0076] Next, the authentication unit 331 determines whether the authentication conditions are met based on whether the recognized authentication pattern matches a pre-registered authentication pattern. If the recognized authentication pattern matches a pre-registered authentication pattern, the authentication unit 331 determines that the authentication conditions are met. On the other hand, if the recognized authentication pattern does not match a pre-registered authentication pattern, the authentication unit 331 determines that the authentication conditions are not met.

[0077] Alternatively, the authentication unit 331 inputs an image acquired from the in-vehicle camera 11 into a classifier that recognizes human faces, thereby recognizing the human face. As such a classifier, a convolutional neural network (CNN) that recognizes the human face contained in an input image is used. For example, as shown in FIG. 8, when a human face Y is captured by the in-vehicle camera 11, the face is identified by such a classifier. However, the recognition unit 332 may detect the human face by any method other than the method using such a classifier.

[0078] Next, the authentication unit 331 determines whether the authentication conditions are met based on whether the recognized person's face matches a pre-registered person's face. If the recognized person's face matches a pre-registered person's face, the authentication unit 331 determines that the authentication conditions are met. On the other hand, if the recognized person's face does not match a pre-registered person's face, the authentication unit 331 determines that the authentication conditions are not met.

[0079] According to this embodiment, the authentication conditions in the authentication unit 331 do not include a condition related to the vehicle's own position, so that the inspection function can be executed even if the vehicle 100 is not located in the designated area.

[0080] In this embodiment, an image of an authentication pattern or an image of a person's face is used for authentication in the authentication unit 331. However, a predetermined authentication image other than an authentication pattern or a person's face, such as an image of palm prints, may also be used.

[0081] Furthermore, in the first and second embodiments, vehicle position information of the vehicle 100 and images acquired from the in-vehicle camera 11 are used to perform authentication in the authentication unit 331. However, parameters other than these may be used to perform authentication in the authentication unit 331. For example, input information from the input device of the HMI 15 may be used to perform authentication in the authentication unit 331. In this case, for example, the authentication unit 331 may determine that the authentication conditions are satisfied when a special input that is different from normal input is made via the input device of the HMI 15.

[0082] However, the authentication performed by the authentication unit 331 may not use gestures recognized by the recognition unit 332. In this case, the authentication conditions for the authentication unit 331 do not include conditions related to the gestures of the occupant. This prevents the occupant from mistakenly performing authentication based on their gestures.

[0083] 9 is a flowchart similar to that of FIG. 5, showing the flow of authentication processing for determining whether authentication conditions for executing the inspection function are satisfied. The illustrated authentication processing is executed by the processor 33 of the ECU 30 at regular time intervals.

[0084] As shown in FIG. 9, when the authentication process starts, the authentication unit 331 determines whether the inspection function is currently protected (step S31). If it is determined in step S31 that the inspection function is protected (protection of the inspection function has not been released), the authentication unit 331 acquires an image from the in-vehicle camera 11 (step S32). Next, the authentication unit 331 determines whether the image acquired in step S32 includes a predetermined authentication pattern (step S33). If it is determined in step S33 that the image acquired in step S32 does not include the predetermined authentication pattern, the authentication unit 331 protects the inspection function (step S34). Therefore, in this case, protection of the inspection function is not released, and the inspection function is not executed even if the occupant of the vehicle 100 makes any gesture.

[0085] On the other hand, if it is determined in step S33 that the image acquired in step S32 includes the predetermined authentication pattern, the authentication unit 331 releases the protection of the inspection function (step S35). As a result, in the inspection process, the inspection function is executed in response to the gesture of the occupant of the vehicle 100.

[0086] Thereafter, when the authentication process is executed again, it is determined in step S31 that the inspection function is not protected. In this case, the authentication unit 331 determines whether or not a termination condition for the inspection function is satisfied (step S36). The termination condition for the inspection function is satisfied, for example, when the recognition unit 332 recognizes that a gesture corresponding to the termination of the inspection function has been made by an occupant of the vehicle 100, or when an accessory switch of the vehicle 100 is turned off. If it is determined in step S36 that the termination condition for the inspection function is not satisfied, the release of protection for the inspection function continues (step S37). On the other hand, if it is determined in step S36 that the termination condition for the inspection function is satisfied, the authentication unit 331 protects the inspection function again.

[0087] 9, the authentication condition is satisfied when the image captured by the in-vehicle camera 11 includes a predetermined image for authentication. However, as described above, a condition related to the location of the vehicle 100, as in the first embodiment, may be added to the authentication condition. In this case, the authentication condition may be determined to be satisfied when the image captured by the in-vehicle camera 11 includes the predetermined image for authentication and the location of the vehicle 100 is within a designated area. Alternatively, the authentication condition may be determined to be satisfied when the image captured by the in-vehicle camera 11 includes the predetermined image for authentication or when the location of the vehicle 100 is within a designated area.

[0088] Although preferred embodiments according to the present disclosure have been described above, the present disclosure is not limited to these embodiments, and various modifications and changes can be made within the scope of the claims. [Explanation of symbols]

[0089] 1 Inspection System 11 In-car camera 12. Exterior camera 13 Distance measurement sensor 14 Positioning sensor 15 HMI 30 ECU 33 processors

Claims

1. An inspection device that performs inspection functions necessary for inspecting vehicle equipment, an authentication unit that determines whether a predetermined authentication condition is satisfied based on a detection result by a detection device provided in the vehicle; a recognition unit that recognizes a gesture of an occupant of the vehicle included in an image captured by an in-vehicle camera that captures an image of the interior of the vehicle; an inspection unit that, when the authentication unit determines that the authentication condition is satisfied, executes the inspection function corresponding to the gesture of the occupant recognized by the recognition unit.

2. The inspection device according to claim 1 , wherein the authentication conditions do not include a condition related to a gesture of the occupant.

3. Further, a control unit is provided to control operations of the vehicle other than the inspection function, The inspection device according to claim 1 or 2, wherein the control unit does not control the operation of the vehicle based on a gesture of an occupant of the vehicle included in the image captured by the in-vehicle camera.

4. the detection device includes a positioning sensor that detects a self-position of the vehicle; The inspection device according to claim 1 or 2, wherein the authentication condition includes that the self-position detected by the positioning sensor is located within a predetermined designated area.

5. the detection device includes an in-vehicle camera that captures an image of the interior of the vehicle; The inspection device according to claim 1 or 2, wherein the authentication condition includes that a predetermined image is included in the image captured by the in-vehicle camera.

6. The inspection device according to claim 1 or 2, wherein the authentication unit, the recognition unit, and the inspection unit are configured to operate even when an ignition switch of the vehicle is off.

7. An inspection method for performing an inspection function required for inspecting equipment on a vehicle, comprising: determining whether a predetermined authentication condition is satisfied based on a detection result by a detection device provided in the vehicle; Recognizing a gesture of an occupant of the vehicle included in an image captured by an in-vehicle camera that captures an image of the interior of the vehicle; and when it is determined that the authentication condition is satisfied, executing the inspection function corresponding to the recognized gesture of the occupant.

8. An inspection program that executes inspection functions necessary for inspecting equipment on a vehicle, determining whether a predetermined authentication condition is satisfied based on a detection result by a detection device provided in the vehicle; Recognizing a gesture of an occupant of the vehicle included in an image captured by an in-vehicle camera that captures an image of the interior of the vehicle; and when it is determined that the authentication condition is satisfied, executing the inspection function corresponding to the recognized gesture of the occupant.

Images