Gesture detection device and gesture detection method

Abstract

A gesture detection device (70) is provided with: a position acquisition unit (73) configured so as to acquire a predetermined position and a reference position, which are positions of a body part of a user in an image captured by a camera (50); a direction determination unit (74) configured so as to determine whether or not the user is facing an opening of a vehicle (10); and a gesture determination unit (75) configured so as to determine whether or not a gesture has been performed by the user. The gesture determination unit (75) is configured so as to determine that at least a predetermined gesture for opening the door is performed by the user when the predetermined position moves in a predetermined direction corresponding to the opening direction of the door with respect to the reference position.

Description

Technical Field

[0001] The invention relates to a gesture detection device and a gesture detection method. Background Art

[0002] Patent Document 1 describes a door control system including a drive device that drives a rear door, a distance sensor that measures the distance to a user's hand located near the rear door, and a control device that controls the drive device.

[0003] When the user moves the hand approaching the rear door in the opening direction or the closing direction, the door control system causes the rear door to open and close. At this time, the door control system implements a follow-up control that causes the moving speed of the rear door to follow the moving speed of the user's hand.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2019-105046 Summary of the invention

[0007] Technical problem to be solved by the invention

[0008] In the above-mentioned door control system, when the user wants to get on the vehicle as soon as possible, the user can quickly open the rear door by quickly moving the hand close to the door handle. However, the user cannot open the rear door unless he approaches the vehicle. The above-mentioned door control system still has room for improvement in terms of improving user convenience.

[0009] Technical means for solving technical problems

[0010] A gesture detection device according to one embodiment of the present invention is applied to a vehicle, the vehicle comprising: a vehicle body having an opening; a door configured to open and close the opening; a camera configured to capture a user facing the opening; and a door driving unit configured to drive the door, wherein the gesture detection device detects a gesture using a predetermined body part of the user, the gesture being a trigger for operating the door. A body part of the user different from the predetermined body part is a reference body part. The gesture detection device comprises: a position acquisition unit configured to acquire a predetermined position and a reference position, the predetermined position being the position of the predetermined body part in an image captured by the camera, and the reference position being the position of the reference body part in the image; a direction determination unit configured to determine whether the user is facing the opening based on the reference position in the image; and a gesture determination unit configured to determine whether the user has performed a gesture based on changes in the predetermined position and the reference position in a plurality of images captured at different times when the direction determination unit determines that the user is facing the opening. The gesture determination unit is further configured to determine that the user has performed at least a predetermined gesture for opening the door when the predetermined position has moved in a predetermined direction corresponding to the opening direction of the door relative to the reference position.

[0011] A gesture detection method according to another aspect of the present invention is applied to a vehicle, the vehicle comprising: a vehicle body having an opening, a door, the door being configured to open and close the opening; a camera being configured to capture a user facing the opening; and a door driving unit being configured to drive the door, wherein the gesture detection method detects a gesture using a predetermined body part of the user, the gesture being a trigger for operating the door. A body part of the user different from the predetermined body part is a reference body part. The gesture detection method comprises: a position acquisition step of acquiring a predetermined position and a reference position, the predetermined position being the position of the predetermined body part in an image captured by the camera, the reference position being the position of the reference body part in the image; a direction determination step of determining whether the user is facing the opening based on the reference position in the image; and a gesture determination step of determining whether the user has performed a gesture based on changes in the predetermined position and the reference position in a plurality of images captured at different times, when the direction determination step determines that the user is facing the opening. The gesture determination step includes determining that the user has at least performed a predetermined gesture for opening the door when the predetermined position has moved in a predetermined direction corresponding to the opening direction of the door relative to the reference position. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic diagram of a vehicle according to one embodiment.

[0013] Figure 2 is set at Figure 1 An example of an image captured by a camera of a vehicle.

[0014] Figure 3 is set at Figure 1 An example of an image captured by a camera of a vehicle, showing a user performing a gesture using his right hand.

[0015] Figure 4 is set at Figure 1 An example of an image captured by a camera of a vehicle, showing a user performing a gesture using his left hand.

[0016] Figure 5 It is set by Figure 1 Flowchart of the process of right outside gesture determination processing performed by the gesture determination unit of the vehicle.

[0017] Figure 6 It is set by Figure 1 Flowchart of the process of right inner gesture determination processing performed by the gesture determination unit of the vehicle. DETAILED DESCRIPTION

[0018] Hereinafter, an embodiment of a vehicle to which a gesture detection device and a gesture detection method are applied will be described with reference to the drawings.

[0019] <Structure of this embodiment>

[0020] like Figure 1 As shown, the vehicle 10 includes a vehicle body 20, a front door 31, a rear door 32, a front door driving unit 41, a rear door driving unit 42, a camera 50, a door control device 60, and a gesture detection device 70. In the following description, the width direction of the vehicle 10, the front-rear direction of the vehicle 10, and the up-down direction of the vehicle 10 are also referred to as the width direction, the front-rear direction, and the up-down direction. In addition, in the width direction, the direction away from the center of the vehicle 10 is referred to as the outside, and the direction close to the center of the vehicle 10 is referred to as the inside.

[0021] <Body 20>

[0022] The vehicle body 20 has a front opening portion 21 and a rear opening portion 22 that are open at the side. In addition, the vehicle body 20 has a center pillar 23 that separates the front opening portion 21 and the rear opening portion 22 from each other. The front opening portion 21 and the rear opening portion 22 are adjacent to each other in the front-to-back direction via the center pillar 23. The front opening portion 21 is located in front of the rear opening portion 22. The front opening portion 21 and the rear opening portion 22 are portions through which a user passes when getting on or off the vehicle 10. In the present embodiment, the front opening portion 21 is equivalent to the "second opening portion", and the rear opening portion 22 is equivalent to the "first opening portion".

[0023] <Front Door 31 and Front Door Driving Unit 41>

[0024] The front door 31 has a shape corresponding to the front opening portion 21 when viewed in the width direction. The front door 31 slides in the front-to-rear direction relative to the vehicle body 20, thereby displacing between a fully closed position in which the front opening portion 21 is fully closed and a fully open position in which the front opening portion 21 is fully opened. The opening direction of the front door 31 is the front, and the closing direction of the front door 31 is the rear. That is, the opening direction of the front door 31 is the opposite direction of the closing direction of the front door 31. The front door drive unit 41 drives the front door 31. The front door drive unit 41 opens the front door 31 toward the fully open position, or closes the front door 31 toward the fully closed position. In the present embodiment, the front door 31 is equivalent to the "second door", and the front door drive unit 41 is equivalent to the "door drive unit".

[0025] <Rear Door 32 and Rear Door Driving Unit 42>

[0026] The rear door 32 has a shape corresponding to the rear opening portion 22 when viewed in the width direction. The rear door 32 is displaced between a fully closed position in which the rear opening portion 22 is fully closed and a fully open position in which the rear opening portion 22 is fully opened by sliding in the front-rear direction relative to the vehicle body 20. The opening direction of the rear door 32 is the rear, and the closing direction of the rear door 32 is the front. That is, the opening direction of the rear door 32 is the opposite direction of the closing direction of the rear door 32. The rear door drive unit 42 drives the rear door 32. The rear door drive unit 42 opens the rear door 32 toward the fully open position, or closes the rear door 32 toward the fully closed position. In the present embodiment, the rear door 32 is equivalent to the "first door", and the rear door drive unit 42 is equivalent to the "door drive unit".

[0027] <Camera 50>

[0028] The camera 50 is disposed above the center pillar 23 of the vehicle body 20 so as to face outward in the width direction. That is, in the front-rear direction, the camera 50 is disposed between the front opening 21 and the rear opening 22. The camera 50 can capture the area directly facing the front opening 21 and the rear opening 22 regardless of the opening and closing conditions of the front door 31 and the rear door 32. The camera 50 outputs the captured image to the gesture detection device 70 for each frame. As an example, the frame rate of the camera 50 may be about 30 fps. Figure 2 2 is an example of an image of a user facing the front opening 21 and the rear opening 22 captured by the camera 50. In the following description, facing the front opening 21 and the rear opening 22, in other words, facing the center pillar 23 can be referred to as "facing the side of the vehicle 10". When the user faces the left side of the vehicle 10, the right half of the user faces the rear opening 22, and the left half of the user faces the front opening 21.

[0029] <Door control device 60>

[0030] The door control device 60 is composed of, for example, a processing circuit including a CPU and a memory. The memory includes a RAM and a ROM, etc., and stores a program or instruction configured to cause the CPU to execute a process. The memory, i.e., a non-transient computer-readable medium, includes all available media that can be accessed by a general-purpose or special-purpose computer. The door control device 60 controls the front door drive unit 41 and the rear door drive unit 42 according to the program stored in the memory and the input command signal. In detail, the door control device 60 opens the front door 31 when receiving an opening command signal for opening the front door 31. On the other hand, the door control device 60 closes the front door 31 when receiving a closing command signal for closing the front door 31. In addition, the door control device 60 opens the rear door 32 when receiving an opening command signal for opening the rear door 32. On the other hand, the door control device 60 closes the rear door 32 when receiving a closing command signal for closing the rear door 32.

[0031] <Gesture Detection Device 70>

[0032] The gesture detection device 70 is composed of, for example, a processing circuit including a CPU and a memory. The gesture detection device 70 determines whether the user has made a gesture using a body part according to a program stored in the memory, or outputs a command signal corresponding to the gesture to the door control device 60. That is, the gesture is an opportunity to operate the front door 31 and the rear door 32.

[0033] First, the gestures to be detected by the gesture detection device 70 will be described.

[0034] like Figure 2 As shown, the gesture detected by the gesture detection device 70 is an action of moving the wrist when the user is facing the side of the vehicle 10. When the user is facing the side of the vehicle 10, the face and upper body of the user are facing the side of the vehicle 10. The gesture includes an action of the user moving the right hand in the opening and closing direction of the rear door 32 or the user moving the left hand in the opening and closing direction of the front door 31.

[0035] like Figure 3As shown, the gesture using the right hand includes a right outer gesture and a right inner gesture. The right outer gesture includes the action of moving the right hand toward the outside of the body from the state where the user puts down the right hand, and the right inner gesture includes the action of moving the right hand toward the inside of the body from the state where the user puts down the right hand. Figure 3 In , the right outer gesture includes the action of the user moving the right hand from the position shown by the solid line to the position shown by the single-point dashed line. Figure 3 , the right inner gesture includes the action of the user moving the right hand from the position indicated by the solid line to the position indicated by the double-dotted line.

[0036] The right outside gesture also includes: a right outside reversal gesture of moving the right hand to the outside of the body and then moving the right hand back to the original position, and a right outside hold gesture of stopping the right hand after moving the right hand to the outside of the body. When the user performs the right outside reversal gesture, the user's right hand moves to the position indicated by the solid line after moving from the position indicated by the solid line to the position indicated by the single-dash line. When the user performs the right outside hold gesture, the user's right hand stays at the position indicated by the single-dash line after moving from the position indicated by the solid line to the position indicated by the single-dash line. In the present embodiment, the right outside gesture is equivalent to the "first gesture (first open gesture)", the right outside reversal gesture is equivalent to the "third gesture", and the right outside hold gesture is equivalent to the "fourth gesture".

[0037] The right-inside gesture also includes: a right-inward reversal gesture that moves the right hand to the original position after moving it toward the inside of the body, and a right-inward hold gesture that stops the right hand after moving it toward the inside of the body. When the user performs the right-inward reversal gesture, the user's right hand moves from the position indicated by the solid line to the position indicated by the double-dashed line, and then moves to the position indicated by the solid line. When the user performs the right-inward hold gesture, the user's right hand moves from the position indicated by the solid line to the position indicated by the double-dashed line, and then stays at the position indicated by the double-dashed line. In this embodiment, the right-inside gesture is equivalent to the "first closing gesture".

[0038] like Figure 4 As shown, the gesture using the left hand includes a left outer gesture and a left inner gesture. The left outer gesture includes an action of moving the left hand toward the outside of the body from the state where the user puts down the left hand, and the left inner gesture includes an action of moving the left hand toward the inside of the body from the state where the user puts down the left hand. Figure 4 In FIG, the left outer gesture includes the action of the user moving the left hand from the position indicated by the solid line to the position indicated by the single-point dashed line. Figure 4 , the left inner gesture includes the action of the user moving the left hand from the position indicated by the solid line to the position indicated by the double-dashed line.

[0039] The left outside gesture also includes: a left outside reversal gesture of moving the left hand to the outside of the body and then moving the left hand to the original position, and a left outside hold gesture of stopping the left hand after moving the left hand to the outside of the body. When the user performs the left outside reversal gesture, the user's left hand moves from the position indicated by the solid line to the position indicated by the single-dash line, and then moves to the position indicated by the solid line. When the user performs the left outside hold gesture, the user's left hand moves from the position indicated by the solid line to the position indicated by the single-dash line, and then stays at the position indicated by the single-dash line. In this embodiment, the left outside gesture is equivalent to the "second gesture (second open gesture)", the left outside reversal gesture is equivalent to the "fifth gesture", and the left outside hold gesture is equivalent to the "sixth gesture".

[0040] The left-inside gesture also includes: a left-inside reversal gesture of moving the left hand toward the inside of the body and then moving the left hand back to its original position, and a left-inside hold gesture of moving the left hand toward the inside of the body and then stopping the left hand. When the user performs the left-inside reversal gesture, the user's left hand moves from the position indicated by the solid line to the position indicated by the double-dashed line, and then moves to the position indicated by the solid line. When the user performs the left-inside hold gesture, the user's left hand moves from the position indicated by the solid line to the position indicated by the double-dashed line, and then stays at the position indicated by the double-dashed line. In this embodiment, the left-inside gesture is equivalent to the "second closing gesture".

[0041] like Figure 1 As shown, when the user faces the left side of the vehicle 10, the user's right hand faces the rear opening 22, and the user's left hand faces the front opening 21. In this case, the user's right hand corresponds to the rear door 32, and the user's left hand corresponds to the front door 31. Therefore, on the left side of the vehicle 10, the gesture using the user's right hand is a gesture for at least opening and closing the rear door 32, and the gesture using the user's left hand is a gesture for at least opening and closing the front door 31. Specifically, the right outer reversal gesture is a gesture for opening only the rear door 32 corresponding to the right hand, and the right outer holding gesture is a gesture for opening the front door 31 and the rear door 32. The left outer reversal gesture is a gesture for opening only the front door 31 corresponding to the left hand, and the left outer holding gesture is a gesture for opening the front door 31 and the rear door 32. On the other hand, the right inner reversal gesture is a gesture for closing only the rear door 32 corresponding to the right hand, and the right inner holding gesture is a gesture for closing the front door 31 and the rear door 32. The left inner reversal gesture is a gesture for closing only the front door 31 corresponding to the left hand, and the left inner hold gesture is a gesture for closing both the front door 31 and the rear door 32 .

[0042] On the other hand, when the user faces the right side of the vehicle 10, the user's right hand faces the front opening 21, and the user's left hand faces the rear opening 22. In this case, the user's right hand corresponds to the front door 31, and the user's left hand corresponds to the rear door 32. Therefore, on the right side of the vehicle 10, a gesture using the user's right hand is a gesture for at least opening and closing the front door 31, and a gesture using the user's left hand is a gesture for at least opening and closing the rear door 32.

[0043] In the following description, for easy understanding, it is assumed that the user performs a gesture while facing the left side of the vehicle 10. That is, the gesture detection device 70 detects a gesture for opening and closing the front door 31 and the rear door 32 on the left side of the vehicle 10.

[0044] Next, each function of the gesture detection device 70 will be described.

[0045] like Figure 1 As shown, the gesture detection device 70 includes a storage unit 71 , an image acquisition unit 72 , a position acquisition unit 73 , a direction determination unit 74 , a gesture determination unit 75 , and an output unit 76 .

[0046] <Storage Unit 71>

[0047] The storage unit 71 stores a learned model that has been machine-learned using teacher data that associates images captured in advance with the positions of the user's body parts. That is, the learned model is a model that takes the image captured by the camera 50 as input and outputs the positions of the user's body parts in the image. Specifically, the output is Figure 2 The model of the user's neck position PN, right shoulder position PRs, left shoulder position PLs, right hand position PRh, left hand position PLh, right toe position PRt, and left toe position PLt is shown. In the present embodiment, the right hand corresponds to the "first body part", the left hand corresponds to the "second body part", and the neck, right shoulder, and left shoulder correspond to the "reference body part". The learned model is created, for example, when the vehicle 10 is designed, and is written into the storage unit 71 when the gesture detection device 70 is manufactured.

[0048] The following describes a method for generating a learned model.

[0049] The method for generating a learned model includes a preparation step for preparing teacher data and a learning step for performing machine learning based on the teacher data. The preparation step includes an acquisition step for acquiring images captured under various conditions with a user standing in a shooting area and a designation step for designating positions of body parts of the user in a plurality of images acquired in the acquisition step.

[0050] The acquisition process is implemented, for example, using an actual vehicle 10. The acquisition process preferably acquires a plurality of images captured by changing conditions related to the user and conditions related to the environment around the vehicle 10. In this way, a learned model that can cope with various situations, in other words, a learned model with high versatility, can be obtained. The designation process designates the position of the user's body part with respect to the acquired image. The position can be designated using, for example, the coordinates of the pixels in the image. In this embodiment, the axis extending horizontally in the image is referred to as the X-axis, and the axis extending vertically in the image is referred to as the Y-axis.

[0051] The learning process generates a model by machine learning using multiple teacher data as learning data. Various machine learning techniques can be selected, such as a convolutional neural network (CNN). The learned model outputs the position of the user's body part captured in the image by inputting the captured image. On the other hand, when the learned model is input with an image that does not capture the user's body part, it cannot output the position of the user's body part. In addition, when the accuracy of the learned model is insufficient, the learned model may incorrectly output the position of the body part.

[0052] <Image Acquisition Unit 72>

[0053] The image acquisition unit 72 acquires an image captured by the camera 50 at each capturing cycle of the camera 50. For example, if the frame rate of the camera 50 is 30 fps, the image acquisition unit 72 acquires an image captured by the camera 50 every 1 / 30 second.

[0054] <Position Acquisition Unit 73>

[0055] The position acquisition unit 73 acquires the position of the user's body part captured in the image captured by the camera 50 by inputting the learned model stored in the storage unit 71. That is, the position acquisition unit 73 acquires the position PN of the user's neck, the position PRs of the right shoulder, the position PLs of the left shoulder, the position PRh of the right hand, the position PLh of the left hand, the position PRt of the right toe, and the position PLt of the left toe. In the present embodiment, the position PRh of the right hand corresponds to the "first position", the position PLh of the left hand corresponds to the "second position", and the position PN of the user's neck, the position PRs of the right shoulder, and the position PLs of the left shoulder correspond to the "reference position".

[0056] <Direction determination unit 74>

[0057] The direction determination unit 74 determines whether the user captured in the image is facing the side of the vehicle 10 based on the right shoulder position PRs and the left shoulder position PLs. Figure 2As shown, the direction determination unit 74 determines whether the user is facing the side of the vehicle 10 based on the shoulder width Ls which is the distance between the right shoulder position PRs and the left shoulder position PLs in the X direction.

[0058] When the user is at the side of the vehicle 10 and faces the side of the vehicle 10, the camera 50 photographs the user from the front. Therefore, the shoulder width Ls calculated by the direction determination unit 74 is a relatively large value. On the other hand, when the user is at the side of the vehicle 10 and faces slightly forward of the front opening 21 or slightly backward of the rear opening 22, the camera 50 photographs the user from the right front or the left front. Therefore, the shoulder width Ls calculated by the direction determination unit 74 is a relatively small value.

[0059] Therefore, when the calculated shoulder width Ls is greater than the preset shoulder width determination value Lsth, the direction determination unit 74 determines that the user is facing the side of the vehicle 10. On the other hand, when the calculated shoulder width Ls is less than the shoulder width determination value Lsth, the direction determination unit 74 determines that the user is not facing the side of the vehicle 10. The shoulder width determination value Lsth is a value preset through experiments and simulations.

[0060] In addition, when the distance from the camera 50 to the user is short, the shoulder width Ls is easily calculated to be large, and when the distance from the camera 50 to the user is long, the shoulder width Ls is easily calculated to be small. Therefore, the shoulder width determination value Lsth can be a variable value corresponding to the distance from the camera 50 to the user. In addition, the shoulder width Ls also varies according to the size of the user's physique. Therefore, in the case where the direction determination unit 74 can determine the size of the user's physique based on the positions of multiple body parts, the shoulder width determination value Lsth can be a variable value corresponding to the user's physique.

[0061] When the user is at the side of the vehicle 10 and faces the front of the vehicle 10 or the rear of the vehicle 10, the camera 50 captures the user from the side. In this case, the position acquisition unit 73 cannot acquire the positions of both the position PRs of the right shoulder and the position PLs of the left shoulder of the user. That is, the direction determination unit 74 cannot calculate the shoulder width Ls. Therefore, when the position acquisition unit 73 cannot acquire at least one of the position PRs of the right shoulder and the position PLs of the left shoulder, in other words, when the direction determination unit 74 cannot calculate the shoulder width Ls, the direction determination unit 74 determines that the user is not facing the side of the vehicle 10.

[0062] <Gesture determination unit 75>

[0063] The gesture determination unit 75 performs an approach determination process for determining whether the user has approached the vehicle 10 and a gesture determination process for determining whether the user has performed a gesture.

[0064] The gesture determination unit 75 determines in the approach determination process whether the position PRt of the right toe and the position PLt of the left toe of the user acquired by the position acquisition unit 73 are present. Figure 2 The judgment area Ax is an area set in the image. Figure 2 In the figure, the determination area Ax is an area below the line segment indicated by the single-dot dash line in the image. The determination area Ax can be a rectangular area or a semicircular area. The lengths of the determination area Ax in the X direction and the Y direction can be appropriately changed. Furthermore, when at least one of the position PRt of the right toe and the position PLt of the left toe of the user exists in the determination area Ax, the gesture determination unit 75 starts the gesture determination process. In addition, the gesture determination unit 75 can also start the approach determination process only when a user holding a portable device bound to the vehicle 10 approaches the vehicle 10. Thus, the gesture determination process is suppressed from being implemented for the gestures of a person who is not a user of the vehicle 10.

[0065] In the gesture determination process, the gesture determination unit 75 determines whether the user has performed a gesture based on a plurality of images taken at different times and changes in the position of the body part in the plurality of images taken continuously. The gesture determination process includes a right-outer gesture determination process, a right-inner gesture determination process, a left-outer gesture determination process, and a left-inner gesture determination process. The right-outer gesture determination process determines whether the user has performed a right-outer reversal gesture or a right-outer hold gesture, and the right-inner gesture determination process determines whether the user has performed a right-inner reversal gesture or a right-inner hold gesture. The left-outer gesture determination process determines whether the user has performed a left-outer reversal gesture or a left-outer hold gesture, and the left-inner gesture determination process determines whether the user has performed a left-inner reversal gesture or a left-inner hold gesture.

[0066] In the right outer gesture determination process and the right inner gesture determination process, the gesture determination unit 75 calculates the reference distance LR from the neck position PN to the right shoulder position PRs in the X direction based on the neck position PN and the right shoulder position PRs acquired by the position acquisition unit 73. Figure 3 As shown, the gesture determination unit 75 sets the right reference area AR1 , the right outer area AR2 , and the right inner area AR3 in the image based on the neck position PN , the right shoulder position PRs , and the reference distance LR.

[0067] The right reference area AR1 includes the position PRs of the right shoulder and is an area extending to the left and right from the position PRs of the right shoulder. The width of the right reference area AR1 is shorter than the reference distance LR. As an example, the right reference area AR1 is an area centered on the position PRs of the right shoulder, and the width of the right reference area AR1 is about 40% of the reference distance LR. The right outer area AR2 is an area extending to the left and right on the right side of the right reference area AR1. The width of the right outer area AR2 is longer than the reference distance LR. As an example, the width of the right outer area AR2 is about 140% of the reference distance LR. The right inner area AR3 includes the position PN of the neck and is an area extending to the left and right from the position PN of the neck. The width of the right inner area AR3 is the same as the reference distance LR. As an example, the right inner area AR3 is an area centered on the position PN of the neck, and the width of the right inner area AR3 is about 100% of the reference distance LR. It is preferred to set a space that does not belong to any area between the right reference area AR1 and the right outer area AR2 and between the right reference area AR1 and the right inner area AR3.

[0068] Furthermore, when the user performs a gesture while approaching, the reference distance LR of the user as seen from the camera 50 changes over time. That is, the position PN of the neck and the position PRs of the right shoulder in the image change relative to the shooting time. Therefore, the reference distance LR is calculated each time a new image is shot, and the area is set each time a new image is shot.

[0069] When the setting of the above-mentioned area is completed, the gesture determination unit 75 determines whether the user has performed a gesture using the right hand based on the movement pattern of the position PRh of the user's right hand in the above-mentioned area.

[0070] In the right outside gesture determination process, when the position PRh of the user's right hand moves from the right reference area AR1 to the right outside area AR2 over time and then returns to the right reference area AR1, the gesture determination unit 75 determines that the user has performed a right outside reversal gesture. In other words, when the position PRh of the user's right hand moves in the opening direction of the rear door 32 and then moves in the closing direction of the rear door 32, the gesture determination unit 75 determines that the user has performed a right outside reversal gesture. In addition, when the position PRh of the user's right hand moves from the right reference area AR1 to the right outside area AR2 over time and then stays in the right outside area AR2, the gesture determination unit 75 determines that the user has performed a right outside hold gesture. In other words, when the position PRh of the user's right hand moves in the opening direction of the rear door 32 and then stops, the gesture determination unit 75 determines that the user has performed a right outside hold gesture. In this regard, in the present embodiment, the opening direction of the rear door 32 is equivalent to the "first direction" corresponding to the opening direction of the rear door 32.

[0071] In the right inner gesture determination process, when the position PRh of the user's right hand moves from the right reference area AR1 to the right inner area AR3 over time and then returns to the right reference area AR1, the gesture determination unit 75 determines that the user has performed a right inner reversal gesture. In other words, when the position PRh of the user's right hand moves in the closing direction of the rear door 32 and then moves in the opening direction of the rear door 32, the gesture determination unit 75 determines that the user has performed a right inner reversal gesture. In addition, when the position PRh of the user's right hand moves from the right reference area AR1 to the right inner area AR3 over time and then stays in the right inner area AR3, the gesture determination unit 75 determines that the user has performed a right inner hold gesture. In other words, when the position PRh of the user's right hand moves in the closing direction of the rear door 32 and then stops, the gesture determination unit 75 determines that the user has performed a right inner hold gesture.

[0072] As described above, in the present embodiment, the right reference area AR1, the right outer area AR2, and the right inner area AR3 are set based on the neck position PN and the right shoulder position PRs. Therefore, the movement of the right hand position PRh to the adjacent area on the right side is equivalent to the movement of the right hand position PRh relative to the neck position PN and the right shoulder position PRs in the opening direction of the rear door 32. In addition, the movement of the right hand position PRh to the adjacent area on the left side is equivalent to the movement of the right hand position PRh relative to the neck position PN and the right shoulder position PRs in the closing direction of the rear door 32.

[0073] In the left outer gesture determination process and the left inner gesture determination process, the gesture determination unit 75 calculates the reference distance LL from the neck position PN to the left shoulder position PLs in the X direction based on the neck position PN and the left shoulder position PLs acquired by the position acquisition unit 73. Figure 4 As shown, the gesture determination unit 75 sets the left reference area AL1 , the left outer area AL2 , and the left inner area AL3 in the image based on the neck position PN, the left shoulder position PLs, and the reference distance LL.

[0074] The left reference area AL1 includes the position PLs of the left shoulder and is an area extending to the left and right from the position PLs of the left shoulder. The width of the left reference area AL1 is shorter than the reference distance LL. As an example, the left reference area AL1 is an area centered on the position PLs of the left shoulder, and the width of the left reference area AL1 is about 40% of the reference distance LL. The left outer area AL2 is an area extending to the left and right on the left side of the left reference area AL1. The width of the left outer area AL2 is longer than the reference distance LL. As an example, the width of the left outer area AL2 is about 140% of the reference distance LL. The left inner area AL3 includes the position PN of the neck and is an area extending to the left and right from the position PN of the neck. The width of the left inner area AL3 is the same as the reference distance LL. As an example, the left inner area AL3 is an area centered on the position PN of the neck, and the width of the left inner area AL3 is about 100% of the reference distance LL. When the reference distance LL and the reference distance LR in the image are equal, the left inner area AL3 overlaps with the right inner area AR3. It is preferable to provide a space not belonging to any area between the left reference area AL1 and the left outer area AL2 and between the left reference area AL1 and the left inner area AL3.

[0075] Furthermore, when the user performs a gesture while approaching, the reference distance LL of the user as seen from the camera 50 changes over time. That is, the position PN of the neck and the position PLs of the left shoulder in the image change relative to the shooting time. Therefore, the reference distance LL is calculated each time a new image is shot, and the area is set each time a new image is shot.

[0076] When the setting of the above-mentioned area is completed, the gesture determination unit 75 determines whether the user has performed a gesture using the left hand based on the movement pattern of the position PLh of the user's left hand in the above-mentioned area.

[0077] In the left outside gesture determination process, when the position PLh of the user's left hand moves from the left reference area AL1 to the left outside area AL2 over time and then returns to the left reference area AL1, the gesture determination unit 75 determines that the user has performed a left outside reversal gesture. In other words, when the position PLh of the user's left hand moves toward the opening direction of the front door 31 and then moves toward the closing direction of the front door 31, the gesture determination unit 75 determines that the user has performed a left outside reversal gesture. In addition, when the position PLh of the user's left hand moves from the left reference area AL1 to the left outside area AL2 over time and then stays in the left outside area AL2, the gesture determination unit 75 determines that the user has performed a left outside hold gesture. In other words, when the position PLh of the user's left hand moves toward the opening direction of the front door 31 and then stops, the gesture determination unit 75 determines that the user has performed a left outside hold gesture. In this regard, in the present embodiment, the opening direction of the front door 31 is equivalent to the "second direction" corresponding to the opening direction of the front door 31.

[0078] In the left inner gesture determination process, when the position PLh of the user's left hand moves from the left reference area AL1 to the left inner area AL3 over time and then returns to the left reference area AL1, the gesture determination unit 75 determines that the user has performed a left inner reversal gesture. In other words, when the position PLh of the user's left hand moves toward the closing direction of the front door 31 and then moves toward the opening direction of the front door 31, the gesture determination unit 75 determines that the user has performed a left inner reversal gesture. In addition, when the position PLh of the user's left hand moves from the left reference area AL1 to the left inner area AL3 over time and then stays in the left inner area AL3, the gesture determination unit 75 determines that the user has performed a left inner hold gesture. In other words, when the position PLh of the user's left hand moves toward the closing direction of the front door 31 and then stops, the gesture determination unit 75 determines that the user has performed a left inner hold gesture.

[0079] As described above, in the present embodiment, the left reference area AL1, the left outer area AL2, and the left inner area AL3 are set based on the neck position PN and the left shoulder position PLs. Therefore, the movement of the left hand position PLh to the adjacent area on the left side is equivalent to the movement of the left hand position PLh relative to the neck position PN and the left shoulder position PLs in the opening direction of the front door 31. In addition, the movement of the left hand position PLh to the adjacent area on the right side is equivalent to the movement of the left hand position PLh relative to the neck position PN and the left shoulder position PLs in the closing direction of the front door 31.

[0080] <Output section 76>

[0081] When it is determined that the user has performed a right-outer reversal gesture, the output unit 76 outputs an opening motion command signal for opening the rear door 32 corresponding to the right hand. In addition, when it is determined that the user has performed a right-outer hold gesture, the output unit 76 outputs an opening motion command signal for opening the front door 31 and the rear door 32. On the other hand, when it is determined that the user has performed a right-inward reversal gesture, the output unit 76 outputs a closing motion command signal for closing the rear door 32 corresponding to the right hand. In addition, when it is determined that the user has performed a right-inward hold gesture, the output unit 76 outputs a closing motion command signal for closing the front door 31 and the rear door 32.

[0082] When it is determined that the user has performed a left-outer reversal gesture, the output unit 76 outputs an opening motion command signal for opening the front door 31 corresponding to the left hand. In addition, when it is determined that the user has performed a left-outer hold gesture, the output unit 76 outputs an opening motion command signal for opening the front door 31 and the rear door 32. On the other hand, when it is determined that the user has performed a left-inner reversal gesture, the output unit 76 outputs a closing motion command signal for closing the front door 31 corresponding to the left hand. In addition, when it is determined that the user has performed a left-inner hold gesture, the output unit 76 outputs a closing motion command signal for closing the front door 31 and the rear door 32.

[0083] That is, when the gesture determination unit 75 determines that the right outer hold gesture and the left outer hold gesture are implemented, the output unit 76 outputs the same signal. In addition, when the gesture determination unit 75 determines that the right inner hold gesture and the left inner hold gesture are implemented, the output unit 76 outputs the same signal.

[0084] <Processing by the Gesture Detection Device 70>

[0085] Below, refer to Figure 5 The flowchart shown in FIG. 1 illustrates the flow of the right outside gesture determination process for determining whether a right outside flip gesture or a right outside hold gesture has been performed. The right outside gesture determination process is a process that is performed every predetermined control cycle when the user is present in the shooting area of ​​the camera 50. Figure 5 In the flowchart shown, in the step of implementing the determination using the position of the body part, or the determination using the right reference area AR1 and the right outer area AR2, the position of the body part is acquired again for the newly acquired image, or the above-mentioned areas are set again.

[0086] The right outside gesture determination process is substantially the same as the left outside gesture determination process for determining whether a left outside reversal gesture or a left outside hold gesture has been performed, except that the hand used for the gesture is different and the door to be opened is different. Therefore, the description of the left outside gesture determination process is omitted.

[0087] like Figure 5 As shown, the gesture detection device 70 determines whether the right toe position PRt and the left toe position PLt in the newly captured image are within the determination area Ax (S11). When neither the right toe position PRt nor the left toe position PLt is within the determination area Ax (S11: No), the gesture detection device 70 temporarily terminates the present process. On the other hand, when it is determined that at least one of the right toe position PRt and the left toe position PLt is within the determination area Ax (S11: Yes), the gesture detection device 70 determines whether the rear door 32 is in the fully open position (S12). When the rear door 32 is in the fully open position (S12: Yes), in other words, when the rear door 32 does not need to be opened, the gesture detection device 70 temporarily terminates the present process.

[0088] On the other hand, when the rear door 32 is not in the fully open position (S12: No), the gesture detection device 70 determines whether the user is facing the side of the vehicle 10 based on the shoulder width Ls, which is the interval between the right shoulder position PRs and the left shoulder position PLs (S13). When the user is not facing the side of the vehicle 10 (S13: No), the gesture detection device 70 temporarily ends the present process. On the other hand, when the user is facing the side of the vehicle 10 (S13: Yes), the gesture detection device 70 determines whether the right hand position PRh is present in the right reference area AR1 (S14).

[0089] When the right hand position PRh does not exist in the right reference area AR1 (S14), the gesture detection device 70 temporarily ends the present process. On the other hand, when the right hand position PRh exists in the right reference area AR1 (S14: Yes), the gesture detection device 70 determines whether the right hand position PRh moves to the right outer area AR2 within the first determination time T1 (S15). Here, within the first determination time T1 means that the elapsed time from the time when the affirmative determination is made in step S14 is within the first determination time T1. In addition, in step S15, the movement determination of the right hand position PRh is repeatedly performed for the newly captured image.

[0090] If the position PRh of the right hand has not moved to the right outer area AR2 within the first determination time T1 (S15: No), the gesture detection device 70 temporarily ends the present process. On the other hand, if the position PRh of the right hand has moved to the right outer area AR2 within the first determination time T1 (S15: Yes), the gesture detection device 70 determines whether the position PRh of the right hand has moved to the right reference area AR1 within the second determination time T2 (S16). Here, within the second determination time T2 means that the elapsed time from the moment when the position PRh of the right hand has moved to the right outer area AR2 is within the second determination time T2. In addition, in step S16, the movement determination of the position PRh of the right hand is repeatedly performed on the newly captured image in the same manner as step S15.

[0091] When the position PRh of the right hand moves to the right reference area AR1 within the second determination time T2 (S16: Yes), that is, when the user performs the right outer reversal gesture, the gesture detection device 70 outputs the opening action command signal of the rear door 32 to the door control device 60 (S17). Thereafter, the gesture detection device 70 ends the present process. On the other hand, when the position PRh of the right hand does not move to the right reference area AR1 within the second determination time T2 (S16: No), the gesture detection device 70 determines whether the position PRh of the right hand continues to exist in the right outer area AR2 for the third determination time T3 (S18). Here, the third determination time T3 means that the time elapsed from the moment when the position PRh of the right hand moves to the right outer area AR2 is the third determination time T3. Therefore, the third determination time T3 is a time longer than the second determination time T2. In addition, in step S18, the movement determination of the position PRh of the right hand is repeatedly performed on the newly captured image in the same manner as step S15.

[0092] If the right hand position PRh does not continue to exist in the right outer area AR2 for the third determination time T3 (S18: No), that is, if the right hand position PRh moves outside the right outer area AR2 within the third determination time T3, the gesture detection device 70 temporarily ends the present process. On the other hand, if the right hand position PRh continues to stay in the right outer area AR2 for the third determination time T3 (S18: Yes), the gesture detection device 70 outputs the opening action instruction signal of the front door 31 and the rear door 32 (S19). The case where the step S18 makes an affirmative determination is the case where it is determined that the user has performed the right outer holding gesture. Thereafter, the gesture detection device 70 ends the present process.

[0093] exist Figure 5 In the flowchart shown, steps S11, S14 to S16, and S18 include a "position acquisition step", step S13 corresponds to a "direction determination step", and steps S14 to S16 and S18 correspond to a "gesture determination step".

[0094] Below, refer to Figure 6 The flowchart shown in FIG. 1 illustrates the flow of the right inner gesture determination process for determining whether a right inner flip gesture or a right inner hold gesture has been performed. The right inner gesture determination process is a process that is performed every predetermined control cycle when the user is present in the shooting area of ​​the camera 50. Figure 6 In the flowchart shown, in the step of implementing the determination using the position of the body part or the determination using the right reference area AR1 and the right inner area AR3, the position of the body part is acquired again for the newly acquired image, or the above-mentioned areas are set again.

[0095] In addition, the right inner gesture determination process is substantially the same as the left inner gesture determination process for determining whether the left inner reversal gesture or the left inner hold gesture has been performed, except that the hand used for the gesture is different and the door to be closed is different. Therefore, the description of the left inner gesture determination process is omitted.

[0096] like Figure 6 As shown, the gesture detection device 70 determines whether the right toe position PRt and the left toe position PLt in the newly captured image are within the determination area Ax (S21). When neither the right toe position PRt nor the left toe position PLt is within the determination area Ax (S21: No), the gesture detection device 70 temporarily terminates the present process. On the other hand, when at least one of the right toe position PRt and the left toe position PLt is within the determination area Ax (S21: Yes), the gesture detection device 70 determines whether the rear door 32 is in the fully closed position (S22). When the rear door 32 is in the fully closed position (S22: Yes), in other words, when the rear door 32 does not need to be closed, the gesture detection device 70 temporarily terminates the present process.

[0097] On the other hand, when the rear door 32 is not in the fully closed position (S22: No), the gesture detection device 70 determines whether the user is facing the side of the vehicle 10 based on the shoulder width Ls, which is the interval between the right shoulder position PRs and the left shoulder position PLs (S23). When the user is not facing the side of the vehicle 10 (S23: No), the gesture detection device 70 temporarily ends the present process. On the other hand, when the user is facing the side of the vehicle 10 (S23: Yes), the gesture detection device 70 determines whether the right hand position PRh is present in the right reference area AR1 (S24).

[0098] If the right hand position PRh does not exist in the right reference area AR1 (S24: No), the gesture detection device 70 temporarily ends the present process. On the other hand, if the right hand position PRh exists in the right reference area AR1 (S24: Yes), the gesture detection device 70 determines whether the right hand position PRh moves to the right inner area AR3 within the first determination time T1 (S25). Here, within the first determination time T1 means that the elapsed time from the time when the affirmative determination is made in step S24 is within the first determination time T1. In addition, in step S25, the movement determination of the right hand position PRh is repeatedly performed on the newly captured image.

[0099] If the position PRh of the right hand has not moved to the right inner area AR3 within the first determination time T1 (S25: No), the gesture detection device 70 temporarily terminates the present process. On the other hand, if the position PRh of the right hand has moved to the right inner area AR3 within the first determination time T1 (S25: Yes), the gesture detection device 70 determines whether the position PRh of the right hand has moved to the right reference area AR1 within the second determination time T2 (S26). Here, within the second determination time T2 means that the time elapsed from the moment when the position PRh of the right hand has moved to the right inner area AR3 is within the second determination time T2. In addition, in step S26, the movement determination of the position PRh of the right hand is repeatedly performed on the newly captured image in the same manner as step S25.

[0100] When the position PRh of the right hand moves to the right reference area AR1 within the second determination time T2 (S26: Yes), that is, when the user performs the right inner reversal gesture, the gesture detection device 70 outputs a closing operation instruction signal of the rear door 32 to the door control device 60 (S27). Thereafter, the gesture detection device 70 ends the present process. On the other hand, when the position PRh of the right hand does not move to the right reference area AR1 within the second determination time T2 (S26: No), the gesture detection device 70 determines whether the position PRh of the right hand continues to exist in the right inner area AR3 for a third determination time T3 (S28). Here, the third determination time T3 means that the time elapsed from the moment when the position PRh of the right hand moves to the right inner area AR3 is the third determination time T3. Therefore, the third determination time T3 is a time longer than the second determination time T2. In addition, in step S28, the movement determination of the position PRh of the right hand is repeatedly performed on the newly captured image in the same manner as step S25.

[0101] If the right hand position PRh does not continue to exist in the right inner area AR3 for the third determination time T3 (S28: No), that is, if the right hand position PRh moves outside the right inner area AR3 within the third determination time T3, the gesture detection device 70 temporarily ends the present process. On the other hand, if the right hand position PRh continues to stay in the right inner area AR3 for the third determination time T3 (S28: Yes), the gesture detection device 70 outputs a closing operation instruction signal for the front door 31 and the rear door 32 (S29). If step S28 makes an affirmative determination, it is determined that the user has performed the right inner holding gesture. Thereafter, the gesture detection device 70 ends the present process.

[0102] exist Figure 6 In the flowchart shown, steps S21, S24 to S26, and S28 include a "position acquisition step", step S23 corresponds to a "direction determination step", and steps S24 to S26, and S28 correspond to a "gesture determination step".

[0103] <Function of this embodiment>

[0104] The operation when the door of the parked vehicle 10 is opened will be described.

[0105] When the user wants to open the rear door 32 of the vehicle 10, the user performs a right-outer reversal gesture while walking toward the left side of the vehicle 10. That is, after the user moves the right hand near the rear door 32 side in the opening direction of the rear door 32, the right hand returns to the original position. Then, the rear door 32 opens. On the other hand, when the user wants to open the front door 31 of the vehicle 10, the user performs a left-outer reversal gesture while walking toward the left side of the vehicle 10. That is, after the user moves the left hand near the front door 31 side in the opening direction of the front door 31, the left hand returns to the original position. Then, the front door 31 opens.

[0106] In addition, when the user wants to open both the front door 31 and the rear door 32 of the vehicle 10, the user performs a right outer holding gesture or a left outer holding gesture while walking toward the side of the vehicle 10. That is, the user moves the right hand in the opening direction of the rear door 32 and then stops, or moves the left hand in the opening direction of the front door 31 and then stops. Then, the front door 31 and the rear door 32 are opened.

[0107] <Effects of the present embodiment>

[0108] (1) When the user opens the rear door 32, the user performs a right-outer reversal gesture while facing the left side of the vehicle 10. That is, when the user opens the rear door 32, the user performs a gesture including an action of moving the right hand near the rear door 32 in the opening direction of the rear door 32. In addition, when the user opens the front door 31, the user performs a left-outer reversal gesture while facing the left side of the vehicle 10. That is, when the user opens the front door 31, the user performs a gesture including an action of moving the left hand near the front door 31 in the opening direction of the front door 31. Therefore, when the user wants to open one of the two doors, he or she can perform a gesture using the hand near the door, that is, a gesture of moving the hand in the opening direction of the door. Therefore, the gesture detection device 70 can improve the convenience of the user.

[0109] In addition, the gesture detection device 70 starts determining whether the user has made a gesture when it can be determined that the user is facing the side of the vehicle 10. Therefore, the gesture detection device 70 can reduce the risk of erroneously determining that the user has made a gesture based on the movement of a person passing by the side of the vehicle 10 while swinging his wrist.

[0110] (2) When the user makes a gesture while approaching the vehicle 10, both the user making the gesture and the user approaching the vehicle 10 will cause the right hand position PRh and the left hand position PLh in the image to change. In this regard, the gesture detection device 70 determines whether the user has made a gesture based on the changes in the right hand position PRh and the left hand position PLh relative to the neck position PN, the right shoulder position PRs, and the left shoulder position PLs. Therefore, regardless of whether the user makes a gesture while approaching the vehicle 10 or the user makes a gesture while standing still, the gesture detection device 70 can accurately determine whether the user has made a gesture.

[0111] (3) When the gesture detection device 70 determines that the user has performed a right-outer reversal gesture, it outputs an opening action command signal for the rear door 32. In addition, when the gesture detection device 70 determines that the user has performed a left-outer determination gesture, it outputs an opening action command signal for the front door 31. The right-outer reversal gesture and the left-outer reversal gesture include an action of moving the right hand or the left hand in one direction and an action of returning the right hand or the left hand to its original position. Therefore, compared with the case where only the action of moving the right hand or the left hand in one direction is used as a gesture for opening the front door 31 and the rear door 32, it becomes difficult for the gesture detection device 70 to misjudge the gesture.

[0112] (4) When the gesture detection device 70 determines that the user has performed a right-outer hold gesture or a left-outer hold gesture, the gesture detection device 70 outputs an opening action instruction signal for the front door 31 and the rear door 32. The right-outer hold gesture and the left-outer hold gesture include an action of moving the right hand or the left hand in one direction and an action of stopping the right hand or the left hand. Therefore, compared with the case where only the action of moving the right hand or the left hand in one direction is used as a gesture for opening the front door 31 and the rear door 32, the gesture detection device 70 becomes less likely to misjudge the gesture.

[0113] (5) The user can select to open only the rear door 32 or both the front door 31 and the rear door 32 by using the right-outer reversal gesture and the right-outer hold gesture separately. In addition, the user can select to open only the front door 31 or both the front door 31 and the rear door 32 by using the left-outer reversal gesture and the left-outer hold gesture separately. Therefore, the gesture detection device 70 can further improve the convenience of the user.

[0114] (6) The gesture detection device 70 determines whether the user is facing the side of the vehicle 10 based on the shoulder width Ls, which is the distance between the right shoulder position PRs and the left shoulder position PLs. Therefore, the gesture detection device 70 can accurately determine whether the user is facing the side of the vehicle 10.

[0115] (7) In the above description of the effects, the effects are described for the case where the user performs the right-outer reversal gesture, the right-outer hold gesture, the left-outer reversal gesture, and the left-outer hold gesture in order to open at least one of the front door 31 and the rear door 32. Such effects are also effective when the user performs the right-inner reversal gesture, the right-inner hold gesture, the left-inner reversal gesture, and the left-inner hold gesture in order to close at least one of the front door 31 and the rear door 32. In addition, the gesture detection device 70 can open or close the front door 31 and the rear door 32 according to the direction of movement of the user's hand.

[0116] (8) Since the camera 50 is disposed between the front opening 21 and the rear opening 22, when the camera 50 captures a user facing the side of the vehicle 10, the right and left halves of the user are easily reflected equally in the image. Therefore, the gesture detection device 70 can easily acquire the right hand position PRh, the right shoulder position PRs, the right toe position PRt, the left hand position PLh, the left shoulder position PLs, and the left toe position PLt.

[0117] The present embodiment can be implemented by modification as follows. The present embodiment and the following modification examples can be implemented in combination with each other within a range without technical contradiction.

[0118] The body part of the user used for the gesture can be changed appropriately. The first body part is preferably a part of the right half of the user's body that the user can easily move alone. For example, the first body part can be the toe of the user's right foot. The second body part is preferably a part of the left half of the user's body that the user can easily move alone. For example, the second body part can be the toe of the user's left foot. The reference body part can be a body part on the center line of the user's body. For example, the reference body part can be the user's face instead of the user's neck, or it can be the user's chest.

[0119] The direction determination unit 74 may use other determination methods to determine whether the user is facing the side of the vehicle 10. Several examples will be described below.

[0120] The direction determination unit 74 may determine that the user is facing the side of the vehicle 10 when the right shoulder position PRs is present on one side of the neck position PN and the left shoulder position PRs is present on the other side of the neck position PN in the X direction. On the other hand, the direction determination unit 74 may determine that the user is not facing the side of the vehicle 10 when the right shoulder position PRs or the left shoulder position PLs is present only on one side of the neck position PN in the X direction. However, even when the right shoulder position PRs and the left shoulder position PRs are present on both sides of the neck position PN, the middle position between the right shoulder position PRs and the left shoulder position PLs may be offset from the neck position PN in the X direction. In this case, the direction determination unit 74 may determine that the user is not facing the side of the vehicle 10.

[0121] The direction determination unit 74 may also determine whether the user is facing the side of the vehicle 10 based on the position of the body parts other than the right shoulder and the left shoulder. For example, the direction determination unit 74 may also determine whether the user is facing the side of the vehicle 10 based on the position of the right eye and the position of the left eye of the user. In addition, the direction determination unit 74 may also determine whether the user is facing the side of the vehicle 10 based on the position of the right ear and the position of the left ear of the user.

[0122] The direction determination unit 74 may also determine that the user is facing the side of the vehicle 10 when the positions of the plurality of body parts that can be photographed by the camera 50 can be fully acquired because the user is facing the side of the vehicle 10. For example, the direction determination unit 74 may determine that the user is facing the side of the vehicle 10 when the position PN of the neck, the position PRs of the right shoulder, the position PLs of the left shoulder, the position PRh of the right hand, the position PLh of the left hand, the position PRt of the right toe, and the position PLt of the left toe can be fully acquired.

[0123] The gesture determination unit 75 may not set the right reference area AR1 and the right outer area AR2 in the right outer gesture determination process. In this case, the gesture determination unit 75 calculates a reference distance LR, which is a distance from the position PN of the neck to the position PRs of the right shoulder in the X direction, and a comparison distance, which is a distance from the position PN of the neck to the position PRh of the right hand in the X direction. Then, the gesture determination unit 75 determines the state of the user's right hand based on the magnitude relationship between the reference distance LR and the comparison distance. Specifically, when the comparison distance changes from a state where the comparison distance is equal to the reference distance LR to a state where the comparison distance is sufficiently larger than the reference distance LR, the gesture determination unit 75 determines that the user's right hand moves in the opening direction of the rear door 32. In this way, the gesture determination unit 75 determines whether the user has performed a gesture by comparing the distances. The same is true in the right inner gesture determination process, the left outer gesture determination process, and the left inner gesture determination process.

[0124] The gestures to be determined by gesture determination unit 75 may include at least a right-out gesture and a left-out gesture.

[0125] The output unit 76 may output an opening instruction signal for opening the front door 31 and the rear door 32 when it is determined that the user has performed a right-outer reversal gesture. The output unit 76 may also output an opening instruction signal for opening the rear door 32 corresponding to the right hand when it is determined that the user has performed a right-outer hold gesture. In addition, the output unit 76 may also output a closing instruction signal for closing the front door 31 and the rear door 32 when it is determined that the user has performed a right-inner reversal gesture. The output unit 76 may also output an opening instruction signal for closing the rear door 32 corresponding to the right hand when it is determined that the user has performed a right-inner hold gesture. The same applies to the cases where the user performs a left-outer reversal gesture, a left-outer hold gesture, a left-inner reversal gesture, and a left-outer hold gesture.

[0126] The user's gesture for opening the rear door 32 may also be a gesture in which the user simply moves the right hand that has been lowered downward to the right. In this case, the gesture determination unit 75 may determine that the user has performed the above gesture when the position PRh of the user's right hand moves from the right reference area AR1 to the right outer area AR2. The same applies to the user's gesture for opening the front door 31.

[0127] The user's gesture for opening the rear door 32 may also be a gesture in which the user simply moves the right hand that has been lowered downward upward. In this case, the upward direction of the movement of the user's right hand corresponds to the "first direction" corresponding to the opening direction of the rear door 32. That is, the "first direction" does not need to be the same direction as the opening direction of the rear door 32. The same applies to the user's gesture for opening the front door 31.

[0128] The vehicle body 20 may not include the center pillar 23. In this case, the front opening 21 and the rear opening 22 may be integrated into one opening. That is, the front side portion of one opening corresponds to the front opening 21, and the rear side portion of one opening corresponds to the rear opening 22.

[0129] The gesture detection device 70 may be configured to detect only a gesture that triggers the operation of the rear door 32. In other words, in the above-mentioned embodiment, the front door 31 may not be opened or closed by a gesture. In this case, the rear opening 22 is equivalent to the "opening" and the rear door 32 is equivalent to the "door". In addition, in this case, when the user is facing the left side of the vehicle 10, for example, the right hand is equivalent to the "prescribed body part", the position PRh of the right hand is equivalent to the "prescribed position", the opening direction of the rear door 32 is equivalent to the "prescribed direction" corresponding to the opening direction of the rear door 32, and the right outer gesture is equivalent to the "prescribed gesture".

[0130] In addition, the gesture detection device 70 may be configured to detect only a gesture that triggers the front door 31 to operate. In other words, in the above-mentioned embodiment, the rear door 32 may not be opened or closed by a gesture. In this case, the front opening 21 is equivalent to the "opening" and the front door 31 is equivalent to the "door". In addition, in this case, when the user is facing the left side of the vehicle 10, for example, the left hand is equivalent to the "prescribed body part", the position PLh of the left hand is equivalent to the "prescribed position", the opening direction of the front door 31 is equivalent to the "prescribed direction" corresponding to the opening direction of the front door 31, and the left outer gesture is equivalent to the "prescribed gesture".

[0131] Alternatively, the vehicle body 20 has only one of the front opening 21 and the rear opening 22, and the vehicle 10 has only a door for opening and closing the one of the front door 31 and the rear door 32. In this case, the gesture detection device 70 only needs to detect a gesture for opening at least the door for opening and closing the one of the front door 31 and the rear door 32.

[0132] The position of the camera 50 can be changed as appropriate. When a plurality of cameras 50 are used, the camera 50 may be provided at both the front door 31 and the rear door 32 .

[0133] The vehicle 10 may also include a back door and a back camera capable of photographing the rear of the vehicle 10 regardless of whether the back door is opened or closed. In addition, the vehicle body 20 may also include an opening portion that opens at the rear and is opened and closed by the back door. Furthermore, the gesture detection device 70 may also open and close the back door according to the content of the gesture performed by the user. In this case, the opening portion that opens at the rear of the vehicle body 20 corresponds to the "opening portion" and the back door corresponds to the "door".

[0134] In addition, in the above structure, when the right side of the opening is defined as the first opening and the left side of the opening is defined as the second opening, the back door may also have a first door for opening and closing the first opening and a second door for opening and closing the second opening. Furthermore, the gesture detection device 70 may open and close the first door or the second door according to the content of the gesture performed by the user.

[0135] The gesture detection device 70 is not limited to a processing circuit that has a CPU and a memory and performs software processing. For example, the gesture detection device 70 may also have a dedicated hardware circuit that performs at least a part of the various processes performed in the above-mentioned embodiments. As a dedicated hardware circuit, for example, ASIC can be listed. ASIC is the abbreviation of "Application Specific Integrated Circuit". That is, the gesture detection device 70 can be any one of the following structures (a) to (c).

[0136] (a) A processing circuit including all processing devices for executing the above-mentioned processing according to a program and a program storage device such as a ROM for storing the program.

[0137] (b) A processing circuit including a processing device and a program storage device for executing part of the above-mentioned processing according to a program, and a dedicated hardware circuit for executing the remaining processing.

[0138] (c) A processing circuit having a dedicated hardware circuit for executing all of the above-mentioned processing.

[0139] Here, the software execution device and the dedicated hardware circuit including the processing device and the program storage device may be plural. The same applies to the door control device 60.

[0140] [Summary of the present embodiment]

[0141] This embodiment has at least the following structures.

[0142] The gesture detection device 70 of this embodiment is applied to a vehicle 10, which includes: a vehicle body 20, which has openings 21, 22; doors 31, 32, which are configured to open and close the openings 21, 22; a camera 50, which is configured to photograph a user facing the openings 21, 22; and door driving units 41, 42, which are configured to drive the doors 31, 32. The gesture detection device detects a gesture, which is an opportunity to operate the doors 31, 32, using a predetermined body part of the user. The body part of the user different from the predetermined body part is a reference body part. The gesture detection device 70 includes: a position acquisition unit 73, which is configured to acquire a specified position PRh, PLh and a reference position PN, PRs, PLs, wherein the specified position is the position of the specified body part in the image captured by the camera 50, and the reference position is the position of the reference body part in the image; a direction determination unit 74, which is configured to determine whether the user is facing the openings 21, 22 based on the reference positions PN, PRs, PLs in the image; and a gesture determination unit 75, which is configured to determine whether the user has performed a gesture based on changes in the specified positions PRh, PLh and the reference positions PN, PRs, PLs in a plurality of images captured at different times when the direction determination unit 74 determines that the user is facing the openings 21, 22. The gesture determination unit 75 is also configured to determine that the user has at least performed a prescribed gesture for opening the doors 31, 32 when the prescribed positions PRh, PLh move in a prescribed direction corresponding to the opening direction of the doors 31, 32 relative to the reference positions PN, PRs, PLs.

[0143] The gesture detection device determines whether the user has performed a gesture when it can be determined that the user is facing the opening. Specifically, when the user moves a predetermined body part in a predetermined direction, the gesture detection device determines that the user has performed a predetermined gesture. In this way, the user can at least open the door by moving the predetermined body part in a predetermined direction corresponding to the door opening direction.

[0144] When a user performs a gesture while approaching a vehicle, the predetermined position in the image changes due to both the user performing the gesture and the user approaching the vehicle. In this regard, the gesture detection device determines whether the user has performed a gesture based on the change in the predetermined position relative to the reference position. Therefore, whether the user performs a gesture while approaching a vehicle or the user performs a gesture while standing still, the gesture detection device can accurately determine whether the user has performed a gesture. In this way, the gesture detection device can improve user convenience.

[0145] In this embodiment, the openings 21 and 22 may be first openings 22, the doors 31 and 32 may be first doors 32, the predetermined body part may be a first body part, the predetermined positions PRh and PLh may be first positions PRh, the predetermined direction may be a first direction, the predetermined gesture may be a first gesture, and the body part of the user different from the first body part and the reference body part may be a second body part. The vehicle body 20 may further include a second opening 21 adjacent to the first opening 22. The vehicle 10 may further include a second door 31 driven by the door driving units 41 and 42 and configured to open and close the second opening 21. The camera 50 may further include a camera that captures the user facing the second opening 21. The position acquisition unit 73 may further include a camera that captures a second position PLh that is a position of the second body part in the image captured by the camera 50. The direction determination unit 74 may further include a camera that determines whether the user faces the second opening 21 based on the reference positions PN, PRs, and PLs in the image. The gesture determination unit 75 is preferably further configured to determine whether the user has performed a gesture based on changes in the second position PLh and the reference positions PN, PRs, PLs in a plurality of images taken at different times, when the direction determination unit 74 determines that the user is facing the first opening 22 and the second opening 21; and when the second position PLh moves in a second direction corresponding to the opening direction of the second door 31 relative to the reference positions PN, PRs, PLs, it is determined that the user has at least performed a second gesture for opening the second door 31.

[0146] The gesture detection device determines whether the user has performed a gesture when it can be determined that the user is facing the first opening and the second opening. Specifically, when the user moves the second body part in the second direction, the gesture detection device determines that the user has performed the second gesture. In this way, the user can at least open the second door by moving the second body part in the second direction corresponding to the opening direction of the second door.

[0147] When the user performs a gesture while approaching the vehicle, the first position and the second position in the image change due to both the user performing the gesture and the user approaching the vehicle. In this regard, the gesture detection device determines whether the user has performed a gesture based on the change of the first position and the second position relative to the reference position. Therefore, whether the user performs a gesture while approaching the vehicle or the user performs a gesture while standing still, the gesture detection device can accurately determine whether the user has performed a gesture. In this way, the gesture detection device can improve the convenience of the user.

[0148] In this embodiment, preferably, when the user is facing the first opening 22 and the second opening 21, the first opening 22 faces the right half of the user's body, the second opening 21 faces the left half of the user's body, the first body part is the user's right hand, and the second body part is the user's left hand.

[0149] When the user is facing the first opening and the second opening, the user can open the door located in the right front by using a gesture of the right hand. In addition, the user can open the door located in the left front by using a gesture of the left hand. Therefore, the gesture detection device can further improve the convenience of the user.

[0150] In the present embodiment, preferably, the gesture determination unit 75 is further configured to determine that the user has performed the first gesture when the first position PRh moves in the first direction relative to the reference position PN, PRs, PLs and then moves in the opposite direction to the first direction, and to determine that the user has performed the second gesture when the second position PLh moves in the second direction relative to the reference position PN, PRs, PLs and then moves in the opposite direction to the second direction.

[0151] The gesture detection device can determine whether the first gesture is performed with better accuracy than, for example, a case where the first position is determined to be performed by the user only by moving in the first direction. Similarly, the gesture detection device can determine whether the second gesture is performed with better accuracy than, for example, a case where the second position is determined to be performed by the user only by moving in the second direction. Therefore, it becomes difficult for the gesture detection device to misjudge the first gesture and the second gesture.

[0152] In the present embodiment, preferably, the gesture determination unit is further configured to determine that the user has performed the first gesture when the first position PRh moves in the first direction relative to the reference position PN, PRs, PLs and then stops, and to determine that the user has performed the second gesture when the second position PLh moves in the second direction relative to the reference position PN, PRs, PLs and then stops.

[0153] The gesture detection device can determine whether the first gesture is performed with better accuracy than, for example, a case where the first position is determined to be performed by the user only by moving in the first direction. Similarly, the gesture detection device can determine whether the second gesture is performed with better accuracy than, for example, a case where the second position is determined to be performed by the user only by moving in the second direction. Therefore, it becomes difficult for the gesture detection device to misjudge the first gesture and the second gesture.

[0154] In this embodiment, preferably, the first gesture includes a third gesture and a fourth gesture, the third gesture is used to open only the first door 32, and the fourth gesture is a gesture different from the third gesture and is used to open the first door 32 and the second door 31. Preferably, the second gesture includes a fifth gesture and a sixth gesture, the fifth gesture is used to open only the second door 31, and the sixth gesture is a gesture different from the fifth gesture and is used to open the first door 32 and the second door 31. Preferably, the gesture determination unit 75 is further configured to determine that the user has performed the third gesture when the first position PRh moves in the opposite direction to the first direction after moving in the first direction relative to the reference position PN, PRs, PLs; to determine that the user has performed the fourth gesture when the first position PRh moves in the first direction relative to the reference position PN, PRs, PLs and then stops; to determine that the user has performed the fifth gesture when the second position PLh moves in the opposite direction to the second direction relative to the reference position PN, PRs, PLs and then moves; and to determine that the user has performed the sixth gesture when the second position PLh moves in the second direction relative to the reference position PN, PRs, PLs and then stops.

[0155] The user can select whether to open only the first door or both the first door and the second door by using the third gesture and the fourth gesture using the first body part separately. In addition, the user can select whether to open only the second door or both the first door and the second door by using the fifth gesture and the sixth gesture using the second body part separately. Therefore, the gesture detection device can further improve the convenience of the user.

[0156] In this embodiment, preferably, the reference body part includes the right shoulder and the left shoulder of the user, and the reference positions PN, PRs, and PLs include the position PRs of the right shoulder and the position PLs of the left shoulder of the user. Preferably, the direction determination unit 74 is further configured to determine whether the user is facing the first opening 22 and the second opening 21 based on a shoulder width Ls, which is the distance between the position PRs of the right shoulder and the position PLs of the left shoulder of the user in the image.

[0157] When the user is facing the first opening and the second opening, the shoulder width is likely to be longer. On the other hand, when the user is not facing the first opening and the second opening, the shoulder width is likely to be shorter or incorrectly calculated. In this regard, the gesture detection device determines whether the user is facing the first opening and the second opening based on the shoulder width. Therefore, the gesture detection device can determine whether the user is facing the first opening and the second opening with good accuracy.

[0158] In this embodiment, preferably, the first opening 22 is opened at the side of the vehicle body 20, and the second opening 21 is opened at a position on the side of the vehicle body 20 that is forward of the first opening 22. Preferably, the first door 32 is a rear door 32, the second door 31 is a front door 31, the first direction is the same direction as the opening direction of the first door 32, and the second direction is the same direction as the opening direction of the second door 31.

[0159] In the gesture detection device, the first direction is the same direction as the opening direction of the first door, and the second direction is the same direction as the opening direction of the second door. Therefore, the user can easily and intuitively understand the moving direction of the first body part when performing the first gesture and the moving direction of the second body part when performing the second gesture.

[0160] In the present embodiment, the camera 50 is preferably disposed between the first opening 22 and the second opening 21 on the vehicle body 20 in the direction in which the first opening 22 and the second opening 21 are arranged. Preferably, the position acquisition unit 73 is further configured to acquire the first position PRh, the second position PLh, and the reference positions PN, PRs, PLs from the image captured by the camera 50.

[0161] Since the camera is disposed between the first opening and the second opening, when the camera photographs the user facing the first opening and the second opening, it is easy to evenly reflect the right and left halves of the user. Therefore, when the gesture detection device selects the first body part from one of the right and left halves of the user and selects the second body part from the other of the right and left halves of the user, it becomes easy to acquire the first position and the second position.

[0162] In the present embodiment, the first gesture may be a first opening gesture, and the second gesture may be a second opening gesture. Preferably, the gesture determination unit 75 is further configured to determine that the user has at least performed a first closing gesture for closing the first door 32 when the first position PRh moves in the opposite direction of the first direction relative to the reference positions PN, PRs, PLs, and to determine that the user has at least performed a second closing gesture for closing the second door 31 when the second position PLh moves in the opposite direction of the second direction relative to the reference positions PN, PRs, PLs.

[0163] The user can close at least the first door by moving the first body part in the opposite direction of the first direction. In addition, the user can close at least the second door by moving the second body part in the opposite direction of the second direction. In this way, the user can selectively open and close the first door and the second door by using gestures separately.

[0164] The gesture detection method of this embodiment is applied to a vehicle 10, which includes: a vehicle body 20, which has openings 21, 22; doors 31, 32, which are configured to open and close the openings 21, 22; a camera 50, which is configured to photograph a user facing the openings 21, 22; and door driving units 41, 42, which are configured to drive the doors 31, 32. The gesture detection method detects a gesture, which is an opportunity to operate the doors 31, 32, using a specified body part of the user. The body part of the user different from the specified body part is a reference body part. The gesture detection method comprises: an acquisition step of acquiring a predetermined position PRh, PLh and a reference position PN, PRs, PLs, wherein the predetermined position is the position of the predetermined body part in the image captured by the camera 50, and the reference position is the position of the reference body part in the image; a direction determination step of determining whether the user is facing the openings 21, 22 based on the reference positions PN, PRs, PLs in the image; and a gesture determination step of determining whether the user has performed a gesture based on changes in the predetermined positions PRh, PLh and the reference positions PN, PRs, PLs in a plurality of images captured at different times when the direction determination step determines that the user is facing the openings 21, 22. The gesture determination step includes the step of determining that the user has at least performed a predetermined gesture for opening the doors 31, 32 when the predetermined position PRh, PLh has moved relative to the reference positions PN, PRs, PLs in a predetermined direction corresponding to the opening direction of the doors 31, 32.

[0165] The gesture detection method of this embodiment can obtain the same operational effects as those of the above-mentioned gesture detection device.

[0166] In the present embodiment, it is preferred that the openings 21 and 22 are first openings 22, the doors 31 and 32 are first doors 32, the predetermined body part is a first body part, the predetermined positions PRh and PLh are first positions PRh, the predetermined direction is a first direction, the predetermined gesture is a first gesture, and the body part of the user different from the first body part and the reference body part is a second body part. The vehicle body 20 preferably further includes a second opening 21 adjacent to the first opening 22. The vehicle 10 preferably further includes a second door 31, which is driven by the door driving units 41 and 42 and is configured to open and close the second opening 21. The camera 50 is preferably further configured to capture the user facing the second opening 21. The position acquisition step preferably includes the step of acquiring a second position PLh, which is the position of the second body part in the image captured by the camera 50. The direction determination step preferably includes the step of determining whether the user faces the second opening 21 based on the reference positions PN, PRs, and PLs in the image. The gesture determination process preferably includes the following process: when the direction determination unit 74 determines that the user is facing the first opening 22 and the second opening 21, whether the user has performed a gesture is determined based on changes in the second position PLh and the reference positions PN, PRs, PLs in a plurality of images taken at different times; and when the second position PLh moves in a second direction corresponding to the opening direction of the second door 31 relative to the reference positions PN, PRs, PLs, it is determined that the user has at least performed a second gesture for opening the second door 31.

[0167] According to the above configuration, it is possible to obtain the same operational effects as those of the corresponding gesture detection device.

[0168] Explanation of symbols

[0169] 10…vehicle, 20…body, 21…front opening (second opening), 22…rear opening (first opening), 23…center pillar, 31…front door (second door), 32…rear door (first door), 41…front door drive unit, 42…rear door drive unit, 50…camera, 60…door control device, 70…gesture detection device, 71…storage unit, 72…image acquisition unit, 73…position acquisition unit, 74…direction determination unit, 75…gesture determination unit, 76…output unit, PN…position of the neck, PRh…position of the right hand, PLh…position of the left hand, PRs…position of the right shoulder, PLs…position of the left shoulder, PRt…position of the right toe, PLt…position of the left toe.

Claims

1. A gesture detection device, applied to a vehicle, the vehicle comprising: a vehicle body having an opening; a door configured to open and close the opening; a camera configured to photograph a user facing the opening; and a door driving unit configured to drive the door, wherein the gesture detection device detects a gesture using a predetermined body part of the user, the gesture being a trigger for operating the door, in, a body part of the user different from the prescribed body part is a reference body part, The gesture detection device comprises: a position acquisition unit configured to acquire a specified position and a reference position, wherein the specified position is the position of the specified body part in the image captured by the camera, and the reference position is the position of the reference body part in the image; a direction determination unit configured to determine whether the user is facing the opening based on the reference position in the image; as well as a gesture determination unit configured to determine whether the user has performed a gesture based on changes in the predetermined position and the reference position in the plurality of images captured at different times, when the direction determination unit determines that the user is facing the opening; The gesture determination unit is further configured to determine that the user has performed at least a predetermined gesture for opening the door when the predetermined position has moved in a predetermined direction corresponding to the opening direction of the door relative to the reference position.

2. The gesture detection device according to claim 1, in, the opening is a first opening, the door is a first door, the prescribed body part is a first body part, the prescribed position is a first position, the prescribed direction is a first direction, the prescribed gesture is a first gesture, and the body part of the user different from the first body part and the reference body part is a second body part, The vehicle body further has a second opening adjacent to the first opening. The vehicle further includes a second door that is driven by the door drive unit and configured to open and close the second opening. The camera is further configured to capture a picture of the user facing the second opening. The position acquisition unit is further configured to acquire a second position, which is a position of the second body part in the image captured by the camera. The direction determination unit is further configured to determine whether the user is facing the second opening based on the reference position in the image. The gesture determination unit is further configured as follows: When the direction determination unit determines that the user is facing the first opening and the second opening, determining whether the user has performed a gesture based on changes in the second position and the reference position in the plurality of images captured at different times, When the second position moves in a second direction corresponding to the opening direction of the second door relative to the reference position, it is determined that the user has performed at least a second gesture for opening the second door.

3. The gesture detection device according to claim 2, in, When the user is facing the first opening and the second opening, The first opening is oriented toward the right half of the user's body, and the second opening is oriented toward the left half of the user's body. The first body part is the user's right hand, and the second body part is the user's left hand.

4. The gesture detection device according to claim 2 or 3, in, The gesture determination unit is further configured as follows: When the first position moves in the first direction relative to the reference position and then moves in the opposite direction to the first direction, it is determined that the user has performed the first gesture, When the second position moves in the second direction relative to the reference position and then moves in the direction opposite to the second direction, it is determined that the user has performed the second gesture.

5. The gesture detection device according to claim 2 or 3, in, The gesture determination unit is further configured as follows: When the first position moves in the first direction relative to the reference position and then stops, it is determined that the user has performed the first gesture, If the second position moves in the second direction relative to the reference position and then stops, it is determined that the user has performed the second gesture.

6. The gesture detection device according to claim 2 or 3, in, The first gesture includes a third gesture and a fourth gesture, the third gesture is used to open only the first door, and the fourth gesture is a gesture different from the third gesture and is used to open the first door and the second door. The second gesture includes a fifth gesture and a sixth gesture, the fifth gesture is used to open only the second door, the sixth gesture is a gesture different from the fifth gesture, and is used to open the first door and the second door, The gesture determination unit is further configured as follows: When the first position moves in the first direction relative to the reference position and then moves in the opposite direction to the first direction, it is determined that the user has performed the third gesture, When the first position moves in the first direction relative to the reference position and then stops, it is determined that the user has performed the fourth gesture, When the second position moves in the second direction relative to the reference position and then moves in the opposite direction to the second direction, it is determined that the user has performed the fifth gesture, When the second position moves in the second direction relative to the reference position and then stops, it is determined that the user has performed the sixth gesture.

7. The gesture detection device according to claim 2 or 3, in, The reference body part includes the right shoulder and the left shoulder of the user, The reference position includes the position of the right shoulder and the position of the left shoulder of the user, The direction determination unit is further configured to determine whether the user is facing the first opening and the second opening based on a shoulder width that is a distance between a right shoulder and a left shoulder of the user in the image.

8. The gesture detection device according to claim 2 or 3, in, The first opening portion opens at a side of the vehicle body, The second opening is opened at a position on the side surface of the vehicle body that is forward of the first opening. The first door is the back door, The second door is the front door, The first direction is the same as the opening direction of the first door, The second direction is the same direction as the opening direction of the second door.

9. The gesture detection device according to claim 2 or 3, in, The camera is disposed between the first opening and the second opening on the vehicle body in a direction in which the first opening and the second opening are arranged. The position acquisition unit is further configured to acquire the first position, the second position, and the reference position from the image captured by the camera.

10. The gesture detection device according to claim 2 or 3, in, The first gesture is a first opening gesture, the second gesture is a second opening gesture, The gesture determination unit is further configured as follows: When the first position moves in the direction opposite to the first direction relative to the reference position, it is determined that the user has at least performed a first closing gesture for closing the first door. When the second position moves in the direction opposite to the second direction relative to the reference position, it is determined that the user has at least performed a second closing gesture for closing the second door.

11. A gesture detection method, applied to a vehicle, the vehicle comprising: a vehicle body having an opening; a door configured to open and close the opening; a camera configured to photograph a user facing the opening; and a door drive configured to drive the door, wherein the gesture detection method detects a gesture using a predetermined body part of the user, the gesture being a trigger for operating the door, in, a body part of the user different from the prescribed body part is a reference body part, The gesture detection method comprises: a position acquisition step of acquiring a specified position and a reference position, wherein the specified position is the position of the specified body part in the image captured by the camera, and the reference position is the position of the reference body part in the image; a direction determination step of determining whether the user is facing the opening based on the reference position in the image; as well as a gesture determination step of determining whether the user has performed a gesture based on changes in the predetermined position and the reference position in the plurality of images captured at different times, when the direction determination step determines that the user is facing the opening; The gesture determination step includes determining that the user has at least performed a predetermined gesture for opening the door when the predetermined position has moved in a predetermined direction corresponding to the opening direction of the door relative to the reference position.

12. The gesture detection method according to claim 11, in, the opening is a first opening, the door is a first door, the prescribed body part is a first body part, the prescribed position is a first position, the prescribed direction is a first direction, the prescribed gesture is a first gesture, and the body part of the user different from the first body part and the reference body part is a second body part, The vehicle body further has a second opening adjacent to the first opening. The vehicle further includes a second door that is driven by the door drive unit and configured to open and close the second opening. The camera is further configured to capture a picture of the user facing the second opening. The position acquisition step includes the following steps: acquiring a second position, which is the position of the second body part in the image captured by the camera; The direction determination step includes the steps of: determining whether the user is facing the second opening based on the reference position in the image; The gesture determination process includes the following steps: When the direction determination unit determines that the user is facing the first opening and the second opening, determining whether the user has performed a gesture based on changes in the second position and the reference position in the plurality of images captured at different times, When the second position moves in a second direction corresponding to the opening direction of the second door relative to the reference position, it is determined that the user has performed at least a second gesture for opening the second door.

Images