Gesture detection device and gesture detection method

Abstract

A gesture detection device (70) includes a position acquisition unit (73) configured to acquire a predetermined position, which is a position of a body part of a user, in an image captured by a camera (50) and a reference position, a direction determination unit (74) configured to determine whether the user is facing an opening portion of a vehicle (10), and a gesture determination unit (75) configured to determine whether the user has performed a gesture. The gesture determination unit (75) determines that the user has performed a predetermined gesture for opening at least one door when the predetermined position is moved relative to the reference position in a predetermined direction corresponding to an opening direction of the door.

Description

TECHNICAL FIELD

[0001] The present invention relates to a gesture detection device and a gesture detection method. BACKGROUND OF THE INVENTION

[0002] JP 2019-105046 A (reference 1) discloses a door control system including a drive device that drives a rear door, a distance sensor that measures a distance to a hand of a user located near the rear door, and a control device that controls the drive device.

[0003] The door control system opens and closes the rear door when the user moves a hand approaching the rear door in an opening or closing direction. At this time, the door control system performs follow-up control to make the rear door's movement speed follow the user's hand's movement speed. PATENT LITERATURE

[0004] Patent Literature 1: JP 2019-105046 A SUMMARY OF THE INVENTION TECHNICAL PROBLEM

[0005] With the door control system described above, a user who wants to quickly enter a vehicle can quickly open the rear door by quickly moving a hand toward a door handle. However, the user cannot open the rear door until the user approaches the vehicle. The door control system described above still has room for improvement in terms of user convenience. SOLUTION TO THE PROBLEM

[0006] A gesture detection device according to an embodiment disclosed herein is applied to a vehicle including a vehicle body having an opening portion, a door configured to open and close the opening portion, a camera configured to image a user facing the opening portion, and a door control unit configured to control the door. The gesture detection device detects a gesture that triggers an operation of the door and is performed by the user using a predetermined body part. A body part of the user that is different from the predetermined body part is a reference body part.The gesture detection device includes a position acquiring unit configured to acquire a predetermined position, which is a position of the predetermined body part, and a reference position, which is a position of the reference body part, in an image captured by the camera, a direction determining unit configured to determine whether the user is facing the opening portion based on the reference position in the image, and a gesture determining unit configured to determine whether the user has performed a gesture based on changes in the predetermined position and the reference position in a plurality of images captured at different imaging times when the direction determining unit determines that the user is facing the opening portion.The gesture determination unit determines that the user has performed a predetermined gesture for opening at least the door when the predetermined position is moved relative to the reference position in a predetermined direction corresponding to an opening direction of the door.

[0007] A gesture detection method according to another embodiment disclosed herein is applied to a vehicle including a vehicle body having an opening portion, a door configured to open and close the opening portion, a camera configured to image a user facing the opening portion, and a door control unit configured to control the door. The gesture detection method detects a gesture that triggers an operation of the door and is performed by the user using a predetermined body part. A body part of the user that is different from the predetermined body part is a reference body part.The gesture detection method includes a position detection step of acquiring a predetermined position, which is a position of the predetermined body part, and a reference position, which is a position of the reference body part, in an image captured by the camera, a direction determination step of determining whether the user is facing the opening portion based on the reference position in the image, and a gesture determination step of determining whether the user has performed a gesture based on changes in the predetermined position and the reference position in a plurality of images captured at different imaging times when it is determined in the direction determination step that the user is facing the opening portion.The gesture determining step includes determining that the user has performed a predetermined gesture for opening at least the door when the predetermined position is moved relative to the reference position in a predetermined direction corresponding to an opening direction of the door. BRIEF DESCRIPTION OF THE DRAWING Fig. 1 shows a schematic diagram showing a vehicle according to an embodiment. Fig. 2 shows an example of an image taken by a camera installed in the Fig. 1 shown vehicle is provided. Fig. Figure 3 shows an example of an image of a user making a gesture with his right hand, taken by the camera used in the Fig. 1 shown vehicle is provided. Fig. Figure 4 shows an example of an image of a user making a gesture with his left hand, taken by the camera used in the Fig. 1 shown vehicle is provided. Fig. Fig. 5 is a flowchart showing a flow of right-out gesture determination processing executed by a gesture determination unit included in the Fig. 1 shown vehicle is provided. Fig. Fig. 6 is a flowchart showing a flow of right-inward gesture determination processing executed by the gesture determination unit shown in Fig. 1 shown vehicle is provided. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0008] Hereinafter, an embodiment of a vehicle to which a gesture detection device and a gesture detection method are applied will be described with reference to the drawings. <Konfiguration des vorliegenden Ausführungsbeispiels>

[0009] As in Fig. 1, a vehicle 10 includes a vehicle body 20, a front door 31, a rear door 32, a front door drive unit 41, a rear door drive unit 42, a camera 50, a door control device 60, and a gesture detection device 70. In the following description, a width direction of the vehicle 10, a front-rear direction of the vehicle 10, and an up-down direction of the vehicle 10 are simply referred to as a width direction, a front-rear direction, and an up-down direction, respectively. In the width direction, a direction away from the center of the vehicle 10 is also referred to as outward, and a direction toward the center of the vehicle 10 is also referred to as inward. <Fahrzeugkarosserie 20>

[0010] The vehicle body 20 includes a front opening portion 21 and a rear opening portion 22 open on one side surface. The vehicle body 20 includes a center pillar 23 separating the front opening portion 21 and the rear opening portion 22. The front opening portion 21 and the rear opening portion 22 are adjacent to each other in the front-rear direction, with the center pillar 23 interposed therebetween. The front opening portion 21 is positioned in front of the rear opening portion 22. The front opening portion 21 and the rear opening portion 22 are portions through which a user passes when getting in and out of the vehicle 10. In the present embodiment, the front opening portion 21 corresponds to a "second opening portion," and the rear opening portion 22 corresponds to a "first opening portion." <Vordere Tür 31 und Vordertüransteuereinheit 41>

[0011] The front door 31 has a shape corresponding to the front opening portion 21 when viewed in the width direction. The front door 31 slides in the front-rear direction relative to the vehicle body 20 so that it can be slid between a fully closed position in which the front opening portion 21 is fully closed and a fully opened position in which the front opening portion 21 is fully opened. An opening direction of the front door 31 is forward, and a closing direction of the front door 31 is rearward. That is, the opening direction of the front door 31 is a direction opposite to the closing direction of the front door 31. The front door drive unit 41 drives the front door 31. The front door drive unit 41 opens the front door 31 toward the fully open position or closes the front door 31 toward the fully closed position.In the present embodiment, the front door 31 corresponds to a “second door” and the front door control unit 41 corresponds to a “door control unit”. <Hintere Tür 32 und Hintertüransteuereinheit 42>

[0012] The rear door 32 has a shape that corresponds to the rear opening portion 22 when viewed in the width direction. The rear door 32 slides in the front-rear direction relative to the vehicle body 20 so that it can be slid between a fully closed position in which the rear opening portion 22 is fully closed and a fully opened position in which the rear opening portion 22 is fully opened. An opening direction of the rear door 32 is rearward, and a closing direction of the rear door 32 is forward. That is, the opening direction of the rear door 32 is a direction opposite to the closing direction of the rear door 32. The rear door drive unit 42 drives the rear door 32. The rear door drive unit 42 opens the rear door 32 toward the fully open position or closes the rear door 32 toward the fully closed position.In the present embodiment, the rear door 32 corresponds to a “first door” and the rear door control unit 42 corresponds to a “door control unit”. <Kamera 50>

[0013] The camera 50 is installed above the center pillar 23 of the vehicle body 20 facing outward in the width direction. That is, the camera 50 is arranged between the front opening portion 21 and the rear opening portion 22 in the front-rear direction. The camera 50 can image an area facing the front opening portion 21 and the rear opening portion 22, regardless of the opening and closing states of the front door 31 and the rear door 32. The camera 50 outputs a captured image to the gesture detection device 70 for each frame. A frame rate of the camera 50 may be, for example, approximately 30 fps. Fig. 2 shows an example of an image of a user facing the front opening portion 21 and the rear opening portion 22, which was captured by the camera 50. In the following description, facing the front opening portion 21 and the rear opening portion 22, in other words, facing the center pillar 23, is referred to as "facing a side surface of the vehicle 10." When a user faces a left side surface of the vehicle 10, a right half of the user's body faces the rear opening portion 22, and a left half of the user's body faces the front opening portion 21. <Türsteuervorrichtung 60>

[0014] The door control device 60 includes, for example, a processing circuit having a CPU and a memory. The memory includes a RAM, a ROM, and the like, and stores a program or an instruction configured to cause the CPU to execute processing. The memory, that is, a non-transitory computer-readable medium, includes any usable medium accessible by a general-purpose or dedicated computer. The door control device 60 controls the front door drive unit 41 and the rear door drive unit 42 according to a program stored in the memory and an input command signal. Specifically, the door control device 60 opens the front door 31 when an open command signal for opening the front door 31 is received. On the other hand, the door control device 60 closes the front door 31 when a close command signal for closing the front door 31 is received.The door control device 60 opens the rear door 32 when an open command signal for opening the rear door 32 is received. On the other hand, the door control device 60 closes the rear door 32 when a close command signal for closing the rear door 32 is received. <Gestenerfassungsvorrichtung 70>

[0015] The gesture detection device 70 includes, for example, a processing circuit with a CPU and a memory. According to a program stored in the memory, the gesture detection device 70 determines whether a user has performed a gesture with a body part and outputs a command signal corresponding to the gesture to the door control device 60. That is, the gesture triggers an operation of the front door 31 and the rear door 32.

[0016] First, a gesture detected by the gesture detection device 70 is described.

[0017] As in Fig. As shown in FIG. 2, a gesture detected by the gesture detection device 70 is a movement of an arm in a state where a user faces a side surface of the vehicle 10. In a state where the user faces the side surface of the vehicle 10, a face and an upper body of the user face the side surface of the vehicle 10. The gesture includes a movement of a right hand of the user in the opening and closing direction of the rear door 32 and a movement of a left hand of the user in the opening and closing direction of the front door 31.

[0018] As in Fig. 3, right-hand gestures include a right outward gesture, which involves an action in which the right hand is moved away from the body (moved outward) from a state in which a user lowers the right hand, and a right inward gesture, which involves an action in which the right hand is moved toward the body (moved inward) from a state in which the user lowers the right hand. Fig. 3, the right outward gesture involves an action in which the user moves the right hand from a position indicated by a solid line to a position indicated by a dotted line. Fig. 3, the right inward gesture involves an action in which the user moves the right hand from the position indicated by the solid line to a position indicated by a dashed line with two dots.

[0019] The right outward gesture also includes a right outward reverse gesture, in which the right hand is returned to the original position after being moved outward from the body, and a right outward hold gesture, in which the right hand is stopped after being moved outward from the body. When a user performs the right outward reverse gesture, the user moves the right hand from the position indicated by the solid line to the position indicated by the dotted line, and then moves the right hand to the position indicated by the solid line. When a user performs the right outward hold gesture, the user moves the right hand from the position indicated by the solid line to the position indicated by the dotted line, and then holds the right hand in the position indicated by the dotted line.

[0020] In the present embodiment, the right outward gesture corresponds to a “first gesture (first opening gesture)”, the right outward reverse gesture corresponds to a “third gesture”, and the right outward hold gesture corresponds to a “fourth gesture”.

[0021] The right inward gesture further includes a right inward reversal gesture, in which the right hand is returned to the original position after the right hand has been moved inward toward the body, and a right inward hold gesture, in which the right hand is paused after the right hand has been moved inward toward the body. When a user performs the right inward reversal gesture, the user moves the right hand from the position indicated by the solid line to the position indicated by the two-dot chain line, and then moves the right hand to the position indicated by the solid line.When a user performs the right inward hold gesture, the user moves the right hand from the position indicated by the solid line to the position indicated by the two-dot chain line, and then holds the right hand in the position indicated by the two-dot chain line. In the present embodiment, the right inward gesture corresponds to a "first close gesture."

[0022] As in Fig. 4, left-hand gestures include a left outward gesture, which involves an action in which the left hand is moved away from the body (moved outward) from a state in which a user lowers the left hand, and a left inward gesture, which involves an action in which the left hand is moved toward the body (moved inward) from a state in which the user lowers the left hand. Fig. 4, the left outward gesture involves an action in which the user moves the left hand from a position indicated by a solid line to a position indicated by a dotted line. Fig. 4, the left inward gesture involves an action in which the user moves the left hand from the position indicated by the solid line to a position indicated by a dashed line with two dots.

[0023] The left outward gesture also includes a left outward reverse gesture, in which the left hand is returned to the original position after the left hand has been moved outward from the body, and a left outward hold gesture, in which the left hand is stopped after the left hand has been moved outward from the body. When a user performs the left outward reverse gesture, the user moves the left hand from the position indicated by the solid line to the position indicated by the dotted line, and then moves the left hand to the position indicated by the solid line. When a user performs the left outward hold gesture, the user moves the left hand from the position indicated by the solid line to the position indicated by the dotted line, and then holds the left hand in the position indicated by the dotted line.In the present embodiment, the left outward gesture corresponds to a “second gesture (second opening gesture)”, the left outward reverse gesture corresponds to a “fifth gesture”, and the left outward hold gesture corresponds to a “sixth gesture”.

[0024] The left inward gesture also includes a left inward reversal gesture, in which the left hand is returned to its original position after moving the left hand inward toward the body, and a left inward hold gesture, in which the left hand is stopped after moving it inward toward the body. When a user performs the left inward reversal gesture, the user moves the left hand from the position indicated by the solid line to the position indicated by the two-dot chain line, and then moves the left hand to the position indicated by the solid line.When a user performs the left inward hold gesture, the user moves the left hand from the position indicated by the solid line to the position indicated by the two-dot chain line, and then holds the left hand in the position indicated by the two-dot chain line. In the present embodiment, the left inward gesture corresponds to a "second close gesture."

[0025] As in Fig. 1, when a user faces a left side surface of the vehicle 10, a right hand of the user faces the rear opening portion 22, and a left hand of the user faces the front opening portion 21. In this case, the user's right hand corresponds to the rear door 32, and the user's left hand corresponds to the front door 31. Accordingly, on the left side of the vehicle 10, a gesture with the user's right hand is a gesture for opening and closing at least the rear door 32, and a gesture with the user's left hand is a gesture for opening and closing at least the front door 31. Specifically, the right outward reverse gesture is a gesture for opening only the rear door 32 corresponding to the right hand, and the right outward hold gesture is a gesture for opening both the front door 31 and the rear door 32.The left outward reverse gesture is a gesture for opening only the front door 31, which corresponds to the left hand, and the left outward hold gesture is a gesture for opening both the front door 31 and the rear door 32. On the other hand, the right inward reverse gesture is a gesture for closing only the rear door 32, which corresponds to the right hand, and the right inward hold gesture is a gesture for closing both the front door 31 and the rear door 32. The left inward reverse gesture is a gesture for closing only the front door 31, which corresponds to the left hand, and the left inward hold gesture is a gesture for closing both the front door 31 and the rear door 32.

[0026] On the other hand, when a user faces a right side surface of the vehicle 10, a right hand of the user faces the front opening portion 21 and a left hand of the user faces the rear opening portion 22. In this case, the user's right hand corresponds to the front door 31 and the user's left hand corresponds to the rear door 32. Accordingly, on the right side of the vehicle 10, a gesture with the user's right hand is a gesture for opening and closing at least the front door 31 and a gesture with the user's left hand is a gesture for opening and closing at least the rear door 32.

[0027] In the following description, it is assumed that a user performs a gesture in a state where the user is facing the left side surface of the vehicle 10 to facilitate understanding of the description. That is, the gesture detection device 70 detects a gesture for opening and closing the front door 31 and the rear door 32 located on the left side of the vehicle 10.

[0028] Next, the gesture detection device 70 for each function is described.

[0029] As in Fig. 1, the gesture detection device 70 includes a storage unit 71, an image acquisition unit 72, a position acquisition unit 73, a direction determination unit 74, a gesture determination unit 75, and an output unit 76. <Speichereinheit 71>

[0030] The storage unit 71 stores a trained model obtained by machine learning using training data in which a pre-captured image and a position of a user's body part are linked. That is, the trained model is a model that receives an image captured by the camera 50 as input and outputs a position of a user's body part in the image. Specifically, the model outputs a neck position PN, a right shoulder position PRs, a left shoulder position PLs, a right hand position PRh, a left hand position PLh, a right toe position PRt, and a left toe position PLt, as shown in Fig. 2. In the present embodiment, the right hand corresponds to a "first body part," the left hand corresponds to a "second body part," and the neck, right shoulder, and left shoulder correspond to a "reference body part." For example, the trained model is created at the time of designing the vehicle 10 and written into the storage unit 71 at the time of manufacturing the gesture sensing device 70.

[0031] A procedure for generating the trained model is described below.

[0032] A method for generating the trained model includes a preparation step for preparing training data and a learning step for performing machine learning based on the training data. The preparation step includes an acquisition step for acquiring an image captured in a state where a user is standing in an image area under various conditions, and a labeling step for labeling a position of a body part of the user in a plurality of images acquired in the acquisition step.

[0033] The acquisition step is performed, for example, using the actual vehicle 10. In the acquisition step, it is preferable to acquire multiple images captured by changing the conditions related to the user and the conditions related to an environment around the vehicle 10. Accordingly, it is possible to obtain a trained model that can be adapted to various situations—in other words, a trained model with high versatility. The labeling step is a step of labeling a position of a body part of the user in the acquired image. For example, coordinates using pixels in the image can be used to label a position.In the present embodiment, an axis extending in a horizontal direction in the image is defined as an X-axis, and an axis extending in a vertical direction in the image is defined as a Y-axis.

[0034] The learning step is a step for generating a model by performing machine learning using a variety of training data as learning data. Various methods can be selected as machine learning methods, such as a convolutional neural network (CNN). When the trained model receives a captured image, the trained model outputs the position of the user's body part that appears in the image. Conversely, if the trained model receives an image in which no user's body part is captured, the trained model cannot output the position of the user's body part. If the accuracy of the trained model is insufficient, the trained model may output an incorrect position of the body part. <Bildbeschaffungseinheit 72>

[0035] The image acquisition unit 72 acquires an image captured by the camera 50 for each imaging cycle of the camera 50. For example, if the frame rate of the camera 50 is 30 fps, the image acquisition unit 72 acquires an image captured by the camera 50 every 1 / 30 of a second. <Positionsbeschaffungseinheit 73>

[0036] The position acquisition unit 73 acquires a position of a body part of a user that appears in an image when the image captured by the camera 50 is input to the trained model stored in the storage unit 71. That is, the position acquisition unit 73 acquires the user's neck position PN, the right shoulder position PRs, the left shoulder position PLs, the right hand position PRh, the left hand position PLh, the right toe position PRt, and the left toe position PLt. In the present embodiment, the right hand position PRh corresponds to a "first position," the left hand position PLh corresponds to a "second position," and the user's neck position PN, the right shoulder position PRs, and the left shoulder position PLs correspond to a "reference position." <Richtungsbestimmungseinheit 74>

[0037] The direction determination unit 74 determines, based on the position of the right shoulder PRs and the position of the left shoulder PLs, whether the user appearing in the image is facing a side surface of the vehicle 10. In particular, the direction determination unit 74 determines, as shown in Fig. 2, based on a shoulder width Ls which is a distance between the position of the right shoulder PRs and the position of the left shoulder PLs in an X direction, whether the user is facing the side surface of the vehicle 10.

[0038] When the user is located on one side of the vehicle 10 and facing the side surface of the vehicle 10, the camera 50 images the user from the front. Therefore, the shoulder width Ls calculated by the direction determination unit 74 is a relatively large value. On the other hand, when the user is located on one side of the vehicle 10 and facing slightly forward toward the front opening portion 21 or slightly rearward toward the rear opening portion 22, the camera 50 images the user from a right front side or a left front side. Therefore, the shoulder width Ls calculated by the direction determination unit 74 is a relatively small value.

[0039] Therefore, when the calculated shoulder width Ls is equal to or greater than a preset shoulder width determination value Lsth, the direction determination unit 74 determines that the user is facing the side surface of the vehicle 10. On the other hand, when the calculated shoulder width Ls is smaller than the shoulder width determination value Lsth, the direction determination unit 74 determines that the user is not facing the side surface of the vehicle 10. The shoulder width determination value Lsth is a value set in advance through experiments and simulations.

[0040] When a distance from the camera 50 to the user is short, the shoulder width Ls is likely to be calculated as a large value, and when the distance from the camera 50 to the user is long, the shoulder width Ls is likely to be calculated as a small value. Therefore, the shoulder width determination value Lsth may be a variable value corresponding to the distance from the camera 50 to the user. The shoulder width Ls also changes according to a user's height. Therefore, when the direction determination unit 74 can determine a user's height based on the positions of a plurality of body parts, the shoulder width determination value Lsth may be a variable value corresponding to the user's physique.

[0041] When the user is located on one side of the vehicle 10 and facing a front of the vehicle 10 or a rear of the vehicle 10, the camera 50 captures the user from the side. In this case, the position acquisition unit 73 cannot acquire both the user's right shoulder position PRs and left shoulder position PLs. That is, the direction determination unit 74 cannot calculate the shoulder width Ls. When the position acquisition unit 73 cannot acquire at least the right shoulder position PRs or the left shoulder position PLs, in other words, when the direction determination unit 74 cannot calculate the shoulder width Ls, the direction determination unit 74 determines that the user is not facing the side surface of the vehicle 10. <Gestenbestimmungseinheit 75>

[0042] The gesture determination unit 75 performs approach determination processing in which it is determined whether a user is approaching the vehicle 10 and gesture determination processing in which it is determined whether a user has performed a gesture.

[0043] In the approach determination processing, the gesture determination unit 75 determines whether the right toe position PRt and the left toe position PLt of a user acquired by the position acquisition unit 73 exist in a determination area Ax as shown in Fig. 2. The determination area Ax is an area defined in an image. In Fig. 2, the determination area Ax is an area below a line indicated by a chain line in the image. The determination area Ax may be a rectangular area or a semicircular area. A length of the determination area Ax in the X direction and Y direction can be appropriately changed. When at least the right toe position PRt or the left toe position PLt of the user exists in the determination area Ax, the gesture determination unit 75 starts gesture determination processing. The gesture determination unit 75 can start approach determination processing only when the user wearing a portable device connected to the vehicle 10 approaches the vehicle 10. Accordingly, the gesture determination processing is prohibited from being performed for a gesture of a person who is not the user of the vehicle 10.

[0044] In gesture determination processing, the gesture determination unit 75 determines whether a user has performed a gesture based on a position change of a body part in a plurality of consecutively acquired images taken at different imaging times. The gesture determination processing includes right out gesture determination processing, right in gesture determination processing, left out gesture determination processing, and left in gesture determination processing. The right out gesture determination processing determines whether a user has performed the right out reverse gesture or the right out hold gesture, and the right in gesture determination processing determines whether a user has performed the right in reverse gesture or the right in hold gesture.The left outward gesture determination processing determines whether a user has performed the left outward reverse gesture or the left outward hold gesture, and the left inward gesture determination processing determines whether a user has performed the left inward reverse gesture or the left inward hold gesture.

[0045] In the right outward gesture determination processing and the right inward gesture determination processing, the gesture determination unit 75 calculates a reference distance LR from the neck position PN to the right shoulder position PRs in the X direction based on the neck position PN and the right shoulder position PRs acquired by the position acquisition unit 73. As shown in Fig. 3, the gesture determination unit 75 sets a right reference area AR1, a right outer area AR2, and a right inner area AR3 in an image based on the neck position PN, the right shoulder position PRs, and the reference distance LR.

[0046] The right reference area AR1 is an area that includes the right shoulder position PRs and extends left and right from the right shoulder position PRs. The width of the right reference area AR1 is shorter than the reference distance LR. For example, the right reference area AR1 may be an area centered on the right shoulder position PRs, and the width of the right reference area AR1 may be approximately 40% of the reference distance LR. The right outer area AR2 is an area located on the right side of the right reference area AR1 and extends left and right. The width of the right outer area AR2 is greater than the reference distance LR. For example, the width of the right outer area AR2 may be approximately 140% of the reference distance LR. The right inner area AR3 is an area that includes the neck position PN and extends left and right from the neck position PN.The width of the right inner region AR3 is equal to the reference distance LR. For example, the right inner region AR3 may be a region centered on the neck position PN, and the width of the right inner region AR3 may be approximately 100% of the reference distance LR. Between the right reference region AR1 and the right outer region AR2, as well as between the right reference region AR1 and the right inner region AR3, a region that does not belong to either region is preferably provided.

[0047] When a user makes a gesture while approaching the vehicle, the user's reference distance LR viewed from the camera 50 changes over time. That is, the neck position PN and the right shoulder position PRs in an image change relative to an imaging time. Therefore, the reference distance LR is calculated each time a new image is captured, and the ranges are adjusted each time a new image is captured.

[0048] When the setting of the areas is completed, the gesture determination unit 75 determines whether a user has performed a right-hand gesture based on a movement mode of the user's right-hand position PRh in the areas. Details are described below.

[0049] In the right outward gesture determination processing, the gesture determination unit 75 determines that a user has performed the right outward reversal gesture when the user moves the position of the right hand PRh from the right reference area AR1 to the right outer area AR2 and then returns the position of the right hand PRh to the right reference area AR1 over time. In other words, the gesture determination unit 75 determines that the user has performed the right outward reversal gesture when the user moves the position of the right hand PRh in an opening direction of the rear door 32 and then moves the position of the right hand PRh in a closing direction of the rear door 32.The gesture determination unit 75 determines that the user has performed the right outward hold gesture when the user moves the position of the right hand PRh from the right reference area AR1 to the right outer area AR2 and then maintains the position of the right hand PRh in the right outer area AR2 over time. In other words, the gesture determination unit 75 determines that the user has performed the right outward hold gesture when the user moves the position of the right hand PRh in the opening direction of the rear door 32 and then maintains the position of the right hand PRh. Regarding this point, in the present embodiment, the opening direction of the rear door 32 corresponds to a "first direction" that corresponds to the opening direction of the rear door 32.

[0050] In the right inward gesture determination processing, the gesture determination unit 75 determines that a user has performed the right inward reversal gesture when the user moves the position of the right hand PRh from the right reference area AR1 to the right inner area AR3 and then returns the position of the right hand PRh to the right reference area AR1 over time. In other words, the gesture determination unit 75 determines that the user has performed the right inward reversal gesture when the user moves the position of the right hand PRh in the closing direction of the rear door 32 and then moves the position of the right hand PRh in the opening direction of the rear door 32.The gesture determination unit 75 determines that the user has performed the right inward hold gesture when the user moves the position of the right hand PRh from the right reference area AR1 to the right inner area AR3 and then maintains the position of the right hand PRh in the right inner area AR3 over time. In other words, the gesture determination unit 75 determines that the user has performed the right inward hold gesture when the user moves the position of the right hand PRh in the closing direction of the rear door 32 and then maintains the position of the right hand PRh.

[0051] As described above, in the present embodiment, the right reference area AR1, the right outer area AR2, and the right inner area AR3 are set based on the neck position PN and the right shoulder position PRs. Therefore, moving the right hand position PRh to a right adjacent area is equivalent to moving the right hand position PRh in the opening direction of the rear door 32 relative to the neck position PN and the right shoulder position PRs. Furthermore, moving the right hand position PRh to a left adjacent area is equivalent to moving the right hand position PRh in the closing direction of the rear door 32 relative to the neck position PN and the right shoulder position PRs.

[0052] In the left outward gesture determination processing and the left inward gesture determination processing, the gesture determination unit 75 calculates a reference distance LL from the neck position PN to the left shoulder position PLs in the X direction based on the neck position PN and the left shoulder position PLs acquired by the position acquisition unit 73. As shown in Fig. 4, the gesture determination unit 75 sets a left reference area AL1, a left outer area AL2, and a left inner area AL3 in an image based on the neck position PN, the left shoulder position PLs, and the reference distance LL.

[0053] The left reference area AL1 is an area that includes the position of the left shoulder PL and extends left and right from the position of the left shoulder PL. The width of the left reference area AL1 is shorter than the reference distance LL. For example, the left reference area AL1 may be an area centered on the position of the left shoulder PL, and the width of the left reference area AL1 may be about 40% of the reference distance LL. The left outer area AL2 is an area located on the left side of the left reference area AL1 and extends left and right. The width of the left outer area AL2 is greater than the reference distance LL. For example, the width of the left outer area AL2 may be about 140% of the reference distance LL. The left inner area AL3 is an area that includes the neck position PN and extends left and right from the neck position PN.The width of the left inner region AL3 corresponds to the reference distance LL. For example, the left inner region AL3 may be a region centered on the neck position PN, and the width of the left inner region AL3 may be approximately 100% of the reference distance LL. When the reference distance LL and the reference distance LR are the same in an image, the left inner region AL3 overlaps the right inner region AR3. Between the left reference region AL1 and the left outer region AL2, and between the left reference region AL1 and the left inner region AL3, a region that does not belong to either region is preferably provided.

[0054] When a user makes a gesture while approaching the vehicle, the user's reference distance LL viewed from the camera 50 changes over time. That is, the neck position PN and the left shoulder position PLs in an image change relative to an imaging time. Therefore, the reference distance LL is calculated each time a new image is captured, and the ranges are adjusted each time a new image is captured.

[0055] When the setting of the areas is completed, the gesture determination unit 75 determines whether a user has performed a left-hand gesture based on a movement mode of the user's left hand position PLh in the areas. Details are described below.

[0056] In the left outward gesture determination processing, the gesture determination unit 75 determines that a user has performed the left outward reversal gesture when the user moves the left hand position PLh from the left reference area AL1 to the left outer area AL2 and then returns the left hand position PLh to the left reference area AL1 over time. In other words, the gesture determination unit 75 determines that the user has performed the left outward reversal gesture when the user moves the left hand position PLh in an opening direction of the front door 31 and then moves the left hand position PLh in a closing direction of the front door 31.The gesture determination unit 75 determines that the user has performed the left outward hold gesture when the user moves the left hand position PLh from the left reference area AL1 to the left outer area AL2 and then maintains the left hand position PLh in the left outer area AL2 over time. In other words, the gesture determination unit 75 determines that the user has performed the left outward hold gesture when the user moves the left hand position PLh in the opening direction of the front door 31 and then maintains the left hand position PLh. Regarding this point, in the present embodiment, the opening direction of the front door 31 corresponds to a "second direction" that corresponds to the opening direction of the front door 31.

[0057] In the left inward gesture determination processing, the gesture determination unit 75 determines that a user has performed the left inward reversal gesture when the user moves the left hand position PLh from the left reference area AL1 to the left inner area AL3 and then returns the left hand position PLh to the left reference area AL1 over time. In other words, the gesture determination unit 75 determines that the user has performed the left inward reversal gesture when the user moves the left hand position PLh in the closing direction of the front door 31 and then moves the left hand position PLh in the opening direction of the front door 31.The gesture determination unit 75 determines that the user has performed the left inward hold gesture when the user moves the left hand position PLh from the left reference area AL1 to the left inner area AL3 and then maintains the left hand position PLh in the left inner area AL3 over time. In other words, the gesture determination unit 75 determines that the user has performed the left inward hold gesture when the user moves the left hand position PLh in the closing direction of the front door 31 and then maintains the left hand position PLh.

[0058] As described above, in the present embodiment, the left reference range AL1, the left outer range AL2, and the left inner range AL3 are set based on the neck position PN and the left shoulder position PLs. Therefore, moving the left hand position PLh to a range adjacent to the left is equivalent to moving the left hand position PLh in the opening direction of the front door 31 relative to the neck position PN and the left shoulder position PLs. Furthermore, moving the left hand position PLh to a range adjacent to the right is equivalent to moving the left hand position PLh in the closing direction of the front door 31 relative to the neck position PN and the left shoulder position PLs. <Ausgabeeinheit 76>

[0059] If it is determined that the user has made the right outward reversal gesture, the output unit 76 outputs an open command signal to open the rear door 32 corresponding to the right hand. If it is determined that the user has made the right outward hold gesture, the output unit 76 outputs an open command signal to open the front door 31 and the rear door 32. On the other hand, if it is determined that the user has made the right inward reversal gesture, the output unit 76 outputs a close command signal to close the rear door 32 corresponding to the right hand. If it is determined that the user has made the right inward hold gesture, the output unit 76 outputs a close command signal to close the front door 31 and the rear door 32.

[0060] If it is determined that the user has made the left outward reversal gesture, the output unit 76 outputs an open command signal to open the front door 31 corresponding to the left hand. If it is determined that the user has made the left outward hold gesture, the output unit 76 outputs an open command signal to open the front door 31 and the rear door 32. On the other hand, if it is determined that the user has made the left inward reversal gesture, the output unit 76 outputs a close command signal to close the front door 31 corresponding to the left hand. If it is determined that the user has made the left inward hold gesture, the output unit 76 outputs a close command signal to close the front door 31 and the rear door 32.

[0061] That is, if the gesture determination unit 75 determines that the user has performed the right outward hold gesture and the left outward hold gesture, the output unit 76 outputs the same signal. If the gesture determination unit 75 determines that the user has performed the right inward hold gesture and the left inward hold gesture, the output unit 76 outputs the same signal. <Verarbeitung durch die Gestenerfassungsvorrichtung 70>

[0062] The following is a flow of the right out gesture determination processing for determining whether a user has made the right out reverse gesture or the right out hold gesture, with reference to a Fig. 5. The right out gesture determination processing is processing that is executed for each predetermined control cycle when a user is present in an imaging range of the camera 50. In the flowchart shown in Fig. 5, in a step of making a determination using a position of a body part or making a determination using the right reference area AR1 and the right outer area AR2 for a newly acquired image, the position of the body part is acquired again or the areas are reset.

[0063] The right out gesture determination processing is essentially the same as the left out gesture determination processing for determining whether a user has performed the left out reverse gesture or the left out hold gesture, except that the hand used to perform the gesture is different and the door to be opened is different. Therefore, the description of the left out gesture determination processing is omitted.

[0064] As in Fig. 5, the gesture detection device 70 determines whether the right toe position PRt and the left toe position PLt in a newly captured image are present in a determination area Ax (S11). If neither the right toe position PRt nor the left toe position PLt are present in the determination area Ax (S11: NO), the gesture detection device 70 temporarily ends the current processing. On the other hand, if at least the right toe position PRt or the left toe position PLt is present in the determination area Ax (S11: YES), the gesture detection device 70 determines whether the rear door 32 is positioned in the fully open position or is fully opened (S12). If the rear door 32 is positioned in the fully open position or is fully opened (S12: YES), that is,When it is not necessary to open the rear door 32, the gesture detection device 70 temporarily terminates the current processing.

[0065] On the other hand, if the rear door 32 is not positioned in the fully open position or is not fully opened (S12: NO), the gesture detection device 70 determines whether the user is facing the side surface of the vehicle 10 based on the shoulder width Ls, which is a distance between the position of the right shoulder PRs and the position of the left shoulder PLs (S13). If the user is not facing the side surface of the vehicle 10 (S13: NO), the gesture detection device 70 temporarily terminates the current processing. On the other hand, if the user is facing the side surface of the vehicle 10 (S13: YES), the gesture detection device 70 determines whether the position of the right hand PRh is present in the right reference area AR1 (S14).

[0066] If the position of the right hand PRh is not present in the right reference area AR1 (S14), the gesture detection device 70 temporarily terminates the current processing. On the other hand, if the position of the right hand PRh is present in the right reference area AR1 (S14: YES), the gesture detection device 70 determines whether the position of the right hand PRh is moved to the right outer area AR2 within a first determination time T1 (S15). Here, "within the first determination time T1" refers to an elapsed time from a time point at which an affirmative determination is made at step S14 within the first determination time T1. At step S15, the movement determination of the position of the right hand PRh is repeatedly performed for a newly captured image.

[0067] If the position of the right hand PRh is not moved to the right outer area AR2 within the first determination time T1 (S15: NO), the gesture detection device 70 temporarily terminates the current processing. On the other hand, if the position of the right hand PRh is moved to the right outer area AR2 within the first determination time T1 (S15: YES), the gesture detection device 70 determines whether the position of the right hand PRh is moved to the right reference area AR1 within a second determination time T2 (S16). Here, "within the second determination time T2" ​​refers to an elapsed time from a time when the position of the right hand PRh is moved to the right outer area AR2 within the second determination time T2. At step S16, the movement determination of the position of the right hand PRh is repeatedly performed for a newly captured image in a similar manner to step S15.

[0068] When the position of the right hand PRh is moved to the right reference area AR1 within the second determination time T2 (S16: YES), that is, when the user performs the right outward reversal gesture, the gesture detection device 70 outputs an opening command signal for the rear door 32 to the door control device 60 (S17). Thereafter, the gesture detection device 70 ends the current processing. On the other hand, when the position of the right hand PRh is not moved to the right reference area AR1 within the second determination time T2 (S16: NO), the gesture detection device 70 determines whether the position of the right hand PRh exists in the right outer area AR2 for a third determination time T3 (S18). Here, the third determination time T3 refers to an elapsed time from a time when the position of the right hand PRh is moved to the right outer area AR2 as the third determination time T3.Therefore, the third determination time T3 is longer than the second determination time T2. At step S18, the movement determination of the right hand position PRh is repeatedly performed for a newly captured image in a similar manner to step S15.

[0069] When the position of the right hand PRh is not present in the right outer area AR2 for the third determination time T3 (S18: NO), that is, when the position of the right hand PRh is moved out of the right outer area AR2 within the third determination time T3, the gesture detection device 70 temporarily ends the current processing. On the other hand, when the position of the right hand PRh is held in the right outer area AR2 for the third determination time T3 (S18: YES), the gesture detection device 70 outputs an opening command signal for the front door 31 and the rear door 32 (S19). A case where an affirmative determination is made in step S18 is a case where it is determined that the user has performed the right outward holding gesture. Thereafter, the gesture detection device 70 ends the present processing.

[0070] In the Fig. In the flowchart shown in Fig. 5, steps S11, S14 to S16 and S18 include a “position acquiring step”, step S13 corresponds to a “direction determining step”, and steps S14 to S16 and S18 correspond to a “gesture determining step”.

[0071] The following is a flow of the right inward gesture determination processing for determining whether a user has executed the right inward reverse gesture or the right inward hold gesture, with reference to the Fig. 6 is described. The right inward gesture determination processing is processing that is executed for each predetermined control cycle when a user is present in an imaging area of ​​the camera 50. In the flowchart shown in Fig. 6, in a step of making a determination using a position of a body part or making a determination using the right reference area AR1 and the right inner area AR3 for a newly acquired image, the position of the body part is acquired again or the areas are reset.

[0072] The right inward gesture determination processing is essentially the same as the left inward gesture determination processing for determining whether a user has made the left inward reverse gesture or the left inward hold gesture, except that a hand used to perform a gesture is different and a door to be closed is different. Therefore, the description of the left inward gesture determination processing is omitted.

[0073] As in Fig. 6, the gesture detection device 70 determines whether the right toe position PRt and the left toe position PLt in a newly captured image are present in a determination area Ax (S21). If neither the right toe position PRt nor the left toe position PLt are present in the determination area Ax (S21: NO), the gesture detection device 70 temporarily ends the current processing. On the other hand, if at least one of the right toe position PRt and the left toe position PLt is present in the determination area Ax (S21: YES), the gesture detection device 70 determines whether the rear door 32 is positioned in the fully closed position or is fully closed (S22). If the rear door 32 is positioned in the fully closed position (S22: YES), that is,When it is not necessary to close the rear door 32, the gesture detection device 70 temporarily terminates the current processing.

[0074] On the other hand, if the rear door 32 is not in the fully closed position (S22: NO), the gesture detection device 70 determines whether the user is facing the side surface of the vehicle 10 based on the shoulder width Ls, which is a distance between the position of the right shoulder PRs and the position of the left shoulder PLs (S23). If the user is not facing the side surface of the vehicle 10 (S23: NO), the gesture detection device 70 temporarily terminates the current processing. On the other hand, if the user is facing the side surface of the vehicle 10 (S23: YES), the gesture detection device 70 determines whether the position of the right hand PRh is present in the right reference area AR1 (S24).

[0075] If the position of the right hand PRh is not present in the right reference area AR1 (S24: NO), the gesture detection device 70 temporarily ends the current processing. On the other hand, if the position of the right hand PRh is present in the right reference area AR1 (S24: YES), the gesture detection device 70 determines whether the position of the right hand PRh is moved to the right inner area AR3 within the first determination time T1 (S25). Here, "within the first determination time T1" refers to an elapsed time from a time point at which an affirmative determination is made at step S24 within the first determination time T1. At step S25, the movement determination of the position of the right hand PRh is repeatedly performed for a newly captured image.

[0076] If the position of the right hand PRh is not moved to the right inner area AR3 within the first determination time T1 (S25: NO), the gesture detection device 70 temporarily ends the current processing. On the other hand, if the position of the right hand PRh is moved to the right inner area AR3 within the first determination time T1 (S25: YES), the gesture detection device 70 determines whether the position of the right hand PRh is moved to the right reference area AR1 within the second determination time T2 (S26). Here, "within the second determination time T2" ​​refers to an elapsed time from a time when the position of the right hand PRh is moved to the right inner area AR3 within the second determination time T2. At step S26, the movement determination of the position of the right hand PRh is repeatedly performed for a newly captured image in a similar manner to step S25.

[0077] When the position of the right hand PRh is moved into the right reference area AR1 within the second determination time T2 (S26: YES), that is, when the user performs the right inward reversal gesture, the gesture detection device 70 outputs a close command signal for the rear door 32 to the door control device 60 (S27). Thereafter, the gesture detection device 70 ends the current processing. On the other hand, when the position of the right hand PRh is not moved into the right reference area AR1 within the second determination time T2 (S26: NO), the gesture detection device 70 determines whether the position of the right hand PRh exists in the right inner area AR3 for the third determination time T3 (S28). Here, the third determination time T3 refers to an elapsed time from a time when the position of the right hand PRh is moved into the right inner area AR3 as the third determination time T3.Therefore, the third determination time T3 is longer than the second determination time T2. At step S28, the movement determination of the right hand position PRh is repeatedly performed for a newly captured image in a manner similar to step S25.

[0078] If the position of the right hand PRh is not present in the right inner area AR3 during the third determination time T3 (S28: NO), that is, if the position of the right hand PRh is moved out of the right inner area AR3 within the third determination time T3, the gesture detection device 70 temporarily ends the current processing. On the other hand, if the position of the right hand PRh is held in the right inner area AR3 for the third determination time T3 (S28: YES), the gesture detection device 70 outputs a close command signal for the front door 31 and the rear door 32 (S29). A case where an affirmative determination is made in step S28 is a case where it is determined that the user has performed the right inward hold gesture. Thereafter, the gesture detection device 70 ends the current processing.

[0079] In the Fig.In the flowchart shown in Fig. 6, steps S21, S24 to S26 and S28 include a “position acquiring step”, step S23 corresponds to a “direction determining step”, and steps S24 to S26 and S28 correspond to a “gesture determining step”. <Funktion des vorliegenden Ausführungsbeispiels>

[0080] A function is described when a door of the stopped vehicle 10 is to be opened.

[0081] When a user wants to open the rear door 32 of the vehicle 10, the user makes the right outward inversion gesture while walking toward a left side surface of the vehicle 10. That is, the user moves a right hand near the rear door 32 in the opening direction of the rear door 32 and then returns the right hand to the original position. Then, the rear door 32 is opened. On the other hand, when the user wants to open the front door 31 of the vehicle 10, the user makes the left outward inversion gesture while walking toward the left side surface of the vehicle 10. That is, the user moves a left hand near the front door 31 in the opening direction of the front door 31 and then returns the left hand to the original position. Then, the front door 31 is opened.

[0082] When the user wants to open both the front door 31 and the rear door 32 of the vehicle 10, the user makes the right outward hold gesture or the left outward hold gesture while walking toward a side surface of the vehicle 10. That is, the user moves the right hand in the opening direction of the rear door 32 and then stops the right hand, or moves the left hand in the opening direction of the front door 31 and then stops the left hand. Then, the front door 31 and the rear door 32 are opened. <Wirkungen des vorliegenden Ausführungsbeispiels>

[0083] (1) When a user opens the rear door 32, the user makes the right outward reverse gesture in a state where the user faces the left side surface of the vehicle 10. That is, when the user opens the rear door 32, the user makes a gesture that includes an action of moving the right hand near the rear door 32 in the opening direction of the rear door 32. When the user opens the front door 31, the user makes the left outward reverse gesture in a state where the user faces the left side surface of the vehicle 10. That is, when the user opens the front door 31, the user makes a gesture that includes an action of moving the left hand near the front door 31 in the opening direction of the front door 31.In this way, when the user wants to open one of the two doors, they can perform a gesture with one hand near one door and can perform a gesture in which the hand is moved in an opening direction of one door. Therefore, the gesture detection device 70 can improve user convenience.

[0084] When it is determined that the user is facing a side surface of the vehicle 10, the gesture detection device 70 begins to determine whether the user has made a gesture. Therefore, the gesture detection device 70 can reduce the possibility of erroneously determining that a user has made a gesture based on the movement of a person walking past a side of the vehicle 10 while waving their arm.

[0085] (2) When a user makes a gesture while approaching the vehicle 10, the position of the right hand PRh and the position of the left hand PLh in an image are changed because the user makes a gesture and approaches the vehicle 10. Regarding this point, the gesture detection device 70 determines whether the user has made a gesture based on changes in the position of the right hand PRh and the position of the left hand PLh relative to the position of the neck PN, the position of the right shoulder PRs, and the position of the left shoulder PLs. Therefore, the gesture detection device 70 can accurately determine whether the user has made a gesture when the user makes a gesture while approaching the vehicle 10 and when the user stops and makes a gesture.

[0086] (3) The gesture detection device 70 outputs an opening command signal for the rear door 32 when the gesture detection device 70 determines that the user has made the right outward reversal gesture. The gesture detection device 70 outputs an opening command signal for the front door 31 when the gesture detection device 70 determines that the user has made the left outward reversal gesture. Each of the right outward reversal gesture and the left outward reversal gesture includes a movement of the right hand or the left hand in one direction and a movement of the right hand or the left hand back to an initial position. Therefore, compared to a case where a gesture for opening the front door 31 and the rear door 32 is only an action of moving the right hand or the left hand in one direction, the gesture detection device 70 is less likely to misdetermine a gesture.

[0087] (4) The gesture detection device 70 outputs an opening command signal for the front door 31 and the rear door 32 when the gesture detection device 70 determines that the user has made the right outward holding gesture, or when the gesture detection device 70 determines that the user has made the left outward holding gesture. Both the right outward holding gesture and the left outward holding gesture include an action of moving the right hand or the left hand in one direction and an action of holding the right hand or the left hand. Therefore, compared with a case where a gesture for opening the front door 31 and the rear door 32 is only an action of moving the right hand or the left hand in one direction, the gesture detection device 70 is less likely to misdetermine a gesture.

[0088] (5) The user can select whether to open only the rear door 32 or both the front door 31 and the rear door 32 by selectively using the right outward reverse gesture and the right outward hold gesture. The user can select whether to open only the front door 31 or both the front door 31 and the rear door 32 by selectively using the left outward reverse gesture and the left outward hold gesture. Therefore, the gesture detection device 70 can further improve the user's convenience.

[0089] (6) The gesture detection device 70 determines whether a user is facing a side surface of the vehicle 10 based on the shoulder width Ls, which is a distance between the right shoulder position PRs and the left shoulder position PLs. Therefore, the gesture detection device 70 can accurately determine whether the user is facing the side surface of the vehicle 10.

[0090] (7) In the above description of the effects, the effects are described in a case where the user performs the right outward reverse gesture, the right outward hold gesture, the left outward reverse gesture, and the left outward hold gesture to open at least one of the front door 31 and the rear door 32. These effects are also achieved in a case where the user performs the right inward reverse gesture, the right inward hold gesture, the left inward reverse gesture, and the left inward hold gesture to close at least one of the front door 31 and the rear door 32. The gesture detection device 70 can open or close the front door 31 and the rear door 32 according to a moving direction of a user's hand.

[0091] (8) Since the camera 50 is installed between the front opening portion 21 and the rear opening portion 22, the right and left halves of a user's body tend to be evenly displayed in an image when the camera 50 images a user facing a side surface of the vehicle 10. Therefore, the gesture detection device 70 can easily acquire the right hand position PRh, the right shoulder position PRs, the right toe position PRt, the left hand position PLh, the left shoulder position PLs, and the left toe position PLt.

[0092] The present embodiment may be modified as follows. The present embodiment and the following modifications may be combined without technical contradiction. • A body part of a user used to perform a gesture can be changed accordingly. The first body part is preferably a part of the right half of the user's body that can be easily moved independently by the user. For example, the first body part can be a right toe tip of the user. The second body part is preferably a part of the left half of the user's body that can be easily moved independently by the user. For example, the second body part can be a left toe tip of the user. The reference body part can be a body part on a midline of the user's body. For example, the reference body part can be a face of the user or a chest of the user instead of the user's neck. • The direction determination unit 74 may determine whether a user is facing a side surface of the vehicle 10 by another determination method. Some examples are described below. • The direction determining unit 74 may determine that the user is facing the side surface of the vehicle 10 when the right shoulder position PRs exists on one side of the neck position PN and the left shoulder position PRs exists on the other side of the neck position PN in the X direction. On the other hand, the direction determining unit 74 may determine that the user is not facing the side surface of the vehicle 10 when only the right shoulder position PRs or the left shoulder position PLs exists on one side of the neck position PN in the X direction. However, even when the right shoulder position PRs and the left shoulder position PRs exist on both sides of the neck position PN, a middle position between the right shoulder position PRs and the left shoulder position PLs may deviate from the neck position PN in the X direction.In this case, the direction determination unit 74 may determine that the user is not facing the side surface of the vehicle 10. • The direction determination unit 74 may determine whether the user is facing the side surface of the vehicle 10 based on the positions of body parts other than the right shoulder and left shoulder. For example, the direction determination unit 74 may determine whether the user is facing the side surface of the vehicle 10 based on a position of the user's right eye and a position of the user's left eye. The direction determination unit 74 may determine whether the user is facing the side surface of the vehicle 10 based on a position of the user's right ear and a position of the user's left ear. • The direction determination unit 74 can determine that the user is facing the side surface of the vehicle 10 in a case where all the positions of a plurality of body parts that can be imaged by the camera 50 when the user is facing the side surface of the vehicle 10 can be acquired. For example, the direction determination unit 74 can determine that the user is facing the side surface of the vehicle 10 when the user's neck position PN, right shoulder position PRs, left shoulder position PLs, right hand position PRh, left hand position PLh, right toe position PRt, and left toe position PLt can be acquired. • The gesture determination unit 75 may not set the right reference area AR1 and the right outer area AR2 in the right outward gesture determination processing. In this case, the gesture determination unit 75 calculates the reference distance LR, which is a distance from the neck position PN to the right shoulder position PRs in the X direction, and a comparison distance, which is a distance from the neck position PN to the right hand position PRh in the X direction. Then, the gesture determination unit 75 determines a state of the user's right hand based on a size relationship between the reference distance LR and the comparison distance. Specifically, the gesture determination unit 75 determines that the user has moved the right hand in the opening direction of the rear door 32 when the comparison distance becomes sufficiently larger than the reference distance LR, starting from a state where the comparison distance is substantially equal to the reference distance LR.In this way, the gesture determination unit 75 can determine whether the user has made a gesture by comparing the distances. The same applies to the right inward gesture determination processing, the left outward gesture determination processing, and the left inward gesture determination processing. • A gesture to be determined by the gesture determining unit 75 may include at least the right outward reverse gesture and the left outward reverse gesture. • When it is determined that a user has made the right outward reversal gesture, the output unit 76 may output an open command signal to open the front door 31 and the rear door 32. When it is determined that the user has made the right outward hold gesture, the output unit 76 may output an open command signal to open the rear door 32 corresponding to the right hand. When it is determined that the user has made the right inward reversal gesture, the output unit 76 may output a close command signal to close the front door 31 and the rear door 32. When it is determined that the user has made the right inward hold gesture, the output unit 76 may output an open command signal to close the rear door 32 corresponding to the right hand. The same applies if the user makes the left outward reverse gesture, the left outward hold gesture, the left inward reverse gesture and the left outward hold gesture. • A user's gesture to open the rear door 32 may be a gesture in which the user simply moves the lowered right hand to the right. In this case, the gesture determination unit 75 may determine that the user has performed the gesture if the user moves the position of the right hand PRh from the right reference area AR1 to the right outer area AR2. The same applies to a user's gesture to open the front door 31.

[0093] A user's gesture to open the rear door 32 may be a gesture in which the user simply moves the downwardly lowered right hand upward. In this case, an upward direction, which is a movement direction of the user's right hand, corresponds to the "first direction," which corresponds to the opening direction of the rear door 32. That is, the "first direction" does not have to coincide with the opening direction of the rear door 32. The same applies to a user's gesture to open the front door 31. • The vehicle body 20 may not include the center pillar 23. In this case, the front opening portion 21 and the rear opening portion 22 may be integrated into one opening portion. That is, a front portion of one opening portion corresponds to the front opening portion 21, and a rear portion of the one opening portion corresponds to the rear opening portion 22. • For example, the gesture detection device 70 may be configured to detect only a gesture that triggers an operation of the rear door 32. In other words, in the above-described embodiment, the front door 31 cannot be opened and closed by a gesture. In this case, the rear opening portion 22 corresponds to an "opening portion" and the rear door 32 corresponds to a "door." In this case, in a state where the user faces, for example, a left side surface of the vehicle 10, the right hand corresponds to a "predetermined body part," the position of the right hand PRh corresponds to a "predetermined position," the opening direction of the rear door 32 corresponds to a "predetermined direction" that corresponds to the opening direction of the rear door 32, and the right outward gesture corresponds to a "predetermined gesture." • For example, the gesture detection device 70 may be configured to detect only a gesture that triggers an operation of the front door 31. In other words, the rear door 32 cannot be opened and closed by a gesture in the above-described embodiment. In this case, the front opening portion 21 corresponds to the "opening portion" and the front door 31 corresponds to the "door." In this case, in a state where the user faces the left side surface of the vehicle 10, for example, the left hand corresponds to the "predetermined body part," the position of the left hand PLh corresponds to the "predetermined position," the opening direction of the front door 31 corresponds to the "predetermined direction," which corresponds to the opening direction of the front door 31, and the left outward gesture corresponds to the "predetermined gesture." • The vehicle body 20 may include only one of the front opening portion 21 and the rear opening portion 22, and the vehicle 10 may include only one of the front door 31 and the rear door 32 that opens and closes the one opening portion. In this case, the gesture detection device 70 may detect at least one door opening gesture that opens and closes the one opening portion. • The position of the camera 50 can be conveniently changed. When a plurality of cameras 50 are used, the cameras 50 can be provided at both the front door 31 and the rear door 32. • The vehicle 10 may include a rear door and a rear camera that can image the rear of the vehicle 10 regardless of the opening and closing state of the rear door. The vehicle body 20 may include an opening portion that opens rearward and is opened and closed by the rear door. The gesture detection device 70 may open and close the rear door according to the content of a gesture performed by a user. In this case, the opening portion that opens toward the rear of the vehicle body 20 corresponds to the "opening portion," and the rear door corresponds to the "door."

[0094] In the above configuration, the rear door may include a first door that opens and closes the first opening portion and a second door that opens and closes the second opening portion, when a right side of the opening portion is defined as the first opening portion and a left side of the opening portion is defined as the second opening portion. The gesture detection device 70 may open and close the first door or the second door depending on the content of a gesture performed by a user. • The gesture sensing device 70 is not limited to a processing circuit that includes a CPU and memory and executes software processing. For example, the gesture sensing device 70 may include a dedicated hardware circuit that executes at least some of the various types of processing performed in the above-described embodiment. Examples of the dedicated hardware circuit include an ASIC. ASIC is an abbreviation for "Application Specific Integrated Circuit." That is, the gesture sensing device 70 may have any of the following configurations (a) to (c). (a) A processing circuit including a processing device that executes all the above processing according to a program, and a program storage device such as a ROM that stores the program. (b) A processing circuit including a processing device and a program storage device that execute part of the above processing according to a program, and a dedicated hardware circuit that executes the remaining processing. (c) A processing circuit including a dedicated hardware circuit for performing all of the above processing.

[0095] Here, there may be a plurality of software execution devices, including a processing device and a program storage device, as well as a plurality of dedicated hardware circuits. The same applies to the door control device 60. [Summary of the present embodiments]

[0096] The present embodiment includes at least the following configurations.

[0097] A gesture detection device (70) according to the present embodiment is applied to a vehicle (10) including a vehicle body (20) having an opening portion (21, 22), a door (31, 32) configured to open and close the opening portion (21, 22), a camera (50) configured to image a user facing the opening portion (21, 22), and a door control unit (41, 42) configured to control the door (31, 32). The gesture detection device (70) detects a gesture that triggers an operation of the door (31, 32) and is performed by the user using a predetermined body part. A body part of the user that is different from the predetermined body part is a reference body part.The gesture detection device (70) includes a position obtaining unit (73) configured to obtain a predetermined position (PRh, PLh), which is a position of the predetermined body part, and a reference position (PN, PRs, PLs), which is a position of the reference body part, in an image captured by the camera (50), a direction determining unit (74) configured to determine whether the user is facing the opening portion (21, 22) based on the reference position (PN, PRs, PLs) in the image, and a gesture determining unit (75) configured to determine whether the user has made a gesture based on changes in the predetermined position (PRh, PLh) and the reference position (PN, PRs, PLs) in a plurality of images captured at different imaging times when the direction determining unit (74) determines that the user is facing the opening portion (21, 22). 22).The gesture determination unit (75) determines that the user has made a predetermined gesture for opening at least the door (31, 32) when the predetermined position (PRh, PLh) is moved relative to the reference position (PN, PRs, PLs) in a predetermined direction corresponding to an opening direction of the door (31, 32).

[0098] If it can be determined that the user is facing the opening portion, the gesture detection device determines whether the user has performed a gesture. Specifically, the gesture detection device determines that the user has performed a predetermined gesture when the user moves the predetermined body part in the predetermined direction. Therefore, the user can open the door at least by moving the predetermined body part in the predetermined direction corresponding to the opening direction of the door.

[0099] When the user makes a gesture while approaching the vehicle, the predetermined position in the image is changed because the user makes the gesture and approaches the vehicle. Regarding this point, the gesture detection device determines whether the user has made a gesture based on a change in the predetermined position relative to the reference position. Therefore, the gesture detection device can accurately determine whether the user has made a gesture when the user makes a gesture while approaching the vehicle and when the user stops and makes a gesture. Therefore, the gesture detection device can improve the user's convenience.

[0100] In the present embodiment, the opening portion (21, 22) may be a first opening portion (22), the door (31, 32) may be a first door (32), the predetermined body part may be a first body part, the predetermined position (PRh, PLh) may be a first position (PRh), the predetermined direction may be a first direction, the predetermined gesture may be a first gesture, and a body part of the user that is different from the first body part and the reference body part may be a second body part. The vehicle body (20) may further include a second opening portion (21) adjacent to the first opening portion (22). The vehicle (10) may further include a second door (31) controlled by the door control unit (41, 42) and configured to open and close the second opening portion (21).The camera (50) can be further configured to image the user facing the second opening portion (21). The position acquisition unit (73) can be further configured to acquire a second position (PLh), which is a position of the second body part, in the image captured by the camera (50). The direction determination unit (74) can be further configured to determine, based on the reference position (PN, PRs, PLs) in the image, whether the user is facing the second opening portion (21).The gesture determination unit (75) may further be configured to determine whether the user has made a gesture based on changes in the second position (PLh) and the reference position (PN, PRs, PLs) in a plurality of images taken at different imaging times when the direction determination unit (74) determines that the user is facing the first opening portion (22) and the second opening portion (21), and to determine that the user has made a second gesture for opening at least the second door (31) when the second position (PLh) is moved relative to the reference position (PN, PRs, PLs) in a second direction corresponding to an opening direction of the second door (31).

[0101] If it can be determined that the user is facing the first opening portion and the second opening portion, the gesture detection device determines whether the user has made a gesture. Specifically, the gesture detection device determines that the user has made the second gesture when the user moves the second body part in the second direction. Therefore, the user can open at least the second door by moving the second body part in the second direction, which corresponds to the opening direction of the second door.

[0102] When the user makes a gesture while approaching the vehicle, the first position and the second position in the image change because the user makes the gesture and approaches the vehicle. Regarding this point, the gesture detection device determines whether the user has made a gesture based on changes in the first position and the second position relative to the reference position. Therefore, the gesture detection device can accurately determine whether the user has made a gesture when the user makes a gesture while approaching the vehicle and when the user stops and makes a gesture. Therefore, the gesture detection device can improve the user's convenience.

[0103] In the present embodiment, in a state where the user faces the first opening portion (22) and the second opening portion (21), the first opening portion (22) may face a right half of the user's body, the second opening portion (21) may face a left half of the user's body, the first body part may be a right hand of the user, and the second body part may be a left hand of the user.

[0104] When the user faces the first opening portion and the second opening portion, the user can open a door located on a right front side with a gesture using the right hand. Furthermore, the user can open a door located on a left front side with a gesture using the left hand. Therefore, the gesture detection device can further improve user convenience.

[0105] In the present embodiment, the gesture determination unit (75) may be further configured to determine that the user has performed the first gesture when the first position (PRh) is moved in the first direction and then moved in a direction opposite to the first direction relative to the reference position (PN, PRs, PLs), and determine that the user has performed the second gesture when the second position (PLh) is moved in the second direction and then moved in a direction opposite to the second direction relative to the reference position (PN, PRs, PLs).

[0106] The gesture detection device can accurately determine whether the user performed the first gesture, compared to, for example, a case where it is determined that the user performed the first gesture only by moving the first position in the first direction. Likewise, the gesture detection device can accurately determine whether the user performed the second gesture, compared to, for example, a case where it is determined that the user performed the second gesture only by moving the second position in the second direction. Therefore, the gesture detection device is less likely to erroneously determine the first and second gestures.

[0107] In the present embodiment, the gesture determination unit (75) may be further configured to determine that the user has performed the first gesture when the first position (PRh) is moved in the first direction and then stopped relative to the reference position (PN, PRs, PLs), and determine that the user has performed the second gesture when the second position (PLh) is moved in the second direction and then stopped relative to the reference position (PN, PRs, PLs).

[0108] The gesture detection device can accurately determine whether the user performed the first gesture, compared to, for example, a case where it is determined that the user performed the first gesture only by moving the first position in the first direction. Likewise, the gesture detection device can accurately determine whether the user performed the second gesture, compared to, for example, a case where it is determined that the user performed the second gesture only by moving the second position in the second direction. Therefore, the gesture detection device is less likely to erroneously determine the first and second gestures.

[0109] In the present embodiment, the first gesture may include a third gesture for opening only the first door (32) and a fourth gesture, which is different from the third gesture and is a gesture for opening both the first door (32) and the second door (31). The second gesture may include a fifth gesture for opening only the second door (31) and a sixth gesture, which is different from the fifth gesture and is a gesture for opening both the first door (32) and the second door (31).The gesture determination unit (75) may be further configured to determine that the user has performed the third gesture when the first position (PRh) is moved in the first direction and then moved in a direction opposite to the first direction relative to the reference position (PN, PRs, PLs), to determine that the user has performed the fourth gesture when the first position (PRh) is moved in the first direction and then stopped relative to the reference position (PN, PRs, PLs), to determine that the user has performed the fifth gesture when the second position (PLh) is moved in the second direction and then moved in a direction opposite to the second direction relative to the reference position (PN, PRs, PLs), and to determine that the user has performed the sixth gesture when the second position (PLh) is moved in the second direction and then stopped relative to the reference position (PN, PRs, PLs).

[0110] The user can select whether to open only the first door or both the first and second doors by selectively using the third gesture and the fourth gesture using the first body part. The user can select whether to open only the second door or both the first and second doors by selectively using the fifth gesture and the sixth gesture using the second body part. Therefore, the gesture sensing device can further improve user convenience.

[0111] In the present embodiment, the reference body part may include a right shoulder and a left shoulder of the user, and the reference position (PN, PRs, PLs) may include a right shoulder position (PRs) and a left shoulder position (PLs) of the user. The direction determination unit (74) may be further configured to determine whether the user is facing the first opening portion (22) and the second opening portion (21) based on a shoulder width (Ls), which is a distance between the right shoulder position (PRs) and the left shoulder position (PLs) of the user in the image.

[0112] If the user faces the first opening portion and the second opening portion, the shoulder width is likely to be long. On the other hand, if the user does not face the first opening portion and the second opening portion, the shoulder width is likely to be short or the shoulder width is not calculated correctly. Regarding this point, the gesture detection device determines whether the user faces the first opening portion and the second opening portion based on the shoulder width. Therefore, the gesture detection device can accurately determine whether the user faces the first opening portion and the second opening portion.

[0113] In the present embodiment, the first opening portion (22) can be opened in a side surface of the vehicle body (20), the second opening portion (21) can be opened in front of the first opening portion (22) in the side surface of the vehicle body (20). The first door (32) can be a rear door (32), the second door (31) can be a front door (31), the first direction can be a direction that is the same as the opening direction of the first door (32), and the second direction can be a direction that is the same as the opening direction of the second door (31).

[0114] In the gesture sensing device, the first direction is a direction equal to the opening direction of the first door, and the second direction is a direction equal to the opening direction of the second door. Therefore, it is easy for the user to intuitively understand a movement direction of the first body part when performing the first gesture and a movement direction of the second body part when performing the second gesture.

[0115] In the present embodiment, the camera (50) and the position acquisition unit (73) can be installed between the first opening portion (22) and the second opening portion (21) in the vehicle body (20) in an arrangement direction of the first opening portion (22) and the second opening portion (21). The first position (PRh), the second position (PLh), and the reference position (PN, PRs, PLs) can be acquired from the image captured by the camera (50).

[0116] Since the camera is installed between the first opening portion and the second opening portion, a right half of the user's body and a left half of the user's body tend to be evenly represented in one image when the camera images the user facing the first opening portion and the second opening portion. Therefore, the gesture detection device can easily acquire the first position and the second position when the first body part is selected from the right half of the user's body or the left half of the user's body, and the second body part is selected from the other of the right half of the user's body or the left half of the user's body.

[0117] In the present embodiment, the first gesture may be a first opening gesture, and the second gesture may be a second opening gesture. The gesture determination unit (75) may be further configured to determine that the user has performed a first closing gesture for closing at least the first door (32) when the first position (PRh) is moved relative to the reference position (PN, PRs, PLs) in the direction opposite to the first direction, and to determine that the user has performed a second closing gesture for closing at least the second door (31) when the second position (PLh) is moved relative to the reference position (PN, PRs, PLs) in the direction opposite to the second direction.

[0118] The user can close at least the first door by moving the first body part in the direction opposite to the first direction. The user can close at least the second door by moving the second body part in the direction opposite to the second direction. In this way, the user can selectively open and close the first door and the second door by selectively using a gesture.

[0119] A gesture detection method according to the present embodiment is applied to a vehicle (10) including a vehicle body (20) with an opening portion (21, 22), a door (31, 32) configured to open and close the opening portion (21, 22), a camera (50) configured to image a user facing the opening portion (21, 22), and a door control unit (41, 42) configured to control the door (31, 32). The gesture detection method detects a gesture that triggers an operation of the door (31, 32) and is performed by the user using a predetermined body part. A body part of the user that is different from the predetermined body part is a reference body part.The gesture detection method includes a position detection step of acquiring a predetermined position (PRh, PLh), which is a position of the predetermined body part, and a reference position (PN, PRs, PLs), which is a position of the reference body part, in an image captured by the camera (50), a direction determination step of determining whether the user is facing the opening portion (21, 22) based on the reference position (PN, PRs, PLs) in the image, and a gesture determination step of determining whether the user has performed a gesture based on changes in the predetermined position (PRh, PLh) and the reference position (PN, PRs, PLs) in a plurality of images captured at different imaging times when it is determined in the direction determination step that the user is facing the opening portion (21, 22).In the gesture determination step, it is determined that the user has performed a gesture for opening at least the door (31, 32) when the predetermined position (PRh, PLh) is moved relative to the reference position (PN, PRs, PLs) in a predetermined direction corresponding to an opening direction of the door (31, 32).

[0120] The gesture detection method according to the present embodiment can obtain the same effect as the gesture detection device described above.

[0121] In the present embodiment, the opening portion (21, 22) may be a first opening portion (22), the door (31, 32) may be a first door (32), the predetermined body part may be a first body part, the predetermined position (PRh, PLh) may be a first position (PRh), the predetermined direction may be a first direction, the predetermined gesture may be a first gesture, and a body part of the user that is different from the first body part and the reference body part may be a second body part. The vehicle body (20) may further include a second opening portion (21) adjacent to the first opening portion (22). The vehicle body (20) may further include a second door (31) controlled by the door control unit (41, 42) and configured to open and close the second opening portion (21).The camera (50) may further be configured to image the user facing the second opening portion (21). The position detection step may include acquiring a second position (PLh), which is a position of the second body part, in the image captured by the camera (50). The direction determination step may include determining whether the user is facing the second opening portion (21) based on the reference position (PN, PRs, PLs) in the image.The gesture determining step may include determining whether the user has performed a gesture based on changes in the second position (PLh) and the reference position (PN, PRs, PLs) in a plurality of images taken at different imaging times when it is determined that the user is facing the first opening portion (22) and the second opening portion (21), and determining that the user has performed a second gesture for opening at least the second door (31) when the second position (PLh) is moved relative to the reference position (PN, PRs, PLs) in a second direction corresponding to an opening direction of the second door (31).

[0122] According to the above configuration, the same effects as the corresponding gesture detection device can be obtained. List of reference symbols 10 vehicles 20 vehicle body 21 front opening section (second opening section) 22 rear opening section (first opening section) 23 Center column 31 front door (second door) 32 rear door (first door) 41 Front door control unit 42 Rear door control unit 50 Camera 60 door control device 70 Gesture detection device 71 storage unit 72 Image Acquisition Unit 73 Position procurement unit 74 Direction determination unit 75 Gesture determination unit 76 Output unit PN neck position PRh Position of the right hand PLh Position of the left hand PRs position of the right shoulder PL's left shoulder position PRt Position of the right toe PLt Position of the left toe QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2019-105046 A [0002, 0004]

Claims

[1] A gesture detection device applied to a vehicle including a vehicle body having an opening portion, a door configured to open and close the opening portion, a camera configured to image a user facing the opening portion, and a door control unit configured to control the door, and detecting a gesture that triggers an operation of the door and is performed by the user using a predetermined body part, wherein a body part of the user that differs from the predetermined body part is a reference body part, wherein the gesture detection device comprises: a position acquiring unit configured to acquire a predetermined position, which is a position of the predetermined body part, and a reference position, which is a position of the reference body part, in an image captured by the camera; a direction determination unit configured to determine whether the user is facing the opening portion based on the reference position in the image; and a gesture determination unit configured to determine whether the user has performed a gesture based on changes in the predetermined position and the reference position in a plurality of images captured at different imaging times when the direction determination unit determines that the user is facing the opening portion, wherein the gesture determination unit determines that the user has performed a predetermined gesture for opening at least the door when the predetermined position is moved relative to the reference position in a predetermined direction corresponding to an opening direction of the door. [2] Gesture detection device according to claim 1, wherein the opening portion is a first opening portion, the door is a first door, the predetermined body part is a first body part, the predetermined position is a first position, the predetermined direction is a first direction, the predetermined gesture is a first gesture, and a body part of the user that is different from the first body part and the reference body part is a second body part, the vehicle body further includes a second opening portion adjacent to the first opening portion, the vehicle further includes a second door which is controlled by the door control unit and is configured to open and close the second opening section, the camera is further configured to image the user facing the second opening portion, the position acquisition unit is further configured to acquire a second position, which is a position of the second body part, in the image captured by the camera, the direction determination unit is further configured to determine whether the user is facing the second opening portion based on the reference position in the image, and the gesture determination unit is further configured to to determine whether the user has performed a gesture based on changes in the second position and the reference position in a plurality of images taken at different imaging times when the direction determining unit determines that the user is facing the first opening portion and the second opening portion, and to determine whether the user has performed a second gesture to open at least the second door when the second position is moved relative to the reference position in a second direction corresponding to an opening direction of the second door. [3] A gesture detection device according to claim 2, wherein in a state where the user faces the first opening portion and the second opening portion, the first opening section faces a right half of the user's body and the second opening section faces a left half of the user's body, and the first body part is a right hand of the user and the second body part is a left hand of the user. [4] A gesture detection device according to claim 2 or 3, wherein the gesture determination unit is further configured to to determine that the user has performed the first gesture when the first position is moved in the first direction and then moved in a direction opposite to the first direction relative to the reference position, and to determine that the user has performed the second gesture when the second position is moved in the second direction and then moved in a direction opposite to the second direction relative to the reference position. [5] A gesture detection device according to claim 2 or 3, wherein the gesture determination unit is further configured to to determine that the user has performed the first gesture when the first position is moved in the first direction and then stopped relative to the reference position, and to determine that the user has performed the second gesture when the second position is moved in the second direction and then stopped relative to the reference position. [6] Gesture detection device according to claim 2 or 3, wherein the first gesture includes a third gesture for opening only the first door and a fourth gesture, which is a different gesture from the third gesture and is a gesture for opening the first door and the second door, the second gesture includes a fifth gesture for opening only the second door and a sixth gesture, which is a different gesture from the fifth gesture and is a gesture for opening the first door and the second door, and The gesture determination unit is further configured to to determine that the user has performed the third gesture when the first position is moved in the first direction and then moved in a direction opposite to the first direction relative to the reference position, to determine that the user has performed the fourth gesture when the first position is moved in the first direction and then stopped relative to the reference position, to determine that the user has performed the fifth gesture when the second position is moved in the second direction and then moved in a direction opposite to the second direction relative to the reference position, and to determine that the user has performed the sixth gesture when the second position is moved in the second direction and then stopped relative to the reference position. [7] A gesture detection device according to claim 2 or 3, wherein the reference body part includes a right shoulder and a left shoulder of the user, the reference position includes a right shoulder position and a left shoulder position of the user, and the direction determination unit is further configured to determine whether the user is facing the first opening portion and the second opening portion based on a shoulder width, which is a distance between the right shoulder position and the left shoulder position of the user in the image. [8] A gesture detection device according to claim 2 or claim 3, wherein the first opening section opens in a side surface of the vehicle body, the second opening section opens in front of the first opening section in the side surface of the vehicle body, the first door is a rear door, the second door is a front door, the first direction is a direction equal to an opening direction of the first door, and the second direction is a direction equal to an opening direction of the second door. [9] A gesture detection device according to claim 2 or 3, wherein the camera is installed between the first opening portion and the second opening portion on the vehicle body in an arrangement direction of the first opening portion and the second opening portion, and the position obtaining unit is further configured to obtain the first position, the second position and the reference position from the image captured by the camera. [10] A gesture detection device according to claim 2 or 3, wherein the first gesture is a first opening gesture and the second gesture is a second opening gesture, and the gesture determination unit is further configured to to determine that the user has performed a first closing gesture to close at least the first door when the first position is moved relative to the reference position in a direction opposite to the first direction, and to determine that the user has performed a second closing gesture to close at least the second door when the second position is moved relative to the reference position in a direction opposite to the second direction. [11] A gesture detection method applied to a vehicle including a vehicle body having an opening portion, a door configured to open and close the opening portion, a camera configured to image a user facing the opening portion, and a door control unit configured to control the door, and detecting a gesture that triggers an operation of the door and is performed by the user using a predetermined body part, wherein a body part of the user that differs from the predetermined body part is a reference body part, wherein the gesture detection method comprises: a position acquiring step of acquiring a predetermined position, which is a position of the predetermined body part, and a reference position, which is a position of the reference body part, in an image captured by the camera; a direction determining step for determining whether the user is facing the opening portion based on the reference position in the image; and a gesture determination step for determining whether the user has performed a gesture based on changes in the predetermined position and the reference position in a plurality of images taken at different imaging times when it is determined in the direction determination step that the user is facing the opening portion, wherein the gesture determining step includes determining that the user has performed a predetermined gesture for opening at least the door when the predetermined position is moved relative to the reference position in a predetermined direction corresponding to an opening direction of the door. [12] A gesture detection method according to claim 11, wherein the opening portion is a first opening portion, the door is a first door, the predetermined body part is a first body part, the predetermined position is a first position, the predetermined direction is a first direction, the predetermined gesture is a first gesture, and a body part of the user that is different from the first body part and the reference body part is a second body part, the vehicle body further includes a second opening portion adjacent to the first opening portion, the vehicle body further includes a second door controlled by the door control unit and configured to open and close the second opening portion, the camera is further configured to image the user facing the second opening portion, the position obtaining step includes obtaining a second position, which is a position of the second body part, in the image captured by the camera, the direction determining step includes determining whether the user is facing the second opening portion based on the reference position in the image, and the gesture determination step includes determining whether the user has performed a gesture based on changes in the second position and the reference position in a plurality of images taken at different imaging times when it is determined that the user is facing the first opening portion and the second opening portion, and determining whether the user has performed a second gesture to open at least the second door when the second position is moved relative to the reference position in a second direction corresponding to an opening direction of the second door.

Images