Vehicle door control device

Abstract

To provide a vehicle door control device that can control the vehicle doors based on user actions. [Solution] The vehicle door control device 1 detects the shoulder movements of a user 43 who is in the vicinity of the vehicle 2, and determines whether the user 43 intends to use the vehicle 2 based on the detection results. The vehicle door control device 1 also estimates which of the doors on the vehicle 2 the user 43 is expected to open. If the vehicle door control device 1 determines that the user intends to use the door, it executes control to open the target door.

Description

Technical Field

[0005]

[0001] The present invention relates to a vehicle door control device for controlling a vehicle door.

Background Art

[0002] The following Patent Document 1 describes a door unlock method for unlocking a vehicle door and opening the door based on face authentication. In the door unlock method of Patent Document 1, the distance between an object outside the vehicle and an infrared distance measurement sensor is continuously acquired by the infrared distance measurement sensor. Next, in response to the acquired distance gradually decreasing and the duration for which the distance becomes less than or equal to a first distance threshold reaching a first time threshold, an image collection module installed in the vehicle is controlled to collect a video stream. Then, face recognition is performed based on the image of the collected video stream, and the vehicle door lock is unlocked in response to successful face recognition.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the door unlock method of Patent Document 1 described above, the user can unlock the door by approaching the vehicle and having their face imaged by the vehicle. However, if the user's face is hidden by a hat, mask, etc., there is a risk of face authentication failure and the door not opening. Therefore, there was room for improvement in detecting the user's actions.

[0005] The present invention has been made to solve the above-described conventional problems, and an object thereof is to provide a vehicle door control device that can control a vehicle door based on a user's actions.

Means for Solving the Problems

[0006] To achieve the above objective, the vehicle door control device according to the present invention comprises: a shoulder motion detection unit that detects the movement of a user's shoulders in the vicinity of the vehicle; a user intent determination unit that determines whether or not the user intends to use the vehicle based on the detection results from the shoulder motion detection unit; a target door estimation unit that estimates a target door among the doors of the vehicle that the user is expected to open; and a control unit that, when the user intent determination unit determines that the user intends to use the vehicle, executes control to open the target door estimated by the target door estimation unit. Furthermore, in this specification, the term "control for opening a door" includes the concept of control that performs the following actions: unlocking the door, opening the door, or both. [Effects of the Invention]

[0007] According to the vehicle door control device of the present invention having the above configuration, it is possible to determine whether or not a user intends to use the vehicle based on the movement of the user's shoulders in the vicinity of the vehicle. If it is determined that the user intends to use the vehicle, the target door estimation unit can estimate the target door, i.e., the door that the user is expected to open, and control can be executed to open the door. This makes it possible to control the vehicle doors based on the user's movements. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram of the vehicle according to the first embodiment. [Figure 2] This is a block diagram showing the configuration of a vehicle door control device according to the first embodiment. [Figure 3] This is a flowchart of the door control processing program according to the first embodiment. [Figure 4] This diagram shows the shoulder movement of a user walking on the right side of a vehicle. [Figure 5]Figure 4 shows a graph illustrating the relationship between the X-coordinate and the shoulder angle when a user walks from the front right to the rear of the vehicle. [Figure 6] This graph shows the relationship between the X-coordinate and the shoulder angle when a user walks from the rear of the vehicle to the front right. [Figure 7] This figure shows the user's movement path and shoulder movements when obstacles are present around the vehicle. [Figure 8] This is a block diagram showing the configuration of a vehicle door control device as an example. [Figure 9] This is a flowchart of an example door control processing program. [Figure 10] This is a flowchart of the door control processing program according to the second embodiment. [Figure 11] This is a flowchart of an example door control processing program. [Modes for carrying out the invention]

[0009] (First Embodiment) Hereinafter, one embodiment of the vehicle door control device according to the present invention will be described in detail with reference to the drawings. First, a vehicle 2 equipped with the vehicle door control device 1 according to the first embodiment will be described below. Figure 1 is a schematic diagram of the vehicle 2 according to the first embodiment. Figure 2 is a block diagram of the vehicle door control device 1 according to the first embodiment. In the following description, the front-rear direction, the left-right direction, and the up-down direction of the vehicle 2 will be simply referred to as the front-rear direction, the left-right direction, and the up-down direction, respectively. In addition, devices on the right side of the vehicle 2 may be denoted with R, and devices on the left side of the vehicle 2 may be denoted with L. Furthermore, in addition to the components shown in Figures 1 and 2, the vehicle 2 also has other basic components as a vehicle 2, but in the following description, the configuration related to the control of opening and closing the doors, and the control related to said configuration will be mainly described.

[0010] As shown in Figure 1, vehicle 2 is, for example, a vehicle with a steering wheel 3 on the right side, and comprises a body 11, a driver's side front door 12R, a passenger side front door 12L, a driver's side rear door 13R, a passenger side rear door 13L, and a back door 14. Hereafter, when the front doors 12R, 12L, rear doors 13R, 13L, and back door 14 are described collectively, they may be referred to as "each door." Each door is, for example, a swing-type door. Vehicle 2 also has door lock devices 15A, 15B, 15C, 15D, 15E that control the lock of each door, and door opening and closing devices 16A, 16B, 16C, 16D, 16E that open and close each door. The door lock devices 15A to 15E control the locks of the front doors 12R, 12L, rear doors 13R, 13L, and back door 14, respectively, in that order. The door lock devices 15A to 15E are devices that switch each door between a locked state (locked) and an unlocked state (unlocked). The door opening and closing devices 16A to 16E are devices that open and close the front doors 12R, 12L, rear doors 13R, 13L, and back door 14, respectively, in this order. The door opening and closing devices 16A to 16E are equipped with a motor as a drive source, for example, and open and close each door by driving the motor.

[0011] Note that the configuration of vehicle 2 shown in Figure 1 is just one example. For example, vehicle 2 is not limited to a vehicle with the handle 3 on the right side, but may also be a vehicle with the handle 3 on the left side. Also, each door is not limited to a swing-type door, but may be a door with other opening and closing methods such as a sliding door. Furthermore, each door may have a different opening and closing method. Therefore, only the rear doors 13R and 13L may be sliding doors. Also, the drive source for the door opening and closing devices 16A to 16E is not limited to a motor, but may also be a drive source such as a hydraulic cylinder. Furthermore, vehicle 2 may be an internal combustion engine vehicle driven by an internal combustion engine (engine, etc.), an electric vehicle driven by an electric motor, a fuel cell vehicle, etc., or a hybrid vehicle having multiple drive sources such as these. Furthermore, there are no particular limitations on the type of vehicle 2, the number of wheels, etc. Also, vehicle 2 may be a vehicle capable of manual driving, a vehicle capable of automatic driving, or a vehicle capable of switching between both types of driving.

[0012] Also, as shown in FIGS. 1 and 2, the vehicle door control device 1 includes a front camera 5, side cameras 6R and 6L, a rear camera 7, various sensors 8, a wireless communication device 9, and a vehicle control ECU (Electronic Control Unit) 10. Hereinafter, when collectively referring to the front camera 5, side cameras 6R and 6L, and rear camera 7, they may be described as each camera.

[0013] Each camera is, for example, an imaging device having a solid-state imaging device such as a CCD, and images the periphery of the vehicle. The front camera 5 is attached, for example, above the front bumper of the vehicle 2 or behind the rearview mirror, and is installed in a state where the optical axis direction faces the front of the vehicle 2. The side cameras 6R and 6L are attached to the left and right side mirrors of the vehicle 2, respectively, and are installed in a state where the optical axis direction faces the side of the vehicle 2. The rear camera 7 is attached, for example, above the license plate attached to the rear of the vehicle 2, and is installed in a state where the optical axis direction faces the rear of the vehicle 2.

[0014] The various sensors 8 are sensors for realizing various functions of the vehicle 2. As the sensor 8, for example, an ultrasonic sensor, a millimeter-wave radar, a laser sensor, etc. can be adopted as sensors for detecting obstacles around the vehicle. Alternatively, as the sensor 8, a vehicle speed sensor, an acceleration sensor, a gyro sensor, a steering sensor, a shift position sensor, etc. can be adopted as sensors used for the running of the vehicle 2.

[0015] The wireless communication device 9 is a device that performs wireless communication with the portable device 41. The portable device 41 is, for example, a so-called electronic key. Alternatively, the portable device 41 may be a smartphone used in a digital key system or other communication terminal capable of wireless communication with the vehicle 2. The portable device 41 has buttons for operating the door lock devices 15A to 15E and the door opening / closing devices 16A to 16E.

[0016] (Regarding the vehicle control ECU 10) The vehicle control ECU (hereinafter simply referred to as ECU) 10 is an electronic control unit that comprehensively controls the entire vehicle 2 including the vehicle door control device 1, and includes a CPU 31 as an arithmetic unit and a control unit, and a RAM 32 used as a working memory when the CPU 31 executes various arithmetic processes. In addition to the control program, it also has an internal storage device such as a ROM 33 in which a door control processing program (see FIG. 3) described later is recorded, and a flash memory 34 that stores programs and flag values read from the ROM 33.

[0017] The ECU 10 realizes various functional units by executing programs with the CPU 31. For example, the shoulder movement detection unit 31A is a functional unit that detects the movement of the shoulders of a user existing around the vehicle. The usage intention determination unit 31B is a functional unit that determines whether the user intends to use the vehicle based on the detection result by the shoulder movement detection unit 31A. The target door estimation unit 31C is a functional unit that estimates the target door, which is the door expected to be opened by the user, among each door of the vehicle. The control unit 31D is a functional unit that executes control to open the door for the target door estimated by the target door estimation unit 31C when the usage intention determination unit 31B determines that the user intends to use the vehicle. The obstacle detection unit 31E is a functional unit that detects obstacles existing around the vehicle. The second target door estimation unit 31F is a functional unit that estimates the target door based on at least one of the position where the moving speed of the user approaching the vehicle has decelerated, the position where the user is expected to stop moving, and the position where the user has stopped moving. That is, the vehicle control ECU 10 is an example of the shoulder movement detection unit, the usage intention determination unit, the target door estimation unit, the control unit, the obstacle detection unit, and the second target door estimation unit in this specification.

[0018] Note that the configuration of the vehicle door control device 1 shown in Figure 2 is an example and can be changed as appropriate. The vehicle door control device 1 may also have the configuration shown in Figure 8. The position detection unit 31G shown in Figure 8 is a functional unit that detects the position of the portable device 41 held by the user. The movement speed detection unit 31H is a functional unit that detects the movement speed of a user present around the vehicle. The orientation detection unit 31I is a functional unit that detects the orientation of a user's body present around the vehicle. In other words, the vehicle control ECU 10 may be an example of the shoulder movement detection unit, usage intention determination unit, target door estimation unit, control unit, obstacle detection unit, second target door estimation unit, position detection unit, movement speed detection unit, and orientation detection unit as described herein.

[0019] Furthermore, the user intent determination unit 31B may also be a functional unit that determines whether or not the user intends to use the vehicle based on information other than the detection result by the shoulder movement detection unit 31A. For example, as will be described later, the user intent determination unit 31B may be configured to determine whether or not the user intends to use the vehicle based on a change in the deceleration of the movement speed detected by the movement speed detection unit 31H. Also, the target door estimation unit 31C may be configured to estimate the target door based on a change in the deceleration of the movement speed detected by the movement speed detection unit 31H. In this specification, a change in the deceleration of movement speed is not limited to a change in movement speed from an arbitrary movement speed to a movement speed slower than that speed, but also includes a change in movement speed until the movement speed becomes zero, that is, a change from a state in which the user is moving to a state in which the user is standing still. Furthermore, in this specification, a change in the deceleration of movement speed is not limited to a change in movement speed when the user moves in the direction of approaching the vehicle 2, but may also include a change in movement speed when the user moves parallel to the side of the vehicle 2, or a change in movement speed when the user moves away from the vehicle 2. This embodiment describes a case where a walking user is the target user, and walking speed is used as the speed of movement.

[0020] Furthermore, the flash memory 34 stores shoulder detection information DB35. The shoulder detection information DB35 stores threshold information for determining whether or not the user intends to use vehicle 2 based on the movement of the user's shoulders around the vehicle, threshold information for estimating the target door, and so on. The information in the shoulder detection information DB35 may be set in advance by the vehicle manufacturer, for example, or it may be set after purchasing vehicle 2 according to the user's height, shoulder width, body shape, etc.

[0021] Furthermore, the ECU 10 is connected to the aforementioned door lock devices 15A-15E, door opening / closing devices 16A-16E, various cameras (such as the front camera 5), ​​various sensors 8, and wireless communication device 9 via an in-vehicle network such as CAN. The ECU 10 performs various calculations based on the information input from each camera and each sensor 8 to control the vehicle 2. For example, based on the image data captured by each camera, the ECU 10 displays bird's-eye view images and overhead view images on the vehicle 2's monitor (not shown) to provide driving assistance.

[0022] Furthermore, the ECU 10 drives the door lock devices 15A to 15E and the door opening / closing devices 16A to 16E to control the locking (keying) and opening / closing of each door. The ECU 10 also communicates wirelessly with the portable device 41 via the wireless communication device 9 and performs key authentication, unlocking of each door, opening / closing of doors, etc., in response to buttons operated on the portable device 41. Note that key authentication and door locking may also be performed by a device other than the ECU 10, such as the wireless communication device 9.

[0023] (Regarding the door control processing program) Next, the door control processing program executed by the ECU 10 in the vehicle door control device 1 having the above configuration will be described with reference to Figure 3. Figure 3 is a flowchart of the door control processing program according to the first embodiment. Here, for example, when the ECU 10 detects that a user holding a portable device 41 has approached the vehicle 2 at a predetermined distance (for example, several meters) from the vehicle 2, with the engine of the vehicle 2 stopped and all doors locked, it starts the door control processing program. The door control processing program determines whether the user intends to use the vehicle 2 based on the user's shoulder movements, etc., and if it is determined that the user intends to use the vehicle, it is a program that executes control to open the door that the user is expected to open (hereinafter referred to as the target door). In the following description, the case in which control is executed to unlock and open the target door will be described as control to open the door.

[0024] In this specification, "control to open a door" may refer to either control to unlock each door or control to open each door. Therefore, in step S9 of Figure 3, which will be described later, the ECU 10 may execute only the control to unlock the target door. Furthermore, the conditions for starting the execution of the door control program are not limited to the conditions described above. The ECU 10 may execute the process in Figure 3 when the engine of the vehicle 2 is running and the portable device 41 is inside the vehicle. For example, the ECU 10 may start the process in Figure 3 for a user who does not have the portable device 41 when the engine of the vehicle 2 is running and all the doors of the vehicle 2 are locked, and the ECU 10 detects that a user has approached the vehicle 2 at a predetermined distance. Alternatively, the ECU 10 may start the door control processing program when the engine of the vehicle 2 is stopped and all the doors are locked, and the ECU 10 detects that all the doors have been unlocked by the portable device 41. In this case, the ECU 10 may execute control to open the target door, which is already unlocked, if the user intends to use the vehicle 2. That is, the "control to open the door" in this specification may only be the control to open the target door. Furthermore, the program shown in the flowchart in Figure 3 below is stored in the RAM 32 and ROM 33 of the vehicle door control device 1 and executed by the CPU 31.

[0025] First, in step 1 of Figure 3 (hereinafter abbreviated as S), the CPU 31 acquires information about the user holding the portable device 41 that has approached to a predetermined distance from the vehicle 2. As a method for determining whether or not the user has approached to a predetermined distance from the vehicle 2, for example, a method based on the location information of the portable device 41 can be employed. For example, the location of the portable device 41 (user) may be detected using triangulation based on the distance between the multiple wireless antennas of the wireless communication device 9 and the portable device 41. Alternatively, the user's location may be detected using a millimeter-wave radar or camera mounted on the vehicle 2.

[0026] Furthermore, in S1, the CPU 31 activates each camera and acquires user information necessary for the processing described in S3 to S8. Specifically, the CPU 31 acquires information such as the user's shoulder movement, user's position, user's direction of movement, user's body orientation, and user's movement speed by processing the image data captured by each camera. Note that the method of acquiring user information is not limited to using the image data captured by the cameras. For example, the CPU 31 may detect the user's shoulder movement, etc., based on point cloud data from millimeter-wave radar.

[0027] Therefore, when the position of the portable device 41 detected by the wireless communication device 9, etc., is within a predetermined distance from the vehicle 2, the CPU 31 executes processes to detect the user's shoulder movements and processes to detect changes in walking speed that indicate a decrease in speed. When the user carrying the portable device 41 is within a predetermined distance from the vehicle 2, the CPU 31 executes the process shown in Figure 3 and, based on the shoulder movements, deceleration position, etc., makes a determination (S3, S5, S7) as to whether the user intends to use the vehicle 2, and estimates the target door (S4, S6, S8), as described later. In other words, the CPU 31 does not execute the process shown in Figure 3 when the portable device 41 (user) is outside the predetermined distance from the vehicle 2. This reduces unnecessary battery consumption by suppressing the activation of each camera, etc., in situations where the distance between the vehicle 2 and the user is long and the accuracy of the determination of intent to use the vehicle or the estimation accuracy of the target door is low. Note that the CPU 31 may also detect the user's shoulder movements, deceleration position, etc., even if the position of the portable device 41 is further than a predetermined distance from the vehicle 2. For example, the CPU 31 may start processing to detect the user's shoulder movements, etc., at the moment it detects the portable device 41 via the wireless communication device 9, that is, within the maximum range in which the position of the portable device 41 can be detected.

[0028] Next, the CPU 31 acquires information about obstacles around the vehicle (S2). Methods for acquiring obstacle information include, for example, using a camera or millimeter-wave radar, similar to the method used to acquire user information described above. As will be discussed later, the user's shoulder movement changes depending on the placement of obstacles. Therefore, in S2, the CPU 31 may only detect obstacles around the user. Furthermore, the CPU 31 modifies the judgment conditions in S3 based on the obstacle information acquired in S2. First, the case without obstacles will be explained, followed by the case with obstacles (see Figure 7).

[0029] Next, the CPU 31 executes, for example, the processing of the combination of S3 and S4, the processing of the combination of S5 and S6, and the processing of the combination of S7 and S8 in parallel. In each of the three sets of parallel processing, the CPU 31 determines whether the user intends to use the vehicle 2 (hereinafter referred to as "intent to use") and estimates the target door. First, the processing of S3 and S4 will be explained. In the processing of S3, the CPU 31 determines whether the user intends to use the vehicle based on the change in shoulder movement, and in the processing of S4, it estimates the target door based on the user's position when the shoulder movement changes. In this specification, the intention to use the vehicle is not limited to the intention to open each door and get in, but also includes the intention to open the door and load luggage, the intention to unload luggage, and other intentions when working without getting in by opening each door. For this reason, the doors in this specification include the back door 14.

[0030] In S3, the CPU 31 detects the shoulder movement of a user approaching vehicle 2 based on the image data captured by each camera. Figure 4 shows, as an example, the state of user 43 walking on the right side of vehicle 2. The straight line L1 in Figure 4 is the straight line connecting both shoulders of user 43. The CPU 31 generates an overhead view image of the area around the vehicle, looking vertically downwards from above, based on real-time image data captured by all cameras, such as the front camera 5, side cameras 6R and 6L, and rear camera 7. From the generated overhead view image, the CPU 31 detects the straight line L1 connecting both shoulders of user 43 around the vehicle, the direction of user 43's movement, etc. Note that the image used to detect shoulder movement is not limited to an overhead view image; a bird's-eye view image looking diagonally downwards from above around the vehicle may also be used. The CPU 31 may also detect the straight line L1 using other devices such as millimeter-wave radar.

[0031] The CPU 31 also sets coordinates for determining the shoulder angle. For example, the CPU 31 sets the origin at the position of the door closest to the user 43's current position and sets coordinate axes along the vehicle 2. The example shown in Figure 4 illustrates the movement of user 43 approaching vehicle 2 from position P1. User 43 is walking backward from position P1, which is diagonally in front of and to the right of vehicle 2, along a direction parallel to the right side of vehicle 2. Therefore, when user 43 is detected approaching vehicle 2 to position P1, the front door 12R is the closest door to user 43 among the doors of vehicle 2. For example, the CPU 31 sets the origin O1 at the position of the door handle of the front door 12R, sets the line passing through the origin O1 in the front-to-back direction (along the right side of vehicle 2) as the X-axis, and sets the line perpendicular to the X-axis and passing through the origin O1 in the left-to-right direction as the Y-axis. The CPU 31 also sets positive values ​​for the area behind the origin O1 and negative values ​​for the area in front of the origin O1 on the X-axis. Furthermore, the CPU 31 sets the Y-axis so that the direction to the right of the origin O1 is a positive value and the direction to the left is a negative value. The CPU 31 also detects the angle that the line L1 makes with the X-axis as the shoulder angle θ. The CPU 31 detects the counterclockwise direction with respect to the X-axis as a positive angle and the clockwise direction as a negative angle. The CPU 31 detects the movement of the shoulder from the angle θ.

[0032] Furthermore, the method for detecting shoulder movement is not limited to detecting the angle θ of the straight line L1 connecting both shoulders. For example, the difference between the X and Y axes for the XY coordinates of each shoulder may be calculated, and shoulder movement may be detected from that difference. In addition, the CPU 31 may change the position of the origin O1 if the door closest to the user 43's current position changes as the user 43 moves. For example, as shown in position P2 in Figure 4, if the user 43 moves backward to a position opposite the rear door 13R in the left-right direction, and the door closest to the user 43 becomes the rear door 13R, the origin O2 may be set at the position of the door handle of the rear door 13R and the XY coordinates may be reset. Similarly, if the user 43 moves to the rear of the vehicle 2, and the door closest to the user 43 becomes the back door 14, the origin O3 may be set at the position of the door handle of the back door 14 and the XY coordinates may be reset. In this case, the line passing through the origin O3 and running horizontally (along the back of vehicle 2) may be set as the X-axis, and the line passing through the origin O3 and running vertically may be set as the Y-axis.

[0033] Figure 5 is a graph showing the relationship between the X coordinate (mm) and the shoulder angle θ (deg) when user 43 walks from the front right to the rear of vehicle 2 as shown in Figure 4. In the case shown in Figure 4, the X coordinate of user 43's position changes from negative to positive. As shown by the arrows of the words "No intention to use" in Figures 4 and 5, if user 43 has no intention to use vehicle 2, user 43 will, for example, walk past vehicle 2 without approaching it and towards the rear. User 43 moves forward with their back turned, moving their shoulders back and forth as they move backward. In this case, the shoulder angle θ of user 43 changes within a predetermined range (for example, +100deg to +80deg) and does not change below a predetermined angle (for example, +80 degrees).

[0034] On the other hand, as shown by the arrows at the words "Intention to use" in Figures 4 and 5, if user 43 intends to use vehicle 2, user 43 approaches vehicle 2 to a certain distance from position P1, then turns 90 degrees to the right toward vehicle 2 and walks towards the right front door 12R. In this case, the angle θ of user 43's shoulder changes outside the predetermined angle range described above (for example, +80 degrees to +5 degrees). Also, the angle θ changes to or below the predetermined angle (for example, +80 degrees).

[0035] Thus, the upper and lower limits of the shoulder angle θ, as well as the amount of change, will differ depending on whether user 43 intends to use vehicle 2 or not. Therefore, the CPU 31 compares user 43's shoulder angle θ and the amount of change in the shoulder angle θ with thresholds to determine whether user 43 intends to use vehicle 2. For example, in the case shown in Figure 5, a threshold of +80deg can be set for the angle θ. Also, if there is an intention to use the vehicle, the shoulder angle θ changes from a state where it changes by ±20deg between +100deg and +80deg to a range of +80deg to +5deg. For example, if the amount of change in the angle θ changes by -25deg or more, it can be considered that user 43 has changed their direction of movement to the right. A threshold of -25deg can be set for the amount of change. This threshold for determining the intention to use the vehicle is stored in the shoulder detection information DB 35. For example, the shoulder detection information DB 35 stores thresholds for the direction of movement, each door where the origin is set, etc. The CPU 31 reads the corresponding threshold from the shoulder detection information DB 35 and uses it to determine whether to use it.

[0036] When CPU 31 starts executing S3, it determines that user 43 intends to use the device if, for example, the angle θ becomes +80 degrees or less and the amount of change in angle θ changes negative by -25 degrees or more. Note that the method of determining intention to use from shoulder movement described above is just one example. For example, CPU 31 may determine intention to use by performing only one of the following methods: comparing the shoulder angle θ with a threshold, or comparing the amount of change in shoulder angle θ with a threshold. Also, CPU 31 may determine whether the threshold has been exceeded multiple times, rather than just once. For example, CPU 31 may determine the angle θ at predetermined time intervals and determine intention to use if the determination result of angle θ being +80 degrees or less is repeated multiple times. Also, CPU 31 may determine intention to use if the state of the amount of change in angle θ changing negative continues for a predetermined time. Therefore, shoulder movement may be judged without using a threshold.

[0037] Furthermore, the CPU 31 may determine the user's intention to use the device based on the range of change in angle θ. For example, the CPU 31 may determine that the user intends to use the device if the angle θ remains below 80 degrees for a predetermined period of time. Alternatively, the CPU 31 may determine the user's intention to use the device using the average value of angle θ over a predetermined period of time, or it may determine the user's intention to use the device based on the slope of a straight line approximating the change in angle θ over predetermined periods of time. Therefore, various calculation and comparison methods can be employed to determine the user's intention to use the device, which can detect the change in shoulder angle θ between when the user intends to use the device and when they do not. Similarly, various calculation and comparison methods can be employed in the method of determining the user's intention to use the device based on changes in walking speed, which will be described later.

[0038] Furthermore, the CPU 31 may change the conditions (such as thresholds) used for judgment according to the physical characteristics of the user 43, such as height and shoulder width. The CPU 31 may also accept registration of the walking style of each individual, such as the owner of the vehicle 2 or their family. The CPU 31 may then set thresholds for each individual based on the characteristics of the walking style that have been accepted. For example, the vehicle door control device 1 may have a mode that detects the shoulder angle θ and the amount of change of the user 43 walking around the vehicle and sets thresholds based on the detected information. The CPU 31 may also change thresholds according to the position and type of the door for which the origin has been set, or according to the positional relationship between the door for which the origin has been set and the user 43. In other words, thresholds may be appropriately changed according to the relative positional relationship between the user 43 and the vehicle 2, and according to the characteristics of the user 43 and the vehicle 2.

[0039] Furthermore, in S4, the CPU 31 acquires positional information to determine the location of the shoulder movement on the vehicle 2 and estimates the target door. For example, the CPU 31 estimates the target door from the position of user 43 at the point when the angle θ is 80 degrees or less and the amount of change in angle θ has changed negative by -25 degrees or more. When the angle θ and the amount of change exceed the threshold at position P3, shown by the solid line in Figure 4, user 43 is in a position opposite the front door 12R in the left-right direction. Therefore, it is highly likely that user 43 intends to open the front door 12R.

[0040] On the other hand, for example, if the angle θ and the amount of change exceed the threshold at position P4, indicated by the dashed line in Figure 4, then in the left-right direction, user 43 is in a position facing the rear door 13R. Therefore, it is highly likely that user 43 intends to open the rear door 13R. Accordingly, the CPU 31 estimates the target door based on user 43's position at the time the angle θ and the amount of change exceed the threshold, and the positions of each door.

[0041] Furthermore, the method for estimating the target door is not limited to the method described above. For example, user 43 may change direction of movement at position P3 and then move diagonally towards the rear door 13R. For this reason, the CPU 31 may estimate the target door based on the subsequent direction of movement, in addition to the position at which the angle θ and the amount of change exceed a threshold. Similarly, in the method for estimating the target door based on changes in walking speed that cause deceleration, described later, the CPU 31 may estimate the target door based on the subsequent direction of movement, in addition to the deceleration position, etc. Also, the CPU 31 may estimate the target door based on a position other than that of user 43 at the point when the angle θ is 80 degrees or less and the amount of change in angle θ has changed negative by -25 degrees or more. For example, in S3, the intention to use the door may be judged from the movement of the shoulder, and in S4, the target door may be estimated from a position other than the movement of the shoulder. In other words, the information used as a criterion for judging the intention to use the door and the information used to estimate the target door may be different information. For example, the CPU 31 may detect the orientation of the user 43's body from the line L1 and the direction of movement, and then estimate the target door from the detected body orientation and direction of movement.

[0042] Furthermore, the CPU 31 may execute the processes of S3 and S4 in parallel. For example, in S4, the CPU 31 estimates the target door from the body orientation and direction of movement, and if the estimated target door changes, it may change the origin of the XY coordinate system. Specifically, if the target door estimated in S4 becomes the rear door 13R, the CPU 31 may set the origin to origin O2, and if the target door becomes the back door 14, it may set the origin to origin O3. The CPU 31 may also change the threshold for determining intent to use in accordance with the change in the origin.

[0043] Then, if the CPU 31 determines in S3 that there is an intention to use the door, it estimates the target door in S4, and in S9, it unlocks the door lock of the estimated target door and executes control to open the door. For example, if the target door is the front door 12R, the CPU 31 controls the door lock device 15A to unlock the door lock of the front door 12R, and then controls the door opening / closing device 16A to open the front door 12R. This makes it possible to determine whether or not there is an intention to use the door from the shoulder movements of the user 43 around the vehicle, and if it is determined that there is an intention to use the door, the target door that the user 43 is expected to want to open can be automatically opened.

[0044] (Regarding the control of other doors) The above explanation mainly focused on the front door 12R, but other doors can be controlled similarly. In this case, thresholds may be changed depending on the direction the user walks, the user's position, the door where the origin is set, etc. Figure 6 is a graph showing the relationship between the X coordinate and the shoulder angle θ when user 43 walks from the right rear to the front of vehicle 2. In the case shown in Figure 6, the X coordinate of user 43's position changes from positive to negative, unlike in Figures 4 and 5. As shown in Figure 6, for example, the value of the shoulder angle θ, the value of the change in angle θ, the manner in which each value changes, and the transition of each value are different from those shown in Figure 5. Specifically, as shown in the upper hatching in the graphs of Figures 5 and 6, the range of angle θ change when there is no intention to use is +100deg to +80deg in the case of Figure 5, while it is +100deg to +75deg in the case of Figure 6. For this reason, in the case of Figure 6, +75deg may be set as the threshold for determining the intention to use from the angle θ. Furthermore, CPU31 may determine that it intends to use the system if the angle θ in S3 is +75 degrees or less.

[0045] (Regarding the presence of obstacles) The S3 process described above determined the user's intention to use the vehicle based on the movement of the user's shoulders. However, the path and width of the road the user 43 travels on are altered and restricted by obstacles around the vehicle. When the travel path changes, the user 43's shoulder movements, body orientation, and shoulder angle θ will differ from those in the absence of obstacles. Figure 7 shows the user 43's travel path and shoulder movements when obstacles 51 are present around the vehicle. Obstacles 51 can be, for example, walls, materials placed in a parking lot, other vehicles, bicycles, guardrails, or other structures. Alternatively, obstacles 51 may be moving objects such as pedestrians.

[0046] As shown in Figure 7, for example, if the width of the path approaching vehicle 2 from the front is too narrow for user 43 to walk facing backward, user 43 will move sideways while facing vehicle 2. Therefore, the movement of user 43's shoulders will differ from that of user 43 walking facing backward as shown in Figure 4. Specifically, the range and amount of change of the shoulder angle θ will be smaller than in the case of Figure 4. For example, the range of change of angle θ when user 43 moves sideways along a wall is +30deg to -30deg. Also, for example, if user 43 starts moving towards vehicle 2 after passing through a passage sandwiched between obstacles 51, the range and amount of change of angle θ will be larger. Furthermore, if obstacle 51 is present on the shortest path from user 43's current position to vehicle 2, user 43 will need to go around the obstacle 51 to reach vehicle 2. As a result, the movement of user 43's shoulders will differ from that of user 43 when there is no obstacle 51 as shown in Figure 4. The relationship between such obstacles 51 and the movement of the user's shoulder 43 is verified in advance, and thresholds for determining the angle θ are set in the shoulder detection information DB 35.

[0047] If the CPU 31 detects an obstacle 51 around the vehicle in S2, it changes the decision conditions in S3. That is, if the detected obstacle 51 hinders the user 43's movement, the CPU 31 changes the conditions for determining the user's intention to use the vehicle based on the user 43's movement path restricted by the obstacle 51 (changes to the path itself, changes to the width of the path, etc.). Specifically, for example, the CPU 31 detects information about the user 43's movement path (expected movement (detour) path, width of the movement path, orientation of the user 43 on the movement path, etc.) based on the position and size of the obstacle 51 detected in S2. The CPU 31 searches for corresponding threshold information from the shoulder detection information DB 35 based on the detected movement path information and determines the threshold. This allows the user to determine their intention to use the vehicle based on a threshold corresponding to the changed range of angle θ, etc., when the range of angle θ change or the amount of angle θ change is changed by the obstacle 51. Therefore, the shoulder detection information DB 35 may have thresholds set according to combinations of movement path, path width, orientation, etc.

[0048] The above-mentioned method of changing the conditions for determining intent to use in response to obstacle 51 is just one example. For example, the CPU 31 may adjust the threshold based on the angle θ of the user's shoulder near obstacle 51. The CPU 31 may also adjust the threshold by multiplying the difference between the change in angle θ when there is no obstacle 51 and the change in angle θ near obstacle 51 by a coefficient. Alternatively, the CPU 31 may be configured not to determine intent to use if the position of the user 43 at the time when intent to use is determined based on shoulder movement is the position of obstacle 51. As mentioned above, the shoulder angle θ fluctuates depending on the presence or absence of obstacle 51. For this reason, for example, the CPU 31 does not need to determine intent to use if the user 43 is near obstacle 51, even if the shoulder angle θ is below the threshold. Furthermore, if the CPU 31 detects obstacle 51 around the vehicle in S2, it may change the method or conditions for estimating the target door in S4. The CPU 31 may change the method or conditions for estimating the target door from the user 43's position depending on whether or not there is an obstacle 51 near the user 43.

[0049] (Deceleration position) Next, the processes of S5 and S6 will be explained. In the process of S5, it is determined that the user 43 intends to use the vehicle 2 based on the fact that the walking speed of the user 43 approaching the vehicle 2 has slowed down, and in the process of S6, the target door is estimated based on the position of the user 43 when the walking speed slows down. In S5, the CPU 31 detects the walking speed of the user 43 approaching the vehicle 2 based on the image data captured by each camera (S5). For example, the CPU 31 detects the walking speed of the user 43 in real time, and when the walking speed falls below a predetermined threshold, it determines that the walking speed has slowed down, that is, that the user intends to use the vehicle, and executes S6. Alternatively, the CPU 31 may determine that the walking speed has slowed down (that the user intends to use the vehicle) if the amount of decrease when the walking speed slows down is greater than or equal to a predetermined threshold. Furthermore, the CPU 31 may determine that the user intends to use the vehicle if the walking speed falls below a predetermined threshold AND the amount of decrease when the walking speed slows down is greater than or equal to a predetermined threshold. Furthermore, as described above, the CPU 31 may determine whether the user intends to use the device based on whether the walking speed exceeds a predetermined threshold multiple times, rather than simply determining whether the walking speed exceeds a predetermined threshold once, similar to how the CPU 31 determines whether the user intends to use the device based on shoulder movement. Alternatively, the CPU 31 may determine that the user intends to use the device if the decrease in walking speed exceeds a predetermined threshold for a predetermined period of time.

[0050] Therefore, the CPU 31 may determine whether or not the user 43 intends to use the device based on the change in the user's walking speed. The CPU 31 may determine that the user intends to use the device if, at least one of the following occurs: the walking speed falls below a predetermined threshold, or the decrease in walking speed exceeds a predetermined threshold. This allows the CPU 31 to determine the user's intention to use the device by comparing the walking speed and the decrease in walking speed with the threshold.

[0051] Furthermore, if the position where user 43 slows down is a predetermined distance away from vehicle 2, CPU 31 does not need to determine that the walking speed has slowed down even if the walking speed falls below a predetermined threshold. In other words, if the distance between vehicle 2 and user 43 is large and it is unclear whether user 43 intends to use the vehicle, CPU 31 does not need to determine that user 43 intends to use the vehicle. Also, if the position where user 43 slows down is the position of obstacle 51 detected in S2, CPU 31 does not need to determine that the walking speed has slowed down even if the walking speed falls below a predetermined threshold. Also, if the position where user 43 slows down is the position of obstacle 51 detected in S2, CPU 31 does not need to determine that the walking speed has slowed down even if the decrease in walking speed exceeds a predetermined threshold. In other words, if the walking speed slows down due to the influence of obstacle 51, there is a possibility that user 43 does not intend to use the vehicle, so CPU 31 does not need to determine that user 43 intends to use the vehicle.

[0052] Therefore, the CPU 31 may be configured not to determine that the user intends to use the device if the user's position at the time a change in walking speed is detected coincides with the position of an obstacle 51 detected by the obstacle detection unit 31E. This prevents the user's intention to use the device from being incorrectly determined due to an obstacle 51.

[0053] In S6, the CPU 31 acquires positional information to determine at what point in the vehicle 2 the walking speed slowed down, and estimates the target door. For example, the CPU 31 estimates the target door from the position of the user 43 at the point when the walking speed falls below a predetermined threshold. Alternatively, the CPU 31 may estimate the target door from the position of the user 43 at the point when the amount of decrease in walking speed exceeds a predetermined threshold. Or, the CPU 31 may estimate the target door from the position of the user 43 at the point when the walking speed falls below a predetermined threshold AND the amount of decrease in walking speed exceeds a predetermined threshold.

[0054] Therefore, the CPU 31 may determine that a deceleration of walking speed has occurred in at least one of the following cases: when the walking speed falls below a predetermined threshold, or when the amount of decrease in walking speed exceeds a predetermined threshold. Based on the position of the user 43 when the deceleration of walking speed is detected, the CPU 31 may then estimate the target door. The user 43 may move closer to the door they want to open and then decelerate. For this reason, the CPU 31 may, for example, estimate the door closest to the position where the user decelerated to below a predetermined threshold as the target door. Alternatively, the CPU 31 may estimate the door closest to the position where the amount of decrease exceeds a predetermined threshold as the target door. In this way, the target door can be estimated by comparing the walking speed and the amount of decrease with the threshold.

[0055] Furthermore, the method for estimating the target door in S6 is not limited to the method described above. For example, the CPU 31 may detect the position of the obstacle 51 and the orientation of the user 43 from the image data, in addition to the deceleration position, and predict the user 43's movement path after deceleration based on the detection results. The CPU 31 may then estimate the door located at the destination of the predicted movement path as the target door.

[0056] Therefore, the CPU 31 may detect the orientation of the user 43's body in the vicinity of the vehicle 2 using the orientation detection unit 31I. The CPU 31 may also estimate the target door based on the change in walking speed and the orientation of the user 43's body. This improves the accuracy of the target door estimation by estimating based on both the change in walking speed and the orientation of the user 43's body. In this specification, the orientation of the body is not limited to the orientation of the user 43's shoulders, but may also include the orientation of the head, chest, feet, etc. Therefore, in this specification, the orientation of the body can be determined from the orientation of various parts of the human body that can determine the orientation of the user 43's body. The orientation of the body can be detected using each camera and sensor 8.

[0057] For example, the CPU 31 may set a predicted movement path in the direction in front of the user 43 based on the orientation of the user 43's body at a position where the user has decelerated to below a predetermined threshold, and estimate the door located at the destination of the predicted movement path as the target door. Alternatively, for example, the CPU 31 may estimate the target door based on the deceleration position and the orientation of the shoulders. When the walking speed of the user 43 is reduced, the CPU 31 may estimate the door at the position opposite the user 43's current position in the left-right direction as the target door if the angle θ or the amount of change of the user 43's shoulders exceeds a threshold.

[0058] Furthermore, the CPU 31 may estimate the target door based on the predicted stopping position or the actual stopping position, as described later, and the orientation of the user's body. For example, when the angle θ of the user's shoulders exceeds a threshold at the position where the user 43 has stopped, the CPU 31 may estimate the door at the position opposite the user 43's position at that time as the target door in the left-right direction.

[0059] If the CPU 31 detects deceleration in S5 and determines that there is an intention to use the door, it unlocks the door lock of the estimated target door in S6 and executes control to open the door (S9). In this way, similar to the movement of the user's shoulders, it is possible to determine whether or not there is an intention to use the door from the deceleration of walking speed, and if there is an intention to use the door, the target door that the user 43 is expected to want to open can be automatically opened.

[0060] Therefore, when the CPU 31 determines in S5 that there is an intention to use the door, it executes control to open the estimated target door in S6. This allows the target door to be opened quickly after detecting the intention to use the door. The target door can be opened before the user 43 arrives at the target door. Alternatively, the CPU 31 may be configured to determine in S5 that there is an intention to use the door, wait for a certain period of time to elapse, then estimate the target door in S6, and execute control to open the estimated target door.

[0061] (Expected stopping position, and actual stopping position) Next, the processes of S7 and S8 will be explained. In the process of S7, it is determined that the user 43 intends to use the vehicle based on the detection that the user 43 approaching the vehicle 2 is about to stop or has stopped. In the process of S6, the target door is estimated based on the position where the user is about to stop or has stopped. In S7, the CPU 31 detects the walking speed of the user 43 approaching the vehicle 2 based on the image data captured by each camera. For example, the CPU 31 detects the walking speed of the user 43 in real time, and when the detected walking speed falls below a predetermined threshold, it is determined that the user 43 is about to stop, that is, that the user intends to use the vehicle, and executes S8. This threshold for determining whether or not the user is about to stop is a smaller value than the threshold for determining deceleration in S5, and is the speed at which a person is likely to stop walking. Alternatively, the CPU 31 may determine that the user 43 is about to stop if the state of walking speed being below a predetermined threshold continues for a predetermined threshold time or longer.

[0062] Furthermore, the CPU 31 determines that user 43 is in a stopped state and executes S8 if user 43 stops and their walking speed remains at zero for a predetermined time, or if user 43's position remains within a predetermined range for a predetermined time. Alternatively, the CPU 31 may determine that user 43 is in a stopped state and execute S8 if user 43 stops at a position where it was expected that user 43 would stop and their walking speed remains at zero for a predetermined time.

[0063] Therefore, the CPU 31 may detect at least one of the following positions based on the change in walking speed: the position where the user 43 is expected to stop walking, and the position where the user 43 has stopped moving (S7). The CPU 31 may determine that the user intends to use the device if the state of zero walking speed continues for a predetermined time at at least one of these positions. This makes it possible to determine the user's intention to use the device based on the condition that the user stops for a predetermined time at the expected stopping position or the stopping position.

[0064] Furthermore, similar to the deceleration position in S5, if the position where the user is about to stop or has stopped is a predetermined distance from the vehicle 2, the CPU 31 may determine that there is no intention to use the vehicle even if it detects that the user is about to stop or has stopped. Also, if the position where the user is about to stop or has stopped is the location of the obstacle 51 detected in S2, the CPU 31 may determine that there is no intention to use the vehicle even if it detects that the user is about to stop or has stopped. In other words, if the user 43 stops due to the influence of the obstacle 51, there is a possibility that the user 43 does not intend to use the vehicle, so it is not necessary to determine that there is an intention to use the vehicle. This prevents the user 43's intention to use the vehicle from being incorrectly judged due to the obstacle 51, similar to the deceleration position described above.

[0065] In S8, the CPU 31 acquires positional information to determine where the vehicle 2 is likely to stop or has stopped, and estimates the target door. For example, the CPU 31 estimates the target door from the position of the user 43 when the walking speed falls below a predetermined threshold (a threshold for determining whether the user is likely to stop). Alternatively, the CPU 31 estimates the target door from the position of the user 43 when the walking speed remains at zero for a predetermined time. Furthermore, for example, the CPU 31 estimates the target door from the position where the user 43 stopped when it was predicted that the user 43 would stop, and the walking speed remained at zero for a predetermined time. The user 43 may move close to the door they want to open and then stop. Therefore, the CPU 31 estimates the door closest to the position where the user decelerated to below a predetermined threshold, or the position where the walking speed remained at zero for a predetermined time, as the target door.

[0066] Therefore, the CPU 31 estimates the target door based on at least one of the positions where the user 43 is expected to stop walking and the position where the user 43 has stopped walking, if the walking speed remains zero for a predetermined period of time at at least one of these positions (S8). This makes it possible to estimate the target door on the condition that the user stops at the expected stopping position or the stopping position for a predetermined period of time.

[0067] Furthermore, the method for estimating the target door in S8 is not limited to the method described above. For example, the CPU 31 may detect the position where walking speed slows down, the amount of decrease in walking speed, the position of the obstacle 51, the orientation of the user 43's body, etc., from the imaging data, and predict the position where the user 43 will stop based on these detection results. The CPU 31 may also estimate the door closest to the predicted stopping position as the target door. Therefore, the CPU 31 may estimate the target door based on the predicted stopping position, the stopping position, the orientation of the user 43's body, etc.

[0068] Furthermore, when estimating the target door based on the deceleration position in S6, or the target door based on the stopping position in S8, the CPU 31 may also use information such as the user 43's gaze and face orientation in addition to the deceleration position and stopping position to estimate the target door. This allows for even more accurate identification of the target door.

[0069] In S7, the CPU 31 detects a state in which the user is about to stop, and if it determines that the user intends to use the door, it unlocks the door lock of the target door estimated in S8 and executes control to open the door (S9). In this way, similar to the movement of the user's shoulders, it is possible to determine whether the user intends to use the door based on the detection of a state in which the user is about to stop or has stopped, and automatically open the target door that the user is expected to want to open.

[0070] The CPU 31 executes three sets of processes in parallel, for example, the combination of S3 and S4, the combination of S5 and S6, and the combination of S7 and S8. Based on the process that first determines that the user intends to use the door, it executes control to open the target door in S9. This allows the CPU 31 to estimate and open the target door based on the deceleration position or the position where the user is likely to stop, even if it is difficult to detect the movement or angle θ of the user's shoulders.

[0071] Furthermore, the CPU 31 may make a comprehensive decision based on the processing results of at least two of the three sets of processing to control the door. For example, the CPU 31 may execute control to open the target door when the angle θ of the user 43's shoulder falls below a threshold (S3) and the walking speed falls below a predetermined threshold (S5). In this case, the CPU 31 may execute control to open the target door only if the target door estimated in S4 matches the target door estimated in S6 (see Figure 11). Alternatively, the CPU 31 may execute control to open the target door when the angle θ of the user 43's shoulder falls below a threshold (S3) and the walking speed remains zero for a predetermined time (S7). In this case, the CPU 31 may execute control to open the target door only if the target door estimated in S4 matches the target door estimated in S8.

[0072] Therefore, the CPU 31 may open the target door if it determines that there is an intention to use it based on at least one of the three sets of processing, or it may open the target door after comprehensively considering the results of all three sets of processing. The CPU 31 may also execute control to open the door if the target door estimated based on the change in walking speed slowing down matches the target door estimated based on the orientation of the user's body (such as the direction of the shoulders). This prevents the CPU 31 from executing control to open a door that the user did not intend to open.

[0073] Furthermore, the CPU 31 may be configured to execute only one or two of the three processes S3, S5, and S7 shown in Figure 3 to determine whether or not there is an intention to use the door. Alternatively, the CPU 31 may be configured to execute only one or two of the three processes S4, S6, and S8 to estimate the target door. Figure 9 shows a flowchart of another example of a door control processing program. As shown in Figure 9, the CPU 31 may be configured to execute the combination of S5 and S6 and the combination of S7 and S8 in parallel. Therefore, in the door control processing program shown in Figure 3 above, the CPU 31 executes processes S3 and S4 and determines the intention to use the door and the target door based on the movement of the user's shoulders. In contrast, in the door control processing program shown in Figure 9, the CPU 31 determines the intention to use the door and the target door based on the change in the user's movement speed, which slows down, without determining the movement of the shoulders.

[0074] In the door control processing program shown in Figure 9, the CPU 31 determines the user's intention to use the door based on the deceleration position in S5 and the predicted stopping position or actual stopping position in S7, as these represent changes in the user's movement speed. Similarly, the CPU 31 estimates the target door based on the deceleration position in S6 and the predicted stopping position or actual stopping position in S8.

[0075] Furthermore, CPU 31 may be configured to execute, for example, the combination of S3 and S4 and the combination of S5 and S6 shown in Figure 3 in parallel. Alternatively, CPU 31 may be configured to execute the combination of S3 and S4 and the combination of S7 and S8 in parallel. Also, CPU 31 may be configured to execute only the combination of S3 and S4, only the combination of S5 and S6, or only the combination of S7 and S8.

[0076] Furthermore, each combination may be swapped. For example, the CPU 31 may be configured to determine the intention to use the door based on the movement of the shoulder (S3) and to estimate the target door based on the deceleration position (S6). Similarly, the CPU 31 may be configured to determine the intention to use the door based on the deceleration position (S5) and to estimate the target door based on the predicted stopping position or the actual stopping position (S8).

[0077] Furthermore, the number of processes for determining the intention to use the door and the number of processes for estimating the target door do not have to be the same. For example, the CPU 31 may be configured to determine the intention to use the door based on the shoulder movement (S3) and the deceleration position (S5), and to estimate the target door based on the predicted stopping position and the actual stopping position (S8). Alternatively, the CPU 31 may be configured to determine the intention to use the door based on the shoulder movement (S3), and to estimate the target door based on the shoulder movement (S4) and the deceleration position (S6).

[0078] Furthermore, in a configuration that executes processing for two or more combinations, if the CPU 31 determines that the user intends to use the door in all combinations, it may execute control to open the door for the target door estimated in the last combination in which the user intended to use the door was determined. For example, the CPU 31 may execute the combination of S5 and S6 and the combination of S7 and S8 shown in Figure 9 in parallel. In this configuration, if the CPU 31 determines that the user intends to use the door in S5 first, and then determines that the user intends to use the door in S7, it may execute control to open the door for the target door estimated in S8, which is the same combination as S7. In other words, even if the target doors estimated in the S6 and S8 processes do not match, the target door estimated in the later executed process may be used as the control target. This allows the target door to be estimated based on the user's more recent state, improving the accuracy of target door estimation.

[0079] Furthermore, in configurations that combine two or more processes for determining intent to use (S3, S5, S7) and estimating the target door (S4, S6, S8), it is possible to set which process's result takes priority. For example, in the combination of S3 and S7, if both processes determine intent to use, the target door may be estimated in S8 based on the position detected in S7. In other words, for target door estimation, S7 and S8 may take priority over S3 and S4.

[0080] (Effects of the first embodiment) As described in detail above, the first embodiment provides the following effects. (1) According to the vehicle door control device 1 and the computer program executed by the vehicle door control device 1 according to the first embodiment, the CPU 31 of the ECU 10 detects the shoulder movements of the user 43 who is in the vicinity of the vehicle 2 (S1, S3), and based on the detection results, determines whether or not the user 43 intends to use the vehicle 2 (S3). The CPU 31 estimates which of the doors of the vehicle 2 the user 43 is expected to open (S4). If the CPU 31 determines that the user intends to use the vehicle, it executes control to unlock and open the target door estimated in S4 (S9).

[0081] According to this, the CPU 31 can determine the user's intention to use the vehicle from the movement of the user's shoulders, and if the user intends to use the vehicle, it can open the estimated target door, that is, the door that the user is expected to open. When the user 43 wants to use the vehicle 2, simply by the user 43 approaching the vehicle 2, the door that the user 43 wants opened can be automatically opened. Furthermore, even when facial recognition is difficult due to a mask or other reasons, the CPU 31 can determine the user's intention to use the vehicle from the movement of the user's shoulders and open the door.

[0082] (2) In addition, in S2, the CPU 31 detects obstacles 51 present around the vehicle. If the detected obstacles 51 hinder the movement of the user 43, the CPU 31 changes the conditions for determining whether or not the user intends to use the vehicle, based on the movement path of the user 43 that is restricted by the obstacles 51. According to this, as shown in Figures 4 and 7, when the orientation of the user's body, the direction of shoulder movement, the magnitude of the angle θ, the movement path, the width of the path, etc., change due to the obstacle 51, thresholds and the like can be set as judgment conditions corresponding to the changes. If the intention to use is judged uniformly using the same judgment conditions regardless of the presence or absence of the obstacle 51, the thresholds and the like may not be appropriate, and the accuracy of the judgment may decrease. For this reason, when the movement path is restricted by the obstacle 51, the accuracy of judging the intention to use can be improved by changing the judgment conditions according to the situation of the obstacle 51 and the movement path.

[0083] (3) The CPU 31 also detects the angle θ of the user's shoulder with respect to the X-axis along the side of the vehicle 2. The CPU 31 compares the detected shoulder angle θ and the amount of change in the shoulder angle θ with a threshold value to determine whether or not the user intends to use the vehicle. According to this, it is possible to determine whether the user intends to use the vehicle based on the shoulder angle θ and the amount of change in that angle θ. After the user 43 approaches the vehicle 2, the movement of changing direction towards the vehicle 2 can be detected from the angle θ, and the door that the user 43 wants to open can be opened in advance.

[0084] (4) The CPU 31 also estimates the target door based on the position where the user 43's movement speed slows down as it approaches the vehicle 2 (S6), the position where the user 43 is expected to stop moving (S8), and the position where the user 43 has stopped moving (S8). If it is difficult to detect the movement of the user 43's shoulders, the CPU 31 executes control to open the target door estimated in S6 or S8 (S9). According to this, for example, if it is raining and user 43 is holding an umbrella, making it difficult to detect the shoulder angle θ etc. from the image data, or if the area around the vehicle is dark and it is difficult to detect the shoulder angle θ etc. from the image data, the target door can be estimated based on user 43's deceleration position or stopping position, and the estimated target door can be opened. If it is difficult to detect shoulder movement, estimating the target door by another means can prevent the target door from failing to open.

[0085] (5) The CPU 31 also detects the movement of the user's shoulder when the position of the portable device 41 detected by the wireless communication device 9 is within a predetermined range from the vehicle 2. This reduces unnecessary battery consumption by not activating the cameras, etc., until the user 43 approaches within the predetermined range.

[0086] (6) Furthermore, according to the vehicle door control device 1 and the computer program executed by the vehicle door control device 1 according to the first embodiment, the CPU 31 of the ECU 10 detects the walking speed of the user 43 present around the vehicle 2 and determines the user's intention to use the door (S5, S7). The CPU 31 estimates the target door based on the change in walking speed that causes it to decelerate (S6, S8). If the CPU 31 determines that the user intends to use the door, it executes control to unlock and open the target door estimated in S6, S8 (S9).

[0087] According to this, the CPU 31 can open a target door estimated from the change in the user 43's movement speed, i.e., the door that user 43 is expected to open, if user 43 intends to use it. If user 43 wants to use vehicle 2, simply by approaching vehicle 2 and slowing down, the door that user 43 wants opened can be automatically opened. Furthermore, even if face recognition is difficult due to a mask or other reasons, the CPU 31 can determine the user's intention to use the vehicle by the user 43's deceleration and open the door.

[0088] (7) The CPU 31 also detects at least one of the positions where the user 43 is expected to stop moving and the position where the user 43 has stopped moving (S8), and estimates the target door based on the detected position. This allows the target door to be estimated based on the expected stopping position and the actual stopping position. The CPU can detect the action of the user 43 stopping to open a door, estimate the target door, and open it.

[0089] (8) The CPU 31 also detects the position of the user 43 when a deceleration of walking speed is detected (S6), and estimates the target door based on the detected position. This allows the target door to be estimated based on the deceleration position. The CPU 31 can detect the user 43's deceleration action to open a door, estimate the target door, and open it.

[0090] (Second Embodiment) Next, a second embodiment of the present invention will be described. In the first embodiment described above, the CPU 31 executed the processes S3 and S4, S5 and S6, and S7 and S8 shown in Figure 3 in parallel. In contrast, the CPU 31 of the second embodiment differs from the first embodiment in that it executes the processes S3 to S8 sequentially, as shown in Figure 10. In the following description of the second embodiment, components similar to those in the first embodiment will be described using the same reference numerals.

[0091] As shown in Figure 10, the CPU 31, similar to the first embodiment, executes S1 to S3 and, as a result of executing S3, determines whether or not it can be determined from the shoulder angle θ and the amount of change whether or not there is an intention to use the door (S11). If the CPU 31 determines that there is an intention to use the door (S11: YES), it estimates the target door from the position where the shoulder was moved in S4 and opens the target door in S9 (S9).

[0092] On the other hand, if the CPU 31 determines that there is no intention to use the device (S11: NO), it executes S5. Therefore, the CPU 31 determines the intention to use the device from the shoulder angle θ, etc., at the time of executing S3, and if it cannot determine that there is an intention to use the device from the shoulder movement, it executes S5 and determines the intention to use the device from the deceleration position. If, as a result of executing S5, the CPU 31 detects, for example, a deceleration of walking speed (S12: YES), it opens the target door based on the deceleration position (S6, S9).

[0093] Similarly, if CPU 31 does not detect deceleration at the time of executing S5 and determines that there is no intention to use the door (S12: NO), it executes S7 and determines the intention to use the door based on the state in which the person is about to stop, etc. If CPU 31 detects a state in which the person is about to stop or has stopped at the time of executing S7 (S13: YES), it opens the target door estimated from the position where that state was detected (S8, S9). If CPU 31 does not detect a state in which the person is about to stop or has stopped at the time of executing S7 (S13: NO), it executes the process from S3 again to determine the movement of the shoulder. In this way, CPU 31 may execute the processes from S3 to S8 in order.

[0094] Furthermore, the CPU 31 may be configured to execute only one or two of the three processes S3, S5, and S7 shown in Figure 10 to determine whether or not there is an intention to use the door. Alternatively, the CPU 31 may be configured to estimate the target door by executing only one or two of the three processes S4, S6, and S8 shown in Figure 10. Figure 11 shows a flowchart of another example of a door control processing program. As shown in Figure 11, the CPU 31 may be configured to execute the combination of S5, S12, and S6 and the combination of S7, S13, and S8 in sequence. Therefore, even in a configuration that processes in the order (serial) as shown in Figure 10, the CPU 31 may determine the intention to use the door and the target door based on the change in the user's movement speed, rather than judging the movement of the shoulder.

[0095] Furthermore, CPU 31 may be configured to execute, for example, the combination of S3, S11, S4 and the combination of S5, S12, S6 shown in Figure 10 in sequence. Alternatively, CPU 31 may be configured to execute the combination of S3, S11, S4 and the combination of S7, S13, S8 in sequence. The combinations may also be swapped.

[0096] As shown in Figure 11, the CPU 31, similar to the first embodiment, executes S1 and S2, then executes S5 to determine the intention to use the door based on the deceleration position. If, as a result of executing S5, the CPU 31 detects, for example, a deceleration of walking speed (S12: YES), it estimates the target door based on the deceleration position (S6).

[0097] If CPU31 does not detect deceleration at the time of executing S5 and determines that there is no intention to use the door (S12: NO), it executes S7 and determines the intention to use the door based on the state in which the vehicle is about to stop, etc. If CPU31 detects a state in which the vehicle is about to stop or has stopped at the time of executing S7 (S13: YES), it estimates the target door from the position where that state was detected (S8).

[0098] Furthermore, the CPU 31 may determine whether the target door estimated in S6 or S8 matches the target door estimated by another method. For example, after executing S6 or S8, the CPU 31 may execute S4 and estimate the target door based on the position where the shoulder angle θ and the amount of change exceed a threshold. Then, if the target door estimated in S6 or S8 matches the target door estimated in S4 (S15: YES), the CPU 31 may execute control to open the matching target door (S16). This ensures that the target door can only be opened when the two estimation methods match, allowing the user 43 to open the door they want with greater accuracy. If the target doors estimated by the two estimation methods do not match (S15: NO), the CPU 31 repeats the process from S5 to determine the deceleration position.

[0099] Furthermore, in the process shown in Figure 11, the CPU 31 may not execute S4 or S15, and may not determine whether the target door estimated by the two estimation methods matches. For example, the CPU 31 may execute S6 or S8, and then execute S9 in Figure 10.

[0100] It should be noted that the present invention is not limited to the embodiments described above, and various improvements and modifications are possible without departing from the spirit of the invention. For example, in each of the above embodiments, the CPU 31 determined that the user intended to use the vehicle when the shoulder angle θ or the amount of change in that angle θ changed by more than a threshold. However, the method for determining the angle θ and the amount of change is not limited to this method. For example, when a user 43 walks in a straight line towards the vehicle 2, it is possible that the angle θ and the amount of change will be below a predetermined value as the user approaches. In such a case, if the CPU 31 detects that the angle θ and the amount of change are below a predetermined threshold and that the user 43 is approaching the vehicle 2 in a straight line, it may estimate the door at the end of that straight path as the target door and execute control to open it. Furthermore, CPU31 does not need to determine whether it is intended to be used in S5 or S7. Furthermore, the CPU 31 estimated the target door based on the position where the user 43's movement speed slowed down as it approached the vehicle 2 (S6), the position where the user 43 was expected to stop moving (S8), and the position where the user 43 had stopped moving (S8). However, the CPU 31 may estimate the target door based on at least one of the three processes. Furthermore, although the above embodiments performed processing on a walking user 43, the system is not limited to this. For example, even if a user 43 is in a wheelchair, the system may determine the user's intention to use the door and estimate the target door based on the user's shoulder movement, angle θ, amount of change, speed of movement, and likely stopping position, and then control the target door. Therefore, movement and speed of movement in this specification are not limited to walking and walking speed, but may also refer to movement in a wheelchair and the speed of movement of the wheelchair.

[0101] Furthermore, the CPU 31 may change the decision condition for at least one of the processes S3 to S8 depending on the information of the obstacle 51 detected in S2. Furthermore, in S3, CPU31 compared the detected shoulder angle θ and the amount of change in the shoulder angle θ with a threshold value to determine whether to use it; however, it may also compare either one of them with a threshold value to determine whether to use it. Furthermore, if the vehicle control ECU 10 is configured to perform a determination of intent to use and estimate the target door based on a change in the speed of movement, it may not be configured to include the shoulder detection information DB 35. Furthermore, in each of the above embodiments, the ECU 10 of the vehicle door control device 1 executes the processing of the door control processing program (Figures 3, 9, 10, and 11), but the execution entity can be changed as appropriate. For example, the processing in Figure 3 may be executed by the control unit of the navigation device or other in-vehicle devices. Furthermore, if it is difficult to open each door, such as when another vehicle is parked adjacent to vehicle 2, the CPU 31 may stop the control to open the target door or reduce the amount it is opened. The configuration of the vehicle door control device 1 is not limited to the configurations of the embodiments described above. For example, the vehicle door control device 1 may be configured to include only the ECU 10, or to include only the ECU 10 and each camera. [Explanation of Symbols]

[0102] 1 Vehicle door control device, 2 Vehicle, 10 Vehicle control ECU (shoulder motion detection unit, intention to use unit, target door estimation unit, control unit, obstacle detection unit, second target door estimation unit, position detection unit, movement speed detection unit, orientation detection unit), 12R, 12L Front door (door), 13R, 13L Rear door (door), 14 Back door (door), 31A Shoulder motion detection unit, 31B Intention to use unit, 31C Target door estimation unit, 31D Control unit, 31E Obstacle detection unit, 31F Second target door estimation unit, 43 User, 51 Obstacle, θ Angle.

Claims

1. A shoulder motion detection unit that detects the movement of the user's shoulders in the vicinity of the vehicle, A unit for determining whether the user intends to use the vehicle, based on the detection results from the shoulder movement detection unit, A target door estimation unit estimates the target door among the doors of the vehicle that is expected to be opened by the user, When the usage intention determination unit determines that the user intends to use the vehicle, the control unit executes control to open the target door estimated by the target door estimation unit. A vehicle door control device equipped with the following features.

2. The vehicle further includes an obstacle detection unit for detecting obstacles present in the vicinity of the vehicle, The aforementioned unit for determining intent to use, The vehicle door control device according to claim 1, wherein if the obstacle detected by the obstacle detection unit becomes an obstruction to the user's movement, the conditions for determining whether the user intends to use the vehicle are changed based on the user's movement path restricted by the obstacle.

3. The shoulder motion detection unit is The angle of the user's shoulder relative to the vehicle is detected, The aforementioned unit for determining intent to use, A vehicle door control device according to claim 1 or 2, wherein the device compares at least one of the angle of the user's shoulder and the amount of change in the shoulder angle detected by the shoulder motion detection unit with a threshold value to determine whether or not the user intends to use the vehicle.

4. The system further includes a second target door estimation unit that estimates the target door based on at least one of the following positions: the position where the user's movement speed slows down as they approach the vehicle, the position where the user is expected to stop moving, and the position where the user has stopped moving. The control unit, If the shoulder motion detection unit has difficulty detecting the user's shoulder movement, the vehicle door control device according to claim 1 or 2 further comprises, in which the second target door estimation unit performs control to open the target door estimated by the second target door estimation unit.

5. The system further includes a location detection unit that detects the location of the portable device held by the user, The shoulder motion detection unit is The vehicle door control device according to claim 1 or 2, wherein the position of the portable device detected by the position detection unit is within a predetermined range from the vehicle, and the movement of the user's shoulder is detected.

Images