Control system, control method, and recording medium

Abstract

This invention provides a control system, a control method, and a recording medium. It improves the convenience for vehicle users in controlling vehicle doors. The control system includes: an exterior image acquisition unit (44) that acquires exterior images; a first detection unit (204) that detects people outside the vehicle based on the exterior images acquired by the exterior image acquisition unit; a second detection unit (205) that detects an approach method for the people outside the vehicle detected by the first detection unit; a comparison unit (206) that compares the approach method detected by the second detection unit (205) with a reference approach method; an authentication unit (208) that authenticates whether the people outside the vehicle detected by the first detection unit are registered users; and a door control unit (102) that, if the authentication unit authenticates that the person is a registered user, controls the vehicle door based on the comparison result of the comparison unit.

Description

[0001] This application is a divisional application of the invention patent application filed on February 8, 2023, with application number 202310142745.0 and entitled "Control System and Control Method of Control System". Technical Field

[0002] This invention relates to control systems and control methods for control systems. Background Technology

[0003] In recent years, efforts have intensified to provide sustainable transportation systems that take into account vulnerable groups such as the elderly and children among transportation participants. To achieve this, research and development are focused on further improving the safety and convenience of transportation through advancements related to vehicle boarding and alighting. Furthermore, technologies related to vehicle door control have long been known as techniques for improving convenience (see, for example, Patent Document 1). Patent Document 1 discloses a technique that non-contactly detects the movements of a human body within a defined area of ​​the vehicle, determines the door's movement based on the detected movements, and controls the door drive unit accordingly.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2013-28903 Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] However, when a user enters a vehicle, the way they approach the vehicle (hereinafter referred to as the "approach method") may not be the same each time, even for the same user. Therefore, by designing the vehicle door control to take into account the user's approach method, the convenience of the vehicle user is improved. However, in the past, vehicle door control structures did not take into account the user's approach method, leaving room for improvement in terms of convenience.

[0009] The present invention was made in view of the above circumstances, and its object is to improve the convenience of vehicle users in the control of vehicle doors.

[0010] Methods for solving problems

[0011] One aspect of the present invention is a vehicle control system comprising: an exterior image acquisition unit that acquires an exterior image of the vehicle; a first detection unit that detects persons outside the vehicle based on the exterior image acquired by the exterior image acquisition unit; a second detection unit that detects a manner of approaching the vehicle, i.e., an approach method, for the person outside the vehicle detected by the first detection unit; a comparison unit that compares the approach method detected by the second detection unit with the reference approach method; an authentication unit that authenticates whether the person outside the vehicle detected by the first detection unit is a pre-registered user of the vehicle; and a door control unit that, if the authentication unit authenticates the person as the registered user, controls the doors of the vehicle based on the comparison result of the comparison unit.

[0012] Invention Effects

[0013] According to one aspect of the present invention, the convenience of vehicle users can be improved in the control of vehicle doors. Attached Figure Description

[0014] Figure 1 This is a diagram showing the structure of the control system.

[0015] Figure 2 This is a diagram showing the structure of the vehicle.

[0016] Figure 3 This is a diagram showing the main structural components of the control system.

[0017] Figure 4 This is a flowchart illustrating the actions entering the ECU and the B-ECU.

[0018] Figure 5 This is a diagram showing the main structural components of the control system according to the second embodiment.

[0019] Figure 6 This is a flowchart illustrating the actions entering the ECU and the B-ECU.

[0020] Explanation of reference numerals in the attached figures

[0021] 1…Control system, 2…Central ECU, 3a…First communication line, 4b…Second communication line, 5…In-vehicle connection link, 27…Door lock mechanism, 23…Area B-ECU, 33A~33E…Door lock mechanism, 30…Entry ECU, 32…Front camera, 33…Rear camera, 34…Right-side camera, 35…Left-side camera, 41…Drive unit, 41A…Right PHD drive unit, 41B…Left PHD drive unit, 41C…Right PSD drive unit, 41D…Left PSD drive unit, 41E…PTG drive unit, 43…Door, 43A…Right electric hinged door, 43B…Left electric hinged door, 43C…Right electric sliding door, 43D…Left electric sliding door, 43E…Electric tailgate, 44…Camera (exterior image acquisition unit), 100… 101… First communication control unit, 102… Door control unit, 103… Door control prohibition unit, 110… First memory, 111… First control program, 200… Second processor, 201… Second communication control unit, 202… Registration unit, 203… Proximity detection unit, 204… External person detection unit (first detection unit), 205… Proximity method detection unit (second detection unit), 206… Comparison unit, 207… Face detection unit (fourth detection unit), 208… Authentication unit, 209… Monitoring unit, 210… Gesture detection unit (third detection unit), 220… Second memory, 221… Second control program, 222… Face database, 224… Baseline proximity method data, 223… Non-registered gesture database, V… Vehicle, VP… External person. Detailed Implementation

[0022] Figure 1 This is a diagram showing the vehicle's control system 1.

[0023] The control system 1 includes a central ECU (Electronic Control Unit) 2 for overall vehicle control and information processing. The central ECU 2 is connected to communication lines including first communication lines 3a and 3b and second communication lines 4a and 4b. The central ECU 2 functions as a gateway managing the transmission and reception of communication data between these communication lines. Furthermore, the central ECU 2 performs OTA (Over-The-Air) management. OTA management includes processing the download of updates to the vehicle's onboard devices from an external server and the control related to applying the downloaded updates to the onboard devices.

[0024] The first communication lines 3a and 3b, and the second communication lines 4a and 4b, are constructed from a bus that performs communication according to standards such as CAN (Controller Area Network) or Ethernet (registered trademark), or from communication lines that perform P2P (Peer-to-Peer) communication. Furthermore, the first communication lines 3a and 3b can be constructed from multiple communication lines performing communication according to the same standard, or from multiple communication lines performing communication according to different standards. The same applies to the second communication lines 4a and 4b.

[0025] The first communication line 3a is connected to the ICB (Infotainment Control Box) 6, speaker 7 and microphone 8 via the in-vehicle connection link 5.

[0026] Additionally, the in-vehicle connectivity link 5 connects a TCU (Telematics Control Unit) 10, a GNSS sensor (GNSS; Global Navigation Satellite System) 11, and a touch panel 12. The TCU 10 is a wireless communication device that conforms to the communication standards of mobile communication systems. The touch panel 12 includes a display 13 and a touch sensor 14.

[0027] ICB6 is the IVI (In-Vehicle Infotainment) ECU. ICB6 uses speakers 7, microphones 8, GNSS sensors 11, and touch panels 12 to provide various information and entertainment to vehicle occupants.

[0028] The in-vehicle connection link 5 consists of multiple communication transmission paths conforming to various communication standards. For example, the in-vehicle connection link 5 may also include multiple network transmission paths. In this case, the multiple network transmission paths may also be interconnected via a device with gateway functionality. Additionally, the in-vehicle connection link 5 may include a transmission path for P2P communication. In the network transmission paths, various communication buses conforming to various standards for network communication can be used. Examples of such standards include CAN, Ethernet, USB (Universal Serial Bus), LIN (Local Interconnect Network), and LVDS (Low Voltage Differential Signaling), but other standards may also be used.

[0029] A driver monitoring camera (DMC) 15 is connected to the first communication line 3b.

[0030] Area A-ECU 20 is connected to the second communication line 4a. Area A-ECU 20 is connected to the drive unit 21 and the battery 22. The drive unit 21 is, for example, a motor or internal combustion engine that drives the vehicle.

[0031] A zone B-ECU 23 is connected to the second communication line 4b. Zone B-ECU 23 is connected to the lamp body 24 and the window motor 25. The lamp body 24 includes, for example, headlights, taillights, and turn signals. The window motor 25 opens and closes the windows. Additionally, Zone B-ECU 23 is connected to the door sensor 26, door locking mechanism 27, ESL (Electronic Steering Lock) 28, and air conditioning unit 29. The door sensor 26 detects door operations. The door locking mechanism 27 locks and unlocks the vehicle doors. Zone B-ECU 23 is connected to the entry ECU 30. The entry ECU 30 is connected to the LF / RF antenna 31, which wirelessly communicates with the vehicle's electronic key. The electronic key is an electronic device with wireless communication capabilities, referred to as a smart key or FOB key. The entry ECU 30 collaborates with other onboard ECUs to handle user access to the control system 1 from outside the vehicle, enabling so-called smart entry.

[0032] The ECU30 is also connected to a front camera 32, a rear camera 33, a right-side camera 34, and a left-side camera 35. In addition, in area B-ECU23, there are five drive units 41 connected to the right PHD (Power Hinge Door) drive unit 41A, the right PSD (Power Slide Door) drive unit 41C, the left PHD drive unit 41B, the left PSD drive unit 41D, and the PTG (Power Tail Gate) drive unit 41E.

[0033] The following diagrams will use the "V" symbol to indicate the vehicle with control system 1.

[0034] Figure 2 This is a diagram showing the structure of vehicle V.

[0035] like Figure 2 As shown, vehicle V is a sedan with a seating capacity of 6, equipped with a driver's seat 42A, a front passenger seat 42B, a second row of right seats 42C, a second row of left seats 42D, a third row of right seats 42E, and a third row of left seats 42F.

[0036] Vehicle V has five doors 43: a right electric hinged door 43A, a left electric hinged door 43B, a right electric sliding door 43C, a left electric sliding door 43D, and an electric tailgate 43E.

[0037] The right electric hinged door 43A includes a right PHD drive unit 41A that drives the opening and closing of the right electric hinged door 43A via an electric actuator. The left electric hinged door 43B includes a left PHD drive unit 41B that drives the opening and closing of the left electric hinged door 43B via an electric actuator. The right electric sliding door 43C includes a right PSD drive unit 41C that drives the opening and closing of the right electric sliding door 43C via an electric actuator. The left electric sliding door 43D includes a left PSD drive unit 41D that drives the opening and closing of the left electric sliding door 43D via an electric actuator. The electric tailgate 43E includes a PTG drive unit 41E that drives the opening and closing of the electric tailgate 43E via an electric actuator.

[0038] Vehicle V is equipped with a door locking mechanism 27. The door locking mechanism 27 includes door locking mechanisms 33A, 33B, 33C, 33D, and 33E.

[0039] Door lock mechanism 33A locks and unlocks the right electric hinged door 43A. Door lock mechanism 33B locks and unlocks the left electric hinged door 43B. Door lock mechanism 33C locks and unlocks the right electric sliding door 43C. Door lock mechanism 33D locks and unlocks the left electric sliding door 43D. Door lock mechanism 33E locks and unlocks the electric tailgate 43E.

[0040] The vehicle V has four cameras 44: a front-facing camera 32 that captures images of the front of the vehicle V, a rear-facing camera 33 that captures images of the rear of the vehicle V, a right-facing camera 34 that captures images of the right side of the vehicle V, and a left-facing camera 35 that captures images of the left side of the vehicle V. The cameras 44 are exterior cameras that capture images of the exterior of the vehicle V. Therefore, the dynamic images captured by the cameras 44 include exterior images of the vehicle V.

[0041] The front-facing camera 32 is located at the front of the vehicle V. The rear-facing camera 33 is located at the rear of the vehicle V. The right-side camera 34 is located on the right side of the vehicle V. The left-side camera 35 is located on the left side of the vehicle V.

[0042] Camera 44 is equivalent to the "exterior image acquisition unit" disclosed herein.

[0043] Figure 3 This is a diagram showing the main structural components of control system 1.

[0044] The central ECU2 is connected to the area B-ECU23 via the second communication line 4b. Additionally, the area B-ECU23 is connected to the door lock mechanism 27, the right PHD drive unit 41A, the right PSD drive unit 41C, the left PHD drive unit 41B, the left PSD drive unit 41D, and the PTG drive unit 41E. Furthermore, the area B-ECU23 is connected to the entry ECU30.

[0045] Region B-ECU23 includes a first processor 100 such as CPU (Central Processing Unit) and MPU (Microprocessor Unit), a first memory 110, and interface circuits for connecting devices, sensors, and other equipment.

[0046] The first memory 110 is a storage device that non-volatilely stores programs and data executed by the first processor 100. The first memory 110 is composed of semiconductor storage elements such as magnetic storage devices, flash memory (ROM), or other types of non-volatile storage devices. Alternatively, the first memory 110 may also include RAM (Random Access Memory) constituting the working area of ​​the first processor 100. Furthermore, the first memory 110 may also include non-volatile storage devices such as HDD (Hard Disk Drive) and SSD (Solid State Drive). The first memory 110 stores data processed by the first processor 100 and the first control program 111 executed by the first processor 100.

[0047] The first processor 100 functions as the first communication control unit 101, the door control unit 102, and the door control prohibition unit 103 by reading and executing the first control program 111 stored in the first memory 110.

[0048] The first communication control unit 101 receives various information from the input ECU 30.

[0049] Door control unit 102 controls door 43.

[0050] The door control unit 102 unlocks and locks the door 43. The door control unit 102 unlocks the door 43 by outputting an unlock signal to the door lock mechanism 27. The door control unit 102 locks the door 43 by outputting a lock signal to the door lock mechanism 27. The door control unit 102 can designate and control the door 43, and can lock and unlock the designated door 43.

[0051] The door control unit 102 opens the door 43. The door control unit 102 controls the drive unit 41 by outputting a control signal to the drive unit 41, thereby opening the door 43. The door control unit 102 can designate the door 43 to be controlled and open the designated door 43.

[0052] The door control unit 102 performs the closing action of the door 43. The door control unit 102 controls the drive unit 41 by outputting a control signal to the drive unit 41, thereby performing the closing action of the door 43. The door control unit 102 can specify the door 43 to be controlled and can perform the closing action of the specified door 43.

[0053] The door control prohibition unit 103 prohibits the door control unit 102 from controlling the door 43. In this embodiment, prohibiting the control of the door 43 includes not executing the control of the door 43, interrupting the control of the door 43 that is currently being executed, and interrupting the control of the door 43 that is currently being executed and returning the door 43 to its state before execution. When the first communication control unit 101 receives a prohibition request information, the door control prohibition unit 103 prohibits the door control unit 102 from controlling the door 43. The door control prohibition unit 103 prohibits the door control unit 102 from controlling the door 43 by outputting a prohibition information to the door control unit 102. When a predetermined trigger, such as unlocking the door 43 based on an electronic key, occurs, the door control prohibition unit 103 releases the prohibition of the control of the door 43. The door control prohibition unit 103 releases the prohibition of the control of the door 43 by outputting a release information to the door control unit 102.

[0054] The ECU 30 has interface circuits that connect to a second processor 200 (such as a CPU and MPU), a second memory 220, and devices such as sensors. Dynamic image data is input to the ECU 30 from the camera 44. This dynamic image data is the data from dynamic images captured by the camera 44.

[0055] The second memory 220 is a storage device that non-volatilely stores programs and data executed by the second processor 200. The second memory 220 is composed of a magnetic storage device, a semiconductor storage element such as flash memory (ROM), or other types of non-volatile storage devices. Alternatively, the second memory 220 may include RAM constituting the working area of ​​the second processor 200. Furthermore, the second memory 220 may also include non-volatile storage devices such as HDDs and SSDs. The second memory 220 stores data processed by the second processor 200, the second control program 221 executed by the second processor 200, a face database 222, a non-registered gesture database 223, and reference proximity mode data 224, etc.

[0056] The face database 222 contains face-related information for each user record in vehicle V. Each record in the face database 222 contains facial feature data representing facial feature quantities. Facial feature quantities are numerical sets obtained by quantifying each feature point of the face. Examples of feature points include the positions of the eyes, nose, and mouth relative to the facial contour.

[0057] The unregistered gesture database 223 contains information related to each unregistered gesture record. An unregistered gesture is a gesture that has not been pre-registered by the user of vehicle V. Each record in the unregistered gesture database 223 contains gesture content information. Gesture content information is information representing the content of the gesture.

[0058] The reference approach mode data 224 is data representing a reference approach mode. In this embodiment, the approach mode is exemplified by the speed at which the vehicle V approaches, i.e., the approach speed. Therefore, the reference approach mode data 224 in this embodiment is data representing a reference approach speed.

[0059] The second processor 200 reads and executes the second control program 221 stored in the second memory 220, and functions as the second communication control unit 201, registration unit 202, proximity detection unit 203, vehicle-exterior person detection unit 204, proximity mode detection unit 205, comparison unit 206, face detection unit 207, authentication unit 208, monitoring unit 209, and gesture detection unit 210.

[0060] The external person detection unit 204 is equivalent to the "first detection unit" of this disclosure. The approach method detection unit 205 is equivalent to the "second detection unit" of this disclosure. The face detection unit 207 is equivalent to the "fourth detection unit" of this disclosure. The gesture detection unit 210 is equivalent to the "third detection unit" of this disclosure.

[0061] The second communication control unit 201 receives dynamic image data from region B-ECU 23. The second communication control unit 201 outputs the received dynamic image data to the various functional units of the second processor 200.

[0062] Registration unit 202 records the facial feature data of the user of vehicle V in facial database 222. Registration unit 202 generates facial feature data based on the dynamic image captured by camera 44 and records it in facial database 222. In this way, registration unit 202 records facial feature data in facial database 222, so the user of vehicle V can register their face with vehicle V by having camera 44 capture their face.

[0063] The proximity detection unit 203 detects the approach of an object to the vehicle V. For example, the proximity detection unit 203 detects the approach of an object to the vehicle V based on the detection values ​​of the sonar sensors equipped on the vehicle V. Alternatively, for example, the proximity detection unit 203 detects the approach of an object to the vehicle V by using the detection of an electronic key via the LF / RF antenna 31. Furthermore, the detection method of the proximity detection unit 203 is merely one example and is not limited to the above example. When the proximity detection unit 203 detects the approach of an object to the vehicle V, the camera 44 of this embodiment is activated and begins to capture images.

[0064] The vehicle exterior person detection unit 204 detects vehicle exterior persons (VPs) existing outside the vehicle V based on the dynamic image image shown in the dynamic image data received by the second communication control unit 201. Specifically, the vehicle exterior person detection unit 204 detects images of vehicle exterior persons (VPs) from the vehicle exterior image using methods such as pattern matching, thereby detecting the vehicle exterior person (VP).

[0065] The approach mode detection unit 205 detects the approach mode of the person outside the vehicle (VP) detected by the person outside the vehicle (Pv) detection unit 204 at a predetermined period. As described above, approach speed is exemplified as an approach mode. Therefore, in this embodiment, the approach mode detection unit 205 detects the approach speed of the person outside the vehicle (VP) detected by the person outside the vehicle (Pv) detection unit 204. The approach mode detection unit 205 tracks the image of the person outside the vehicle (VP) detected by the person outside the vehicle (Pv) in the moving image captured by the camera 44. The approach mode detection unit 205 detects the approach speed based on the area change of the image of the person outside the vehicle (VP) being tracked.

[0066] The comparison unit 206 compares the approach speed of the person outside the vehicle (VP) detected by the approach mode detection unit 205 with a reference approach speed at a predetermined interval. Specifically, the comparison unit 206 compares the approach speed of the person outside the vehicle (VP) detected by the approach mode detection unit 205 with the approach speed shown in the reference approach mode data 224. The comparison unit 206 outputs comparison result information indicating the comparison result to the second communication control unit 201 at a predetermined interval. The second communication control unit 201 sends the comparison result information received from the comparison unit 206 to the area B-ECU 23.

[0067] The face detection unit 207 detects the face of the person outside the vehicle (VP) detected by the person outside the vehicle (VP) detection unit 204 at a predetermined period. The face detection unit 207 tracks the image of the person outside the vehicle (VP) detected by the person outside the vehicle (VP) in the moving image captured by the camera 44. The face detection unit 207 detects the face image from the tracked image of the person outside the vehicle (VP) using methods such as pattern matching, thereby detecting the face of the person outside the vehicle (VP).

[0068] The authentication unit 208 authenticates whether the person outside the vehicle (VP) detected by the external person detection unit 204 is a registered user. The authentication unit 208 authenticates whether the user is a registered user based on the face detected by the face detection unit 207. A registered user refers to a person who has pre-registered with vehicle V as a user of vehicle V. The authentication unit 208 calculates facial feature values ​​based on the face image detected by the face detection unit 207. The authentication unit 208 compares the calculated facial feature values ​​with the facial feature value data recorded in the face database 222, and determines whether the face database 222 contains facial feature value data with a consistency rate of at least a threshold that matches the calculated facial feature values. If the authentication unit 208 determines that the face database 222 contains facial feature value data with a consistency rate of at least the threshold that matches the calculated facial feature values, the authentication unit 208 authenticates the person outside the vehicle (VP) detected by the external person detection unit 204 as a registered user. On the other hand, if the authentication unit 208 determines that the facial feature data with a consistency rate of more than the threshold with the calculated facial feature data is not included in the facial database 222, it authenticates that the external person VP detected by the external person detection unit 204 is not a registered user.

[0069] The monitoring unit 209 monitors the distance between a person outside the vehicle (VP) certified as a registered user by the authentication unit 208 and the vehicle (V). The monitoring unit 209 calculates the distance between the person outside the vehicle (VP) and the vehicle (V) at predetermined intervals and outputs distance information representing the calculated distance to the second communication control unit 201. Furthermore, the monitoring unit 209 tracks the image of the person outside the vehicle (VP) certified as a registered user by the authentication unit 208 in the dynamic image captured by the camera 44, and calculates the distance between the person outside the vehicle (VP) and the vehicle (V) based on the size or area of ​​the tracked image. If the second communication control unit 201 receives distance information from the monitoring unit 209, it sends the received distance information to the area B-ECU 23.

[0070] Monitoring unit 209 monitors whether a person outside the vehicle (VP) certified as a registered user by authentication unit 208 stops while approaching vehicle V, and whether they leave vehicle V while approaching vehicle V. Furthermore, stopping while approaching vehicle V means approaching vehicle V but stopping before reaching it. Similarly, leaving vehicle V while approaching it means approaching vehicle V but leaving it before reaching it.

[0071] The monitoring unit 209 tracks the image of an external person (VP) certified as a registered user by the authentication unit 208 in the dynamic image captured by the camera 44. If the area of ​​the tracked external person (VP) image remains unchanged within a specified range and period, the monitoring unit 209 determines that the person is approaching the vehicle V and has stopped. If the area of ​​the tracked external person (VP) image decreases, the monitoring unit 209 determines that the person is leaving the vehicle V while approaching it. The monitoring unit 209 outputs a prohibition request message to the second communication control unit 201 if it determines that the person has stopped while approaching the vehicle V or left the vehicle V while approaching it. If the second communication control unit 201 receives the prohibition request message from the monitoring unit 209, it sends the received prohibition request message to the area B-ECU 23.

[0072] The gesture detection unit 210 detects unregistered gestures from external figures (VPs) authenticated as registered users by the authentication unit 208. The gesture detection unit 210 tracks images of external figures (VPs) authenticated as registered users in dynamic images captured by the camera 44. The gesture detection unit 210 performs image analysis on the tracked image of the external figure (VP) and extracts features of the VP's movements (e.g., motion vectors). Next, the gesture detection unit 210 identifies the content of the VP's movements based on the extracted features. Then, if the identified VP's movement content matches a gesture recorded in the unregistered gesture database 223, the gesture detection unit 210 determines that an unregistered gesture has been detected.

[0073] Next, the operation of each part of the control system 1 will be explained.

[0074] Figure 4 This is a flowchart illustrating the actions entering ECU30 and region B-ECU23. Figure 4 In the flowchart, FA represents the action entering ECU30, and FB represents the action in area B-ECU23.

[0075] As shown in flowchart FA, the vehicle exterior person detection unit 204 determines whether the proximity detection unit 203 has detected the approach of an object (step SA1). If the vehicle exterior person detection unit 204 determines that the proximity detection unit 203 has not detected the approach of an object (step SA1: no), the determination in step SA1 is performed again.

[0076] When the vehicle exterior person detection unit 204 determines that the proximity detection unit 203 has detected the approach of an object (step SA1: Yes), it begins the detection of the vehicle exterior person VP (step SA2).

[0077] Next, the proximity detection unit 205 determines whether the external person detection unit 204 has detected an external person VP (step SA3). If it is determined that no external person VP has been detected (step SA3: no), the second processor 200 ends the process.

[0078] If the vehicle external person detection unit 204 determines that a person VP has been detected outside the vehicle (step SA3: Yes), the approach mode detection unit 205 starts to detect the approach speed of the person VP detected by the vehicle external person detection unit 204 (step SA4).

[0079] Next, the comparison unit 206 begins to compare the approach speed detected by the approach mode detection unit 205 with the approach speed represented by the reference approach mode data 224 (step SA5).

[0080] Next, the authentication department 208 begins to verify whether the user is a registered user (step SA6).

[0081] Next, the second communication control unit 201 determines whether the authentication unit 208 has authenticated the user as a registered user (step SA7). If the second communication control unit 201 determines that the user is not a registered user (step SA7: No), the second processor 200 ends the process.

[0082] On the other hand, if the authentication is determined to be that of a registered user (step SA7: Yes), the second communication control unit 201 sends the comparison result information to the area B-ECU 23 (step SA8). In step SA8, the second communication control unit 201 sends the latest comparison result information from the comparison result information received from the comparison unit 206 to the area B-ECU 23.

[0083] As shown in flowchart FB, the first communication control unit 101 receives comparison result information from the input ECU 30 (step SB1).

[0084] Next, the door control unit 102 performs the opening action of the door 43 at the start time corresponding to the comparison result indicated by the comparison result information received in step SB1 (step SB2).

[0085] Step SB2 is described in detail.

[0086] When the comparison result indicates that the calculated approach speed is the same as the approach speed of the reference approach mode data 224, the door control unit 102 starts the door opening operation when the distance indicated by the distance information output by the monitoring unit 209 becomes a first distance. That is, the door control unit 102 starts the door opening operation when the registered user's approach speed is the same as the reference approach speed and the distance between the registered user and the vehicle V becomes a first distance.

[0087] If the door control unit 102 finds that the calculated approach speed is slower than the approach speed of the reference approach mode data 224, and the distance indicated by the distance information output by the monitoring unit 209 is a second distance shorter than the first distance, then the door control unit 102 initiates the opening operation of the door 43. In other words, the door control unit 102 initiates the opening operation of the door 43 when the registered user's approach speed is slower than the reference approach speed, and the distance between the registered user and the vehicle V becomes a second distance.

[0088] If the comparison result indicates that the calculated approach speed is faster than the approach speed of the reference approach mode data 224, and the distance indicated by the distance information output by the monitoring unit 209 becomes a third distance longer than the first distance, the door control unit 102 starts the door opening operation of the door 43. That is, if the approach speed of the registered user is faster than the reference approach speed, and the distance between the registered user and the vehicle V becomes a third distance, the door control unit 102 starts the door opening operation of the door 43.

[0089] As described above, when the approach speed of the registered user is slower than the reference approach speed, the control system 1 delays the start time of the door 43 opening action, thus preventing the door 43 from unnecessarily remaining open for an extended period and improving the safety of the vehicle V. Furthermore, by opening the door 43 at an appropriate time relative to the arrival time of the registered user in the vehicle V, the control system 1 enables smooth and seamless entry into the vehicle V.

[0090] In addition, when the approach speed of the registered user is faster than the reference approach speed, the control system 1 advances the start time and starts the opening action of the door 43, so that it can achieve smooth and seamless entry into the vehicle V in response to situations such as rainy days when the registered user is in a hurry.

[0091] In this embodiment, if a registered user makes a non-registered gesture when approaching vehicle V, the control system 1 performs an opening action on door 43, so that the opening degree of door 43 becomes a predetermined opening degree (e.g., half).

[0092] (Actions related to non-registered gestures)

[0093] When the gesture detection unit 210 enters the ECU 30 and is authenticated as a registered user by the authentication unit 208, it begins detecting unregistered gestures from the point where the external person VP is authenticated as a registered user by the authentication unit 208. If the gesture detection unit 210 does not detect any unregistered gestures within a predetermined period from the start of the unregistered gesture detection, it terminates the unregistered gesture detection. On the other hand, if the gesture detection unit 210 detects an unregistered gesture, it outputs detection information indicating that an unregistered gesture has been detected to the second communication control unit 201. The second communication control unit 201 then sends the received detection information to the area B-ECU 23.

[0094] If the first communication control unit 101 receives the detection information, the door control unit 102 of region B-ECU 23 opens the door 43 so that the opening degree of the door 43 becomes a predetermined opening degree. At the moment the first communication control unit 101 receives the detection information, if the opening degree of the door 43 is greater than or equal to the predetermined opening degree, the door control unit 102 may also close the door 43 so that the opening degree of the door 43 becomes the predetermined opening degree.

[0095] As described above, when the control system 1 detects a non-registered gesture from a registered user, it opens the door 43 in such a way that the opening degree of the door 43 becomes a predetermined opening degree. For example, if the non-registered gesture is a skip, the registered user can specify the opening degree of the door 43 as a predetermined opening degree by skipping when approaching the vehicle V.

[0096] However, there are instances where a registered user stops while approaching vehicle V, or leaves vehicle V while approaching it. Therefore, control system 1 performs actions related to these situations after authentication by authentication department 208.

[0097] (Actions related to situations where the approach process stops)

[0098] The monitoring unit 209 of ECU 30 determines whether the external person VP, certified as a registered user by the authentication unit 208, stops while approaching vehicle V. If the monitoring unit 209 determines that the person stopped while approaching vehicle V, it outputs a prohibition request message to the second communication control unit 201. Conversely, if the monitoring unit 209 determines that the person did not stop while approaching vehicle V, it does not output a prohibition request message to the second communication control unit 201. If the second communication control unit 201 receives a prohibition request message from the monitoring unit 209, it sends the received prohibition request message to area B-ECU 23.

[0099] If the first communication control unit 101 receives a prohibition request message, the door control prohibition unit 103 of region B-ECU 23 prohibits the door control unit 102 from controlling door 43. When the door control unit 102 is about to execute door 43 control, it does not perform the planned door 43 control. When the door control unit 102 is controlling door 43, it either interrupts the door 43 control or returns the state of door 43 to what it was before control began.

[0100] As described above, control system 1 disables control of door 43 if the registered user stops while approaching vehicle V. If the registered user stops while approaching vehicle V, they may not use vehicle V after stopping. Therefore, by disabling control of door 43, control system 1 ensures the safety of vehicle V.

[0101] (Actions taken when leaving during the approach process)

[0102] The monitoring unit 209 of ECU 30 determines whether a person outside the vehicle (VP) certified as a registered user by the authentication unit 208 has left the vehicle V while approaching it. If the monitoring unit 209 determines that the person left the vehicle V while approaching it, it outputs a lock request to the second communication control unit 201. Conversely, if the monitoring unit 209 determines that the person did not leave the vehicle V while approaching it, it does not output a lock request to the second communication control unit 201. If the second communication control unit 201 receives a lock request from the monitoring unit 209, it sends the received lock request to area B-ECU 23.

[0103] If the first communication control unit 101 receives a lock request message, the door control unit 102 of area B-ECU23 locks the door 43. If the door control unit 102 has performed an opening action on the door 43, it interrupts the opening action of the door 43, performs a closing action on the door 43, and locks the door 43.

[0104] As described above, the control system 1 locks the door 43 when a registered user leaves the vehicle V while approaching it, thus ensuring the safety of the vehicle V.

[0105] However, due to factors such as the registered user being off-center from the field of view of camera 44, control system 1 may be unable to determine if a registered user exists around vehicle V. Therefore, control system 1 performs the operation in this situation after authentication by authentication department 208.

[0106] (Actions related to situations where the existence of a registered user cannot be determined)

[0107] The face detection unit 207 of ECU 30 determines whether, within a predetermined period from the time the person is certified as a registered user by the authentication unit 208, the face of the external person VP cannot be detected. This predetermined period is set from the time the external person VP is certified as a registered user until entering the vehicle V, and is determined through prior testing, simulation, etc. If the face detection unit 207 determines that the face of the external person VP cannot be detected, it outputs a prohibition request message to the second communication control unit 201. Upon receiving the prohibition request message from the face detection unit 207, the second communication control unit 201 sends the received prohibition request message to area B-ECU 23.

[0108] When the first communication control unit 101 receives a prohibition request message, the door control unit 102 prohibits the door control unit 102 from controlling the door 43. Before the execution of door 43 control, the door control unit 102 does not perform the planned door 43 control. When the door control unit 102 is controlling the door 43, it either interrupts the door 43 control or returns the door 43 to its state before control began.

[0109] As described above, control system 1 disables control of door 43 if it cannot determine that a registered user exists in the vicinity of vehicle V. Therefore, control system 1 can ensure the safety of vehicle V.

[0110] The second embodiment will be described.

[0111] Regarding the structure of the second embodiment, the same reference numerals are used for structures identical to those in the first embodiment described above, and detailed descriptions are omitted.

[0112] Figure 5 This is a diagram showing the main structural components of the control system 1 according to the second embodiment.

[0113] Compare Figure 5 and Figure 2 It can be seen that the second memory 220 also stores the registered gesture database 225.

[0114] The gesture registration database 225 contains gesture-related information for each registered gesture record. A registered gesture refers to a gesture pre-registered by the user of vehicle V on vehicle V. Each record in the gesture registration database 225 contains gesture content information and door control content information. The door control content information represents the control content of door 43. For example, in the gesture registration database 225, gesture content information representing a gesture such as "raise your left hand" corresponds to door control content information representing a control content such as "open the right electric sliding door".

[0115] In the second embodiment, the registration unit 202 records the combination of the gesture content of the registered gesture and the control content of the door 43 in the registration gesture database 225. The registration unit 202 in the second embodiment identifies the gesture content of the gesture performed by the user of vehicle V based on the dynamic image captured by camera 44 using methods such as image analysis. Then, the registration unit 202 in the second embodiment records the door control content information representing the control content of the door 43 specified by the user of vehicle V in the registration gesture database 225, corresponding to the gesture content information represented by the identified gesture content. In this way, the registration unit 202 in the second embodiment records the combination of gesture content and door 43 control content in the registration gesture database 225, thus allowing the user of vehicle V to register the desired gesture for performing the desired control with vehicle V.

[0116] The gesture detection unit 210 of the second embodiment performs non-registered gesture detection in the same way as the gesture detection unit 210 of the first embodiment. Furthermore, the gesture detection unit 210 of the second embodiment detects registered gestures from external person / entity (VP) detected by the external person / entity detection unit 204. The gesture detection unit 210 tracks the image of the external person / entity (VP) detected by the external person / entity detection unit 204 in the dynamic image captured by the camera 44. The gesture detection unit 210 performs image analysis on the tracked image of the external person / entity (VP) and extracts the features of the external person / entity's actions (e.g., motion vectors). Next, the gesture detection unit 210 identifies the action content of the external person / entity (VP) based on the extracted action features. Then, if the identified action content of the external person / entity (VP) is a gesture recorded in the registered gesture database 225, the gesture detection unit 210 determines that a registered gesture has been detected.

[0117] Next, the operation of each part of the control system 1 in the second embodiment will be explained.

[0118] Figure 6 This is a flowchart illustrating the actions entering ECU30 and region B-ECU23. Figure 6 In the flowchart, FC represents the action entering ECU30, and FD represents the action in region B-ECU23.

[0119] exist Figure 6 In the flowchart, for and Figure 4 The same steps are labeled with the same step number and their detailed descriptions are omitted.

[0120] As shown in flowchart FD, the door control unit 102 determines whether the comparison result information received in step SB1 indicates that the calculated approach speed is slower than the approach speed of the reference approach mode data 224 (step SD1). That is, in step SD1, the door control unit 102 determines whether the approach speed of the registered user is slower than the reference approach speed.

[0121] If the door control unit 102 determines that the approach speed of the registered user is slower than the reference approach speed (step SD1: Yes), it unlocks the door 43 (step SD2).

[0122] On the other hand, if the door control unit 102 determines that the approach speed of the registered user is not slower than the reference approach speed (step SD1: no), it performs the door 43 opening operation at the start time corresponding to the comparison result indicated by the comparison result information received in step SB1 (step SD3).

[0123] Step SD3 will be described in detail.

[0124] When the door control unit 102 finds that the calculated approach speed is the same as the approach speed in the reference approach mode data 224, and the distance indicated by the distance information output by the monitoring unit 209 becomes a first distance, the door control unit 102 initiates the opening operation of the door 43. In other words, the door control unit 102 initiates the opening operation of the door 43 when the registered user's approach speed is the same as the reference approach speed and the distance between the registered user and the vehicle V becomes a first distance.

[0125] If the door control unit 102 finds that the calculated approach speed is faster than the speed of the reference approach mode data 224, and the distance indicated by the distance information output by the monitoring unit 209 is a third distance longer than the first distance, then the door control unit 102 initiates the opening operation of the door 43. In other words, the door control unit 102 initiates the opening operation of the door 43 when the registered user's approach speed is faster than the reference approach speed, and the distance between the registered user and the vehicle V becomes a third distance.

[0126] As described above, when the approach speed of the registered user is slower than the reference approach speed, the control system 1 unlocks the door 43 but does not open it. Therefore, the door 43 can be opened at the moment the registered user intends to open, thus ensuring the safety of the vehicle V and improving the convenience for the user of the vehicle V.

[0127] In this embodiment, the control system 1 detects registration gestures when only unlocking the door 43 is performed as control of the door 43. That is, the control system 1 detects registration gestures when the approach speed of the registered user is slower than a reference. This situation will be explained below.

[0128] (Actions related to the case of only unlocking the door)

[0129] When the entry ECU 30 sends a comparison result message to the area B-ECU 23 indicating that the approach speed is slower than the reference approach mode data 224, the gesture detection unit 210 of the entry ECU 30 begins detecting registered gestures. If the gesture detection unit 210 does not detect a registered gesture within a predetermined period from the start of the registration gesture detection, it ends the registration gesture detection. If the gesture detection unit 210 detects a registered gesture, it retrieves the door control content information corresponding to the detected registered gesture from the registration gesture database 225. Then, the second communication control unit 201 sends the door control content information obtained by the gesture detection unit 210 to the area B-ECU 23.

[0130] If the first communication control unit 101 receives door control content information, the door control unit 102 of region B-ECU23 performs the door 43 opening action according to the control content indicated by the received door control content information.

[0131] For example, in the registered gesture database 225, gesture content information representing a gesture such as "raising the left hand" is set to correspond to door control content information representing a control action such as "opening the right electric sliding door". Furthermore, when the gesture detection unit 210 begins detecting a registered gesture, at least the right electric sliding door 43C is unlocked. Additionally, the gesture detection unit 210 detects a registered gesture of raising the left hand. In this example, the door control unit 102 performs the opening action of the right electric sliding door 43C.

[0132] (Actions related to non-registered gestures)

[0133] In this embodiment, if a registered user makes a non-registered gesture while approaching vehicle V, the control system 1 also opens door 43 to a predetermined opening degree (e.g., halfway). The control system 1 in this embodiment performs the same operation as the control system 1 in the first embodiment, setting the door 43 to a predetermined opening degree upon detecting a non-registered gesture.

[0134] (Actions related to situations where the approach process stops)

[0135] In this embodiment, if the registered user stops while approaching vehicle V, the control system 1 also disables control of door 43. The control system 1 in this embodiment performs the same operation as the control system 1 in the first embodiment, disabling control of door 43 if the registered user stops while approaching vehicle V.

[0136] (Actions taken when leaving during the approach process)

[0137] In this embodiment, when a registered user leaves vehicle V while approaching it, the control system 1 also locks door 43. The control system 1 in this embodiment performs the same operation as the control system 1 in the first embodiment, locking door 43 when a registered user leaves vehicle V while approaching it.

[0138] (Actions related to situations where the existence of a registered user cannot be determined)

[0139] In this embodiment, if the face of the person outside the vehicle (VP) authenticated as a registered user cannot be detected, the control system 1 also disables control of the door 43. The control system 1 in this embodiment performs the same operation as the control system 1 in the first embodiment, disabling control of the door 43 if the face of the person outside the vehicle (VP) authenticated as a registered user cannot be detected.

[0140] Furthermore, in this embodiment, if the face of an external person VP authenticated as a registered user by the authentication unit 208 cannot be detected, the control system 1 stops detecting the registered gesture. This stopping can also be performed together with the prohibition of the control of the door 43, or in lieu of the prohibition of the control of the door 43.

[0141] The face detection unit 207 of the ECU 30 determines whether the face of the external person VP, who is certified as a registered user, can no longer be detected within a predetermined period since the user was certified as a registered user. If the face detection unit 207 determines that the face of the external person VP, who is certified as a registered user, can no longer be detected, it outputs a stop request message to the gesture detection unit 210. If the gesture detection unit 210 receives the stop request message from the face detection unit 207, it stops the detection of gestures.

[0142] Therefore, if the control system 1 cannot determine that there is a registered user in the vicinity of the vehicle V, it will not control the door 43 based on gestures, thus further ensuring the safety of the vehicle V.

[0143] Furthermore, the control system 1 of this embodiment controls the gate 43 even if the registered user stops during the approach process, provided that a registration gesture is detected. This will be explained below.

[0144] (Actions related to the detection of a registered gesture after the approach process has stopped)

[0145] If a registered gesture is detected after the door control prohibition unit 103 prohibits the control of door 43, the gesture detection unit 210 of the ECU 30 retrieves door control content information corresponding to the gesture content of the detected registered gesture from the registered gesture database 225. Then, the second communication control unit 201 sends the door control content information obtained by the gesture detection unit 210 to the region B-ECU 23.

[0146] If the first communication control unit 101 receives door control content information, the door control prohibition unit 103 of region B-ECU 23 will output a release information to the door control unit 102. Then, the door control unit 102 will open the door 43 according to the control content shown in the received door control content information.

[0147] The above embodiments are merely one way of doing things and can be modified and applied in any way.

[0148] In the first embodiment described above, a structure was illustrated in which the start time of opening the door 43 differs based on the comparison result. However, a structure could also be used where the speed of opening the door 43 differs based on the comparison result. In this structure, if the comparison result indicates that the registered user's approach speed is slower than a reference approach speed, the door control unit 102 will make the speed of opening the door 43 slower than when the registered user's approach speed is higher than the reference approach speed. Furthermore, in this structure, if the comparison result indicates that the registered user's approach speed is faster than the reference approach speed, the door control unit 102 will make the speed of opening the door 43 faster than when the registered user's approach speed is lower than the reference approach speed. Moreover, in this structure, the start time of opening the door is the same regardless of the comparison result.

[0149] In the first embodiment described above, the door 43 is configured to open based on a comparison result, but it can also be configured to unlock based on a comparison result. In this configuration, if the comparison result indicates that the registered user's approach speed is slower than a reference approach speed, the door control unit 102 delays the unlocking time of the door 43 compared to when the registered user's approach speed is higher than the reference approach speed. Conversely, if the comparison result indicates that the registered user's approach speed is faster than the reference approach speed, the door control unit 102 advances the unlocking time of the door 43 compared to when the registered user's approach speed is lower than the reference approach speed.

[0150] In the second embodiment described above, the door 43 is opened when the approach speed is above the registered user's approach speed, but it can also be unlocked. In this case, if the comparison result indicates that the registered user's approach speed is faster than the reference approach speed, the door control unit 102 unlocks the door 43 earlier than when the approach speed is below the registered user's approach speed.

[0151] In the above embodiments, the approach speed is exemplified as the approach method, but the approach method is not limited to the approach speed. For example, it can also be the distance between the person VP outside the vehicle and the vehicle V.

[0152] The first processor 100 and the second processor 200 can be composed of multiple processors or a single processor. These processors can also be hardware programmed to implement the aforementioned functional units. In this case, these processors are, for example, composed of an ASIC (Application Specific Integrated Circuit) or a FPGA (Field Programmable Gate Array).

[0153] in addition, Figure 1 The structure of each part of the control system 1 shown is an example, and the specific installation method is not particularly limited. That is, it is not necessarily necessary to install hardware corresponding to each part; of course, it can also be configured so that the functions of each part are implemented by a processor executing a program. In addition, in the above embodiment, a part of the function implemented by software can be implemented as hardware, or a part of the function implemented by hardware can be implemented by software.

[0154] in addition, Figure 4 and Figure 6 The steps shown are divided based on the main processing content, and this invention is not limited to the method or name of the processing unit division. Depending on the processing content, it can also be divided into more step units. Alternatively, it can be divided so that one step unit contains more processes. Furthermore, the order of these steps can be appropriately changed without affecting the spirit of this invention.

[0155] Furthermore, when using a processor to implement the control method of the control system 1 described above, the program executed by the processor can also be configured as a recording medium or a transmission medium for transmitting the program. That is, the first control program 111 can also be implemented with the first control program 111 recorded on a removable information recording medium. Examples of information recording media include magnetic recording media such as hard disks, optical recording media such as CDs, USB (Universal Serial Bus) memory, SSD (Solid State Drive) and other semiconductor storage devices, but other recording media can also be used. In addition, the second control program 221, like the first control program 111, can also be implemented with the second control program 221 recorded on a removable information recording medium.

[0156] The above implementation supports the following structures.

[0157] (Structure 1)

[0158] A vehicle control system includes: an exterior image acquisition unit that acquires an exterior image of the vehicle; a first detection unit that detects a person outside the vehicle based on the exterior image acquired by the exterior image acquisition unit; a second detection unit that detects an approach mode, i.e., an approach method, for the person outside the vehicle detected by the first detection unit; a comparison unit that compares the approach mode detected by the second detection unit with the reference approach mode; an authentication unit that authenticates whether the person outside the vehicle detected by the first detection unit is a pre-registered user of the vehicle; and a door control unit that, if the authentication unit authenticates the person as the registered user, controls the vehicle door based on the comparison result of the comparison unit.

[0159] According to the control system of Structure 1, the vehicle door can be controlled by taking into account the approach mode of the registered user, thus improving the convenience of the vehicle user in the control of the vehicle door.

[0160] (Structure 2)

[0161] In the control system of structure 1, the second detection unit detects the approach speed approaching the vehicle as the approach mode. If the comparison result indicates that the approach speed detected by the second detection unit is slower than the reference approach speed, the door control unit delays the start time of the door opening action compared to when the approach speed detected by the second detection unit is above the reference approach speed, or slows down the action speed of the door opening action compared to when the approach speed detected by the second detection unit is above the reference approach speed, and performs the door opening action.

[0162] According to the control system of Structure 2, the vehicle door can be unlocked or opened at the appropriate time when the registered user arrives at the vehicle. Therefore, smooth and seamless entry into the vehicle can be achieved, further improving the convenience for the vehicle user through the control of the vehicle door.

[0163] (Structure 3)

[0164] In a control system of structure 1 or structure 2, the second detection unit detects the approach speed approaching the vehicle as the approach mode. If the comparison result indicates that the approach speed detected by the second detection unit is faster than a reference approach speed, the door control unit performs the door opening action by either advancing the start time of the door opening action compared to when the approach speed detected by the second detection unit is below the reference approach speed, or by making the speed of the door opening action faster compared to when the approach speed detected by the second detection unit is below the reference approach speed.

[0165] According to the control system of Structure 3, the vehicle doors can be unlocked or opened at the appropriate time when a registered user arrives at the vehicle. Therefore, smooth and seamless entry into the vehicle can be achieved, further improving the convenience for vehicle users through the control of the vehicle doors.

[0166] (Structure 4)

[0167] In the control system of structure 1, the second detection unit detects the approach speed of the approaching vehicle as the approach method. If the comparison result indicates that the approach speed detected by the second detection unit is slower than the reference approach speed, the door control unit unlocks the door but does not open the door.

[0168] According to the control system of Structure 4, since the door is unlocked but not opened, it can be opened at the moment intended by the vehicle user. Therefore, the convenience of the vehicle user can be further improved in the control of the vehicle's doors.

[0169] (Structure 5)

[0170] The control system of structure 4 includes a third detection unit that detects the gestures of the person outside the vehicle detected by the first detection unit. If the third detection unit detects a pre-registered gesture after the door is unlocked, the door control unit performs the door opening action based on the registered gesture detected by the third detection unit.

[0171] According to the control system of structure 5, the door can be opened by registering a gesture, thus further improving the convenience of vehicle users in the control of vehicle doors.

[0172] (Structure 6)

[0173] The control system of structure 5 includes a fourth detection unit that detects the face of the person outside the vehicle detected by the first detection unit. If the third detection unit stops detecting the face after the fourth detection unit has detected the face, the third detection unit stops detecting the gesture.

[0174] According to the control system of structure 6, if it cannot be determined whether there is a registered user in the vicinity of the vehicle, the door will not be opened due to the registration gesture, thus ensuring the safety of the vehicle.

[0175] (Structure 7)

[0176] A control system of any one of structures 1 to 6, wherein the control system includes a third detection unit that detects the gestures of the person outside the vehicle detected by the first detection unit, and if the third detection unit detects an unregistered gesture that has not been pre-registered, the door control unit performs the door opening action in a manner that opens to a predetermined opening degree.

[0177] According to the control system of structure 7, the door opening degree can be specified to a predetermined opening degree by making a non-registration gesture even without registration, thus further improving the convenience of vehicle users in the control of vehicle doors.

[0178] (Structure 8)

[0179] A control system of any one of structures 1 to 7, wherein the control system includes a door control prohibition unit that, when the person outside the vehicle, identified by the authentication unit as the registered user, stops while approaching the vehicle, prohibits the door control unit from controlling the door.

[0180] If a registered user stops during the approach process, the registered user may not use the vehicle, but according to the control system of structure 8, the safety of the vehicle can be ensured even in such a case.

[0181] (Structure 9)

[0182] The control system of structure 5 or structure 6 includes a door control prohibition unit. When an external person, identified by the authentication unit as the registered user, stops while approaching the vehicle, the door control prohibition unit prohibits the door control unit from controlling the door. After the door control is prohibited, if the third detection unit detects a pre-registered registration gesture, the door control prohibition unit releases the prohibition of door control. After the prohibition of door control is released, the door control unit controls the door based on the registration gesture detected by the third detection unit.

[0183] According to the control system of Structure 9, even if the registered user stops, the door can be controlled based on the detected registered gesture if the registered gesture is detected. Therefore, users other than the registered user can control the door through registered gestures. Thus, the convenience of vehicle users in controlling the vehicle's doors can be improved.

[0184] (Structure 10)

[0185] A control system of any one of structures 1 to 9, wherein, when the person outside the vehicle, who is identified by the authentication unit as the registered user, leaves the vehicle during the process of approaching the vehicle, the door control unit locks the door.

[0186] According to the control system of structure 10, the door is locked when a registered user leaves the vehicle while approaching it, thus ensuring the safety of the vehicle.

[0187] (Structure 11)

[0188] A control system of any one of structures 1 to 10, wherein the control system comprises: a fourth detection unit that detects the face of the person outside the vehicle detected by the first detection unit; and a door control prohibition unit that prohibits the door control unit from controlling the door when the fourth detection unit can no longer detect the face after detecting it.

[0189] According to the control system of structure 11, if it cannot be determined that there are registered users around the vehicle, the control of the door is prohibited, thus ensuring the safety of the vehicle.

[0190] (Structure 12)

[0191] A control method for a vehicle control system includes: acquiring an exterior image of the vehicle as an image of the vehicle's exterior; detecting a person outside the vehicle based on the acquired exterior image; detecting a manner of approaching the vehicle, i.e., an approach mode, for the detected person; comparing the detected approach mode with a reference approach mode; authenticating whether the detected person is a pre-registered user of the vehicle, i.e., a registered user; and controlling the vehicle's doors based on the comparison result if the person is authenticated as a registered user.

[0192] The control method of the control system of structure 12 achieves the same effect as the control system of structure 1.

Claims

1. A vehicle control system, comprising: The vehicle exterior image acquisition unit acquires an exterior image of the vehicle as an image of the exterior of the vehicle; A registered gesture database records the combination of gesture content and vehicle door control content for each registered gesture; and The registration department, based on the exterior images acquired by the exterior image acquisition unit, identifies the gesture content of the user of the vehicle, and records the identified gesture content in the registration gesture database by matching it with the door control content specified by the user of the vehicle.

2. The control system according to claim 1, wherein, A record in the registered gesture database contains gesture content information representing the gesture content and door control content information representing the control content of the vehicle's doors.

3. The control system according to claim 1 or 2, wherein, The registration department identifies the gestures made by the vehicle user by performing image analysis on the exterior images of the vehicle.

4. The control system according to any one of claims 1 to 3, wherein, The control content of the door includes information on identifying the door among a plurality of doors that becomes the controlled object.

5. The control system according to any one of claims 1 to 4, wherein, The control content of the door includes the opening action of the door, which is the object of control.

6. The control system according to any one of claims 1 to 5, wherein, The control system includes: The first detection unit detects persons outside the vehicle based on the exterior images acquired by the exterior image acquisition unit; and The authentication department verifies whether the person outside the vehicle detected by the first detection department is a pre-registered user of the vehicle.

7. The control system according to claim 6, wherein, The control system includes a third detection unit that detects the gestures of the person outside the vehicle detected by the first detection unit. The third detection unit tracks the image of the person outside the vehicle detected by the first detection unit in the image outside the vehicle acquired by the image outside the vehicle acquired by the image outside the vehicle acquired by the image outside the vehicle acquired by the first detection unit, performs image analysis on the image of the person outside the vehicle being tracked, extracts the features of the person outside the vehicle's movements, identifies the content of the person outside the vehicle's movements based on the extracted features of the person outside the vehicle's movements, and determines that a registered gesture has been detected if the identified content of the person outside the vehicle's movements is a gesture recorded in the registered gesture database.

8. The control system according to claim 6 or 7, wherein, The control system includes: The second detection unit detects the approach speed of the person outside the vehicle detected by the first detection unit as the approach method. The comparison unit compares the approach speed detected by the second detection unit with a reference approach speed; as well as Door control unit If the comparison result of the comparison unit indicates that the approach speed detected by the second detection unit is slower than the reference approach speed, the door control unit unlocks the door but does not open the door.

9. The control system according to claim 8, wherein, The control system includes a third detection unit that detects the gestures of the person outside the vehicle detected by the first detection unit. When the authentication unit authenticates the user as the registered user and the third detection unit detects a registered gesture registered in the registration gesture database after the door is unlocked, the door control unit controls the door according to the door control content corresponding to the registered gesture in the registration gesture database.

10. The control system according to claim 9, wherein, The control system includes a fourth detection unit that detects the face of the person outside the vehicle detected by the first detection unit. If the third detection unit stops registering gesture detection after detecting the face in the fourth detection unit, the third detection unit stops registering the gesture detection in the case where the face cannot be detected in the fourth detection unit.

11. The control system according to any one of claims 8 to 10, wherein, The control system includes a third detection unit that detects the gestures of the person outside the vehicle detected by the first detection unit. When the third detection unit detects an unregistered gesture from an external person who is identified by the authentication unit as the registered user, the door control unit opens the door to a predetermined opening degree.

12. A method for controlling a vehicle, wherein, Obtain exterior images of the vehicle as images of its exterior. Based on the acquired images of the vehicle exterior, the gesture content of the user's gestures is identified. The recognized gesture content is matched with the door control content of the vehicle specified by the user and recorded in the gesture registration database.

13. The control method according to claim 12, wherein, The process of recording in the registered gesture database includes the following steps: recording a record containing gesture content information representing the recognized gesture content and door control content information representing the door control content in the registered gesture database.

14. A recording medium having a program recorded thereon that causes a vehicle's processor to perform the following processes: Obtain exterior images of the vehicle as images of its exterior; Based on the external images of the vehicle, identify the gesture content of the user's gesture in the vehicle; The recognized gesture content is matched with the door control content of the vehicle specified by the user and registered in the gesture database.

Images