Apparatus, method, and computer-readable medium for controlling vehicle doors

Abstract

The present disclosure makes it possible to improve user convenience in vehicle door control. A door determining device (11) performs a first door determining process to determine a door to be controlled based on an image obtained by imaging a user who intends to enter a vehicle having a plurality of doors. The door controlling device (12) performs an operation on the door determined by the door determining device (11). The querying device (13) asks the user whether to permit the operation on the door determined by the door determining device (11). If the result of the query indicates refusal, the door controlling device (12) performs a process to cancel the operation performed on the door determined by the door determining device (11) and performs an operation on a door other than the determined door.

Description

Technical area

[0001] The present disclosure relates to a vehicle door control device, a vehicle door control method and a computer-readable medium. Background Art

[0002] Patent reference 1 describes an intelligent access system. In this system, a user carries an electronic key and approaches a vehicle. A device located inside the vehicle authenticates the electronic key and communicates wirelessly with it. Simultaneously, a camera inside the vehicle captures an image of the user carrying the electronic key. Inside the vehicle, the user directs their gaze toward an area corresponding to a door, which is to be selected from a variety of predefined areas. A computing device detects the user's gaze direction based on the image captured by the camera and determines the door selected by the user based on this detected gaze direction.

[0003] The computing device sends a signal to the user via a voice output device indicating that the door is ready to be operated. The voice output device 20 issues a confirmation message, such as "Is it okay to open the driver's side door?", to ask the user if it is okay to unlock and open the specific door. If the user has not diverted their gaze from the area they were looking at, a door release device unlocks the door selected by the user. If the user's gaze deviates from the area they were looking at, the computing device determines that the user has rejected the signal indicating that the door can be operated. In this case, the user unlocks the door manually by touching the handle of the door they wish to open. Citation list for patent literature

[0004] Patent literature 1: Publication No. 2008-2095 of the uncontested Japanese patent application Summary of the invention: Technical problem

[0005] In patent literature 1, the computing device sends a signal to the user indicating that the door is ready for operation, and the user can determine which door is to be operated by listening to the confirmation message. If the door identified by the computing device differs from the door desired by the user, the user can avoid operating the wrong door by redirecting their gaze. However, in patent literature 1, the door does not automatically unlock if the user rejects the signal indicating that the door is ready for operation, and the user must manually unlock the desired door by touching it. Therefore, it is desirable to further improve the user-friendliness of a device for controlling a vehicle door.

[0006] In view of the above circumstances, one objective of the present disclosure is to provide a vehicle door control device, a vehicle door control method and a computer-readable medium capable of improving the user's convenience in controlling the vehicle door. Solution to the problem

[0007] To achieve the aforementioned objective, the present disclosure provides, as a first aspect, a vehicle door control device. The vehicle door control device comprises: a door identification device for performing a first door identification process to determine a door to be controlled based on an image obtained by imaging a user intending to enter a vehicle with multiple doors; a door control device for performing an operation at the identified door; and a query device for querying the user as to whether they wish to allow the operation at the door. In a case where the result of the query indicates a refusal, the door control device performs an operation to abort the operation being performed at the identified door and performs an operation at a door other than the identified one.

[0008] The present disclosure provides, as a second aspect, a vehicle door control method. A vehicle door control method comprises: performing a first door determination process to identify a door to be controlled based on an image obtained by mapping a user intending to enter a vehicle with multiple doors; performing an operation at the identified door; querying the user to determine whether they wish to allow the operation at the door; in a case where the result of the query indicates a refusal, performing an operation to abort the operation being performed at the identified door; and performing an operation at a door other than the identified door.

[0009] The present disclosure provides a computer-readable medium as a third aspect. A non-transitory computer-readable medium stores a program to cause a processor to perform the following: performing a first door-determination process to identify a door to be controlled based on an image obtained by mapping a user intending to enter a vehicle with multiple doors; performing an operation at the identified door; querying the user to determine whether they wish to allow the operation at the door; in a case where the result of the query indicates a refusal, performing an operation to abort the operation being performed at the identified door; and performing an operation at a door other than the identified door. Advantageous effects of the invention

[0010] The vehicle door control device, the vehicle door control method and the computer-readable medium according to the present disclosure can improve user convenience in vehicle door control. Brief description of the drawings Fig. Figure 1 is a block diagram showing an example of a schematic configuration of a vehicle door control device according to the present disclosure. Fig. Figure 2 is a block diagram showing a vehicle door control device according to an embodiment of the present disclosure. Fig. Figure 3 is a flowchart illustrating the operating sequence of the vehicle control door device. Fig. Figure 4 is a block diagram showing a hardware configuration of the vehicle control device. Example of embodiment

[0011] Before describing exemplary embodiments of the present disclosure, an overview of the present disclosure will first be given. Fig. Figure 1 shows an example of a schematic configuration of a vehicle door control device according to the present disclosure. The vehicle door control device 10 comprises a door detection device 11, a door control device 12, and a query device 13.

[0012] The door identification device 11 performs an initial door identification process based on an image obtained by capturing a picture of a user intending to enter a vehicle and identifies a door to be controlled from among a multitude of vehicle doors. The door control device 12 performs an operation on the door identified by the door identification device 11. The query device 13 asks the user whether to allow the operation performed by the door control device 12 on the door. If the result of the query indicates a refusal, the door control device 12 performs an operation to cancel the operation performed on the door identified in the initial door identification process. Furthermore, the door control device 12 performs an operation on a different door than the one identified in the initial door identification process.

[0013] In the present disclosure, if the result of the query detected by the query device 13 indicates a refusal, the door control device 12 performs an operation to abort the operation performed on the door determined in the first door determination process and performs an operation on a different door than the one determined in the first door determination process. In the present disclosure, if the door determined in the first door determination process as the door to be controlled is not the door the user wishes to operate, the door control device 12 performs an operation on a different door. In this way, the user does not need to touch the vehicle, even if the door determined in the first door determination process as the door to be controlled is not the door the user wishes to operate.Therefore, the vehicle door control device 10 can improve the user's comfort when controlling the vehicle door.

[0014] The embodiments of this disclosure are described in detail below with reference to the drawings. Note that omissions and simplifications have been made in the following description and in the drawings for the sake of clarity. Furthermore, identical and similar elements in the drawings are identified by the same reference numerals, and redundant descriptions are omitted where necessary.

[0015] Fig. Figure 2 shows a vehicle door control device according to an embodiment of the present disclosure. The vehicle door control device 100 comprises an occupant detection unit 101, a facial authentication unit 102, a door identification unit 103, a door control unit 104, and a query unit 105. The vehicle door control device 100 can be physically configured as a single device comprising, for example, one or more memories and one or more processors. At least some of the functions of the units in the vehicle door control device 100 can be implemented by one or more processors operating according to an instruction read from one or more memories. The vehicle door control device 100 is, for example, mounted on a vehicle. The vehicle door control device 100 corresponds to the one described in Figure 2. Fig. 1 vehicle door control unit shown 10.

[0016] The occupant detection unit 101 receives a camera image from a camera 130. Camera 130 is a vehicle-mounted camera that captures an image of the area around the vehicle. Camera 130 captures an image of a person intending to enter the vehicle, i.e., an occupant. The vehicle occupant is also referred to as a user. Camera 130 can be installed, for example, in a B-pillar of the vehicle, i.e., in a pillar between a front door and a rear seat back.

[0017] Although in Fig. Although only one camera 130 is shown in Figure 2, the number of cameras 130 is not limited to one. The occupant detection unit 101 can acquire camera images from a variety of cameras 130. For example, the occupant detection unit 101 can acquire camera images from a camera that captures an image of an area to the left of the vehicle, a camera that captures an image of an area to the right of the vehicle, and a camera that captures an image of an area behind the vehicle.

[0018] The occupant detection unit 101 uses the camera image to identify the user intending to enter the vehicle. For example, the occupant detection unit 101 extracts a facial image or facial area of ​​a person near the vehicle from the camera image. Once the facial image or facial area is extracted, the occupant detection unit 101 determines that the user has been detected.

[0019] The facial authentication unit 102 authenticates the user recognized by the occupant detection unit 101. In the present embodiment, the facial authentication unit 102 authenticates the user by facial authentication using the camera image from the camera 130. For example, a facial image of a person who is authorized to enter the vehicle, or a set of features extracted from the facial image, is registered in the facial authentication unit 102. The facial authentication unit 102 compares a facial image contained in the camera image or a feature value extracted from the facial image with the registered facial image or feature value.The facial authentication unit 102 calculates a verification score that indicates the degree of similarity between the facial image contained in the camera image, or the feature set extracted from the facial image, and the registered facial image or feature set. The facial authentication unit 102 determines that the user's facial authentication is successful if the verification score is greater than or equal to a predetermined threshold.

[0020] The door identification unit 103 performs an initial door identification process to determine a controllable door based on the camera image. In this initial process, the door identification unit 103 detects, for example, at least one user position, one user line of sight, and one user gesture from the camera image. The door identification unit 103 then identifies a controllable door based on the detected user position, line of sight, and / or gesture. The door identification unit 103 can determine a controllable door using a combination of these factors.

[0021] In a case where the door to be controlled is determined based on the user's position, the door detection unit 103 recognizes the user's position using the camera image. For example, the door detection unit 103 determines whether the user is approaching the vehicle from the right or from the left. The door detection unit 103 can also determine whether the user is approaching the front or rear door of the vehicle. The door detection unit 103 has, for example, a table that specifies a correspondence relationship between a user's position and a door to be controlled. The door detection unit 103 can determine a door to be controlled according to a user's position using the table. Alternatively, the door detection unit 103 can determine the door that is closest to the user's position as the door to be controlled.

[0022] In a case where the door to be controlled is determined based on the user's line of sight, the door determination unit 103 detects the user's eyes from the image of the face area in the camera image, i.e., the face image, and determines the direction in which the user's eyes are directed. The direction of gaze can be determined using any known algorithm. Several areas are defined in the vehicle, corresponding to several doors 150. Based on the detected direction of gaze, the door determination unit 103 determines whether the user is looking at an area corresponding to a specific door. For example, if the user has looked at the area corresponding to a specific door for a predetermined time or longer, the door determination unit 103 can identify that specific door as the door to be controlled.

[0023] In a case where the door to be controlled is determined based on the user's gesture, the door identification unit 103 recognizes physical expressions such as posture, head movement, hand movement, and leg movement from the camera image. The door identification unit 103 has, for example, a table that specifies a correspondence relationship between a gesture and a door to be controlled. Using this table, the door identification unit 103 determines whether the user's physical expressions indicate a specific gesture associated with a particular door. If the user's physical expressions indicate the specific gesture, the door identification unit 103 identifies the door associated with the specific gesture as a door to be controlled. The door identification unit 103 can identify one or more doors 150 from the multitude of doors 150 present in the vehicle as doors to be controlled.The door determination unit 103 corresponds to the one in . Fig. 1 door identification device shown 11.

[0024] In a case where the user has been successfully authenticated by the facial authentication unit 102, the door control unit 104 performs an operation on the door identified by the door determination unit 103. For example, the door control unit 104 unlocks the door identified by the door determination unit 103. The door control unit 104 can both open and unlock the door. If the door is, for example, an electric sliding door, the door control unit 104 transmits a control signal to a control device, such as an electronic control unit (ECU), that controls the electric sliding door to control a motor that drives the electric sliding door. If the door is, for example, a hinged door, the door control unit 104 can open the door via a specific mechanism. The door control unit 104 corresponds to the one described in Fig. 1 door control device shown 12.

[0025] The query unit 105 asks the user whether they want to allow the operation of the door detected by the door detection unit 103. The query unit 105 notifies the user of the door to be operated, e.g., by voice, light, or similar means, and asks the user whether they want to allow the operation. The query unit 105 can notify the user of the door to be operated by instructing the door control unit 104 to carry out an operation determined by the door detection unit 103 to partially open the door.

[0026] Query unit 105 captures a user's response to the query as the query result. For example, query unit 105 captures the query result based on a camera image. Specifically, it can recognize a user gesture from the camera image and capture a query result corresponding to that gesture. For instance, if the user shakes their head vertically, query unit 105 captures "yes" as the query result. Conversely, if the user shakes their head sideways, query unit 105 captures "no" as the query result. Query unit 105 corresponds to the one in Fig. 1. Query methods shown 13.

[0027] In a case where the query received by query unit 105 returns a denial, door control unit 104 performs an operation to cancel the operation performed on the door identified as the control door in the initial door identification process. Furthermore, door control unit 104 performs an operation on a different door than the one identified as the control door in the initial door identification process. For example, door control unit 104 can perform an operation to close a door that was partially opened in the user query and open a different door. In a case where the query returns a positive response, door control unit 104 can open the door that was partially opened in the user query.

[0028] In a case where the query recorded by the query unit 105 indicates a refusal, the door identification unit 103 can perform a second door identification process based on a camera image and again identify a door to be controlled. In the second door identification process, the door identification unit 103, for example, captures at least one of the user's positions, the user's line of sight, and one of the user's gestures from the camera image. Based on the captured user position, the user's line of sight, and / or the user's gesture, the door identification unit 103 identifies a door to be controlled again. In the second door identification process, the door identification unit 103 can identify a different door to be controlled than the one identified in the first door identification process. The door control unit 104 then performs an operation on the door identified as the door to be controlled in the second door identification process.

[0029] The first and second door determination processes can exhibit different correspondence relationships between the user's position, line of sight, or gesture and the door to be controlled. For example, the operation performed by the user to determine a specific door in the first door determination process may differ from the operation performed by the user to determine the door in the second door determination process. In the second door determination process, the door determination unit 103 can detect the user's line of sight and gesture in a shorter time than in the first door determination process. It is assumed that the second door determination process is performed in a state where the user is closer to the vehicle than during the first door determination process.In a case where the user's line of sight and gesture are detected in a shorter time, it is possible to shorten the time the user waits until another door is operated.

[0030] In the first door identification process, the door identification unit 103 can identify a first candidate door and a second candidate door as candidates for the door to be controlled and designate the first candidate door as the door to be controlled. In this case, if the result of the query recorded by the query unit 105 indicates a rejection, the door identification unit 103 can designate the second candidate door as the door to be controlled without carrying out the second door identification process.

[0031] In the present embodiment, the door identification unit 103 can perform the first door identification process once the distance between the user and the vehicle reaches a certain threshold. In other words, if the user approaches the vehicle within this threshold, the door identification unit 103 can perform the first door identification process to determine which door to control. Similarly, the facial authentication unit 102 can authenticate the user by facial authentication once the distance between the user and the vehicle is within this threshold. The determination of the door to be controlled by the door identification unit 103 and the facial authentication by the facial authentication unit 102 can be performed in parallel.

[0032] Furthermore, the door control unit 104 can operate the door after the distance between the user and the vehicle falls within a second distance that is shorter than the first distance. In addition, before the door control unit 104 operates the door to be controlled, the query unit 105 can inform the user about the door that was identified as the object to be controlled in the first door determination process and ask the user whether they wish to allow the door to be operated. That is, the query unit 105 can ask the user whether they should allow the door to be operated if the distance between the user and the vehicle is greater than the second distance.

[0033] For example, the door detection unit 103 identifies a door to be controlled, and the facial authentication unit 102 performs facial authentication when the user approaches the vehicle to within 5 m. Upon querying, the door control unit 104 can partially open the door identified by the door detection unit 103.

[0034] The door control unit 104 operates the door when the user approaches the vehicle to within 1 meter. In this case, the determination of the door to be controlled, facial authentication, and acquisition of the query result can be completed before the user approaches the vehicle, and the door can be operated after the user has approached the vehicle sufficiently.

[0035] In a case where query unit 105 receives a query result indicating that the user is refusing the operation at the door, and the operation at the door determined in the first door determination process has already been performed, door control unit 104 performs an operation to cancel the operation already performed. For example, door control unit 104 performs an operation to close the door that was partially opened in the query. Door determination unit 103 performs a second door determination process to determine a door to be controlled. Door control unit 104 then performs an operation on the door determined in the second door determination process.

[0036] Next, an operating procedure will be described. Fig. Figure 3 shows an operating sequence of the vehicle door control device 100. The operating sequence of the vehicle door control device 100 corresponds to a vehicle door control procedure. The occupant detection unit 101 detects a user near the vehicle based on a camera image from camera 130 (step S1). The facial authentication unit 102 performs facial authentication on the user detected in step S1 (step S2). The door determination unit 103 performs an initial door determination process to identify a door to be controlled (step S3). In step S3, the door determination unit 103 can detect a line of sight or a gesture from the user and determine a door to be controlled based on the detected line of sight or gesture.

[0037] The door control unit 104 performs an operation on the door as determined in step S3 (step S4). The query unit 105 asks the user whether they want to allow the operation determined in step S3 at the door (step S5). For example, in step S4, the query unit 105 can cause the door control unit 104 to perform an operation to partially open the door determined in step S3 and inform the user about the door determined in step S3. In step S5, the query unit 105 can ask the user whether they want to allow the operation at the door by outputting a message such as "Is it okay to open the driver's side door?" via a speaker.

[0038] The user performs an action that is a response to the request. If the door the user wants to open is the one identified in step S3, the user submits a request to confirm the operation at the door. If the door the user wishes to open is different from the one identified in step S3, the user submits a request to reject the operation at the door. The query unit 105 determines whether the user has rejected the operation at the door (step S6). If the user confirms the operation at the door, the operation is terminated. If the user confirms the operation at the door, the door control unit 104 can perform an operation to open the partially opened door.

[0039] If the user refuses the operation at the door, the user performs an operation to determine a different door. In a case where the user refuses the operation at the door, the door determination unit 103 performs a second door determination process to determine a door to be controlled again (step S7). A request indicating the refusal as a query result and a request to determine a different door can be executed simultaneously. Furthermore, in a case where the user makes a request to determine a different door, the query unit 105 can determine in step S6 that the user refuses the operation at the door determined in the first door determination process. The door control unit 104 performs an operation to cancel the operation performed in step S4 and executes the operation at the door determined in step S7 (step S8).For example, in step S8 the door control unit 104 can close the door that is half open in the query and open the door specified in step S7.

[0040] In the present embodiment, the door detection unit 103, when a resident is detected, performs a first door detection process to determine a door to be controlled. The door control unit 104 performs an operation on the door identified in the first door detection process. The query unit 105 prompts the user to allow the operation on the door identified in the first door detection process. The door detection unit 103 performs a second door detection process to again identify a door to be controlled. If the query result indicates a rejection, the door control unit 104 performs an operation to abort the operation on the door identified in the first door detection process and performs an operation on the door identified in the second door detection process.In this way, the vehicle door control device 100 can perform an operation on a different door according to the user's response to the request, even if the door identified as the door to be controlled in the first door determination process is not the door the user wishes to operate. In the present embodiment, the user can operate the door without touching the vehicle, thus increasing the user's convenience in controlling the vehicle door.

[0041] Next, a hardware configuration of the vehicle door control unit 100 will be described. Fig.Figure 4 shows an example of a hardware configuration of the vehicle door control device 100. The vehicle door control device 100 comprises a processor (central processing unit (CPU)) 501, a read-only memory (ROM) 502, and a random-access memory (RAM) 503. In the vehicle door control device 100, the processor 501, the ROM 502, and the RAM 503 are interconnected via a bus 504. The vehicle door control device 100 may include other circuitry, such as a peripheral circuit, a communication circuit, and an interface circuit, which are not shown.

[0042] The ROM 502 is a non-volatile memory. For example, a semiconductor memory such as flash memory with a relatively small capacity is used for the ROM 502. The ROM 502 stores a program executed by the processor 501.

[0043] The program described above contains a set of instructions (or software code) that, when read by the computer, cause it to perform one or more of the functions described in the exemplary embodiments. The program may be stored on a non-transient, computer-readable medium or a physical storage medium. The computer-readable or physical storage medium may be, for example, RAM, ROM, flash memory, a solid-state drive (SSD) or other storage technology, a compact disc (CD), a digital versatile disc (DVD), a Blu-ray disc (registered trademark) or other optical storage medium, a magnetic cartridge, magnetic tape, a magnetic disk storage device, or other magnetic storage device. The program may be transferred on a transient, computer-readable medium or a communication medium.The transitory, computer-readable medium or communication medium can be, for example, an electrical, optical, acoustic or other form of transmitted signal, without being limited to that.

[0044] RAM 503 is a volatile memory device. Various types of semiconductor memory devices, such as dynamic random access memory (DRAM) or static random access memory (SRAM), are used as RAM 503. RAM 503 can be used as an internal buffer for the temporary storage of data and similar information.

[0045] The processor 501 loads the program stored in ROM 502 into RAM 503 and executes the program. The functions of the units in the vehicle door control unit 100 can be implemented by the CPU 501, which executes the program.

[0046] It should be noted that the vehicle door control unit 100 is not necessarily a single device. The vehicle door control unit 100 can consist of a number of spatially separated devices. Furthermore, the vehicle door control unit 100 does not necessarily have all the functions that are mounted on the vehicle. For example, at least one of the door identification unit 103 and the facial authentication unit 102 in the vehicle door control unit 100 may be located outside the vehicle. In this case, a functional unit located inside the vehicle and a functional unit located outside the vehicle can send and receive information via a wireless communication network.

[0047] Although the exemplary statements of the present disclosure have been described in detail above, the present disclosure is not limited to the above-mentioned exemplary statements, and the present disclosure also includes changes or modifications that have been made to the exemplary statements described above without deviating from the core of the present disclosure. Reference character list 10 VEHICLE DOOR CONTROL DEVICE 11 DOOR HOLDING AGENTS 12 DOOR CONTROL ITEMS 13 TESTING EQUIPMENT 100 VEHICLE DOOR CONTROL UNIT 101 OCCUPANT DETECTION UNIT 102 FACE AUTHENTICATION UNIT 103 DOOR LOCKING UNIT 104 DOOR CONTROL DEVICE 105 SURVEY UNIT 130 CAMERA 150 DOOR 501 PROCESSOR 502 ROM 503 RAM 504 BUS QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2008-2095

[0004]

Claims

[1] A vehicle door control device comprising the following: a door determination device for carrying out an initial door determination process to determine a door to be controlled on the basis of an image obtained by imaging a user intending to enter a vehicle with a plurality of doors; a door control device for performing an operation on the specified door; and a query tool that can ask the user whether they want to allow the process at the door, wherein, in a case where a query result indicates a refusal, the door control device performs an operation to cancel the operation being performed at the specified door and performs an operation at a door other than the specified door. [2] Vehicle door control unit according to claim 1, wherein In a case where the query result indicates a refusal, the door identification device performs a second door identification process to identify a door different from the one identified in the first door identification process as a control object based on the image, and The door control device performs an operation on the door determined in the second door determination process. [3] Vehicle door control device according to claim 2, wherein the door determination device detects at least one position of the user, one line of sight of the user and one gesture of the user from the image in the first door determination process and in the second door determination process and determines the door to be controlled on the basis of the detected at least one position of the user, the line of sight of the user and the gesture of the user. [4] Vehicle door control device according to claim 3, wherein a correspondence relationship between the user's position, the user's line of sight and / or the user's gesture and the door to be controlled in the first door determination process differs from a correspondence relationship between the user's position, the user's line of sight and / or the user's gesture and the door to be controlled in the second door determination process. [5] Vehicle door control unit according to one of claims 1 to 4, wherein the query device determines a result of the query based on the image. [6] Vehicle door control device according to any one of claims 1 to 5, wherein the query device informs the user about a door to be operated by causing the door control device to perform a process to half-open the door determined in the first door determination process. [7] Vehicle door control unit according to claim 6, wherein in a case where a result of the query indicates an affirmative response, the door control unit opens the half-open door. [8] Vehicle door control device according to claim 6 or 7, wherein in a case where a query result indicates a rejection, the door control device performs a process to close the half-open door. [9] Methods for controlling vehicle doors, comprising: Performing an initial door determination process to identify a door to be controlled based on an image obtained by mapping a user intending to enter a vehicle with multiple doors; the performance of an operation at the specific door; User prompt to allow the process at the door; in a case where the query result indicates a refusal to perform an operation, to cancel the operation performed at the specific door; and Performing a procedure at a different door than the identified door. [10] A non-transferable, computer-readable medium that stores a program to instruct a processor to perform the following: Performing an initial door determination process to determine a door to be controlled based on an image obtained by mapping a user intending to enter a vehicle with multiple doors; the performance of an operation at the specific door; User prompt to allow the process at the door; in a case where the query result indicates a refusal to perform an operation, to cancel the operation performed at the specific door; and Performing a procedure at a different door than the identified door.

Images