Vehicle control device and vehicle control method

By using a low-resolution camera and a high-resolution camera in the vehicle control system, combining object detection sensors, controlling the status of the equipment based on illuminance and detection signals, the problem of high power consumption of the camera in the intelligent entry system is solved, and high-precision object authentication is achieved and power management.

CN119078731APending Publication Date: 2024-12-06HONDA MOTOR CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411315212.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2021-03-02
Filing Date
2022-02-10
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing intelligent entry system uses the camera to identify the object, resulting in increased power consumption of the camera and inefficient problems.

Method used

Two cameras and object detection sensors with different resolutions are adopted. The first camera with low resolution is used for conventional shooting, and the second camera with high resolution is used for facial authentication. In combination with the object detection sensor, the camera is controlled according to the illuminance and detection signal. and the working state of the sensor to reduce power consumption.

Benefits of technology

Through efficient power management, the on-board cameras are used to verify the objects around the vehicle, reducing overall power consumption and improving the energy efficiency of the system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119078731A_ABST
    Figure CN119078731A_ABST
Patent Text Reader

Abstract

The invention provides a vehicle control device and a vehicle control method, which can suppress the power consumption of a vehicle-mounted camera, and can authenticate a target person in the periphery of a vehicle with high precision through a captured image of the vehicle-mounted camera. A vehicle control device is provided with: a peripheral object recognition unit that recognizes an object present in the periphery of a vehicle on the basis of a detection signal from an object detection sensor; a person determination unit that, when the object is recognized, captures an image of the object by means of a first camera having a first resolution, and executes a person determination process for determining whether the object is a person; and a user authentication unit that, when the object is determined to be a person, captures a first object determined to be a person by the person determination unit with a second camera having a second resolution higher than the first resolution, and performs a first user authentication process of comparing a face image of the first object with a face image of a registered user of the vehicle, and authenticating the first user by using a second camera having a second resolution higher than the first resolution. It is determined whether or not the first subject is a registered user.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application of the invention patent application with application date of February 10, 2022, application number 202210124391.2, and invention name “Vehicle Control Device and Vehicle Control Method”. Technical Field

[0002] The invention relates to a vehicle control device and a vehicle control method. Background Art

[0003] Prior art literature

[0004] In the past, the following smart entry system has been disclosed: the periphery of the vehicle is photographed by a vehicle-mounted camera, and the iris information of the subject person extracted from the image of the subject person approaching the vehicle is used to authenticate the user, thereby controlling entry into the vehicle (for example, refer to Patent Document 1).

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2003-138817 Summary of the invention

[0007] Problem that the invention aims to solve

[0008] When identifying a person approaching the vehicle through images captured by a camera, as in the above-mentioned smart entry system, the camera needs to constantly capture images and identify the person based on the captured images to prevent the person from approaching. Therefore, there is a disadvantage that the power consumption of the camera increases.

[0009] The present invention has been completed in view of this background, and its purpose is to provide a vehicle control device and a vehicle control method that can suppress the power consumption of a vehicle-mounted camera and accurately authenticate a target person existing around the vehicle through the camera image of the vehicle-mounted camera.

[0010] Means used to solve problems

[0011] As a first method for achieving the above-mentioned purpose, a vehicle control device is cited, which is installed in a vehicle having multiple object detection sensors, lighting, a first camera and a second camera, wherein the vehicle control device comprises: a surrounding object recognition unit, which recognizes the illumination based on the image captured by the first camera, and determines whether it is night or day. During the period when it is determined that it is night, each of the object detection sensors is always set to an operating state, and objects existing around the vehicle are recognized based on the detection signals of each of the object detection sensors; a human judgment unit, which, when the surrounding object recognition unit recognizes that an object exists around the vehicle based on the detection signals of any of the object detection sensors, turns on the first camera and the lighting corresponding to the object detection sensor that detected the object, and through The first camera corresponding to the object detection sensor that detects the object is turned on to photograph the object and perform a human determination process for determining whether the object is a person based on the image of the object; and a user authentication unit, which, when the object is determined to be a person by the human determination unit, photographs the first subject determined to be a person by the human determination unit through the second camera corresponding to the object detection sensor that detects the object, and performs the following first user authentication process: comparing the facial image of the first subject with the facial image of the registered user of the vehicle to determine whether the first subject is the registered user, the resolution of each of the second cameras is higher than the resolution of the corresponding first camera, and the power consumption of each of the object detection sensors is less than the power consumption of the corresponding first camera.

[0012] In the above-mentioned vehicle control device, it can also be configured that the vehicle control device has an object detection sensor control unit, which, when the proportion of the objects determined as humans by the human determination unit among the multiple objects identified by the surrounding object recognition unit is below a specified value, performs at least one of a process of reducing the detection sensitivity of the object detection sensor to objects, a process of narrowing the detection range of the object detection sensor, and a process of prohibiting the operation of the object detection sensor within a specified time.

[0013] In the above-mentioned vehicle control device, it can also be configured that, when the user authentication unit determines through the first user authentication process that the first object person is not the registered user for more than a predetermined number of times, the user authentication unit prohibits the execution of the first user authentication process until the object determined to be a person cannot be extracted from the image captured by the first camera through the human determination process.

[0014] In the above-mentioned vehicle control device, it can also be configured that, when the user authentication unit determines through the first user authentication processing that the first object person is not the registered user for more than a prescribed number of times, the operation of the object detection sensor is stopped, and when the operation of the touch sensor set on the door of the vehicle is identified, the second object person who operates the touch sensor is photographed by the second camera, and the following second user authentication processing is performed: the facial image of the second object person and the facial image of the registered user are compared to determine whether the second object person is the registered user.

[0015] In the vehicle control device, the user authentication unit may be configured to transmit a predetermined notification to a user terminal used by the registered user when the user authentication unit determines that the second target person is not the registered user through the second user authentication process.

[0016] In the vehicle control device, the user authentication unit may be configured to stop the operation of the object detection sensor when the user authentication unit determines through the first user authentication process that the first target person existing around the vehicle is not the registered user after the vehicle stops.

[0017] In the vehicle control device, the user authentication unit may continue the operation of the object detection sensor when the user authentication unit determines through the first user authentication process that the first target person existing around the vehicle is the registered user after the vehicle stops.

[0018] In the above-mentioned vehicle control device, it can also be configured that when the user authentication unit recognizes that the registered user has got off the vehicle through the image captured by the second camera, the object detection sensor stops working when the registered user cannot be recognized through the image captured by the second camera thereafter.

[0019] As a second method for achieving the above-mentioned purpose, a vehicle control method is cited, which is used in a vehicle having multiple object detection sensors, lighting, a first camera and a second camera, and is executed by a computer, wherein the vehicle control method includes the following steps: a surrounding object recognition step, identifying the illumination based on the image captured by the first camera, judging whether it is night or day, and during the period when it is judged that it is night, always setting each of the object detection sensors to an operating state, and identifying objects existing around the vehicle based on the detection signals of each of the object detection sensors; a human judgment step, when an object existing around the vehicle is recognized based on the detection signals of any of the object detection sensors through the surrounding object recognition step, turning on the first camera and the lighting corresponding to the object detection sensor that detected the object. The light is a light, the first camera corresponding to the object detection sensor that detects the object is turned on to photograph the object, and a human determination process is performed to determine whether the object is a person based on the image of the object; and a user authentication step, when the object is determined to be a person by the human determination step, the first subject person determined to be a person by the human determination step is photographed by the second camera corresponding to the object detection sensor that detects the object, and the following first user authentication process is performed: the facial image of the first subject person and the facial image of the registered user of the vehicle are compared to determine whether the first subject person is the registered user, the resolution of each of the second cameras is higher than the resolution of the corresponding first camera, and the power consumption of each of the object detection sensors is less than the power consumption of the corresponding first camera.

[0020] As a third method for achieving the above-mentioned purpose, a vehicle control device is cited, wherein the vehicle control device comprises: a surrounding object recognition unit, which recognizes objects existing in the vicinity of the vehicle based on a detection signal of an object detection sensor mounted on the vehicle and detecting objects existing in the vicinity of the vehicle; a human determination unit, which, when the surrounding object recognition unit recognizes the object, photographs the object by using a first camera mounted on the vehicle and having a first resolution, and performs human determination processing to determine whether the object is a person based on the image of the object; and a user authentication unit, which, when the human determination unit determines that the object is a person, performs human determination processing by using a first camera mounted on the vehicle and having a first resolution. The vehicle is equipped with a second camera having a second resolution higher than the first resolution, photographing a first subject person determined to be a person by the person determination unit, and performing the following first user authentication processing: comparing a facial image of the first subject person with a facial image of a registered user of the vehicle to determine whether the first subject person is the registered user, and if the user authentication unit determines that the first subject person is not the registered user for more than a predetermined number of times through the first user authentication processing, prohibiting execution of the first user authentication processing until no subject determined to be a person is extracted from the image captured by the first camera through the person determination processing.

[0021] As a fourth method for achieving the above-mentioned purpose, a vehicle control device is cited, wherein the vehicle control device comprises: a surrounding object recognition unit, which recognizes objects existing in the vicinity of the vehicle based on a detection signal of an object detection sensor mounted on the vehicle and detecting objects existing in the vicinity of the vehicle; a human determination unit, which, when the object is recognized by the surrounding object recognition unit, photographs the object by means of a first camera mounted on the vehicle and having a first resolution, and performs human determination processing based on the image of the object to determine whether the object is a person; and a user authentication unit, which, when the object is determined to be a person by the human determination unit, photographs the object by means of a second camera mounted on the vehicle and having a second resolution higher than the first resolution. The first object person is a person, and the following first user authentication processing is performed: the facial image of the first object person is compared with the facial image of the registered user of the vehicle, so as to determine whether the first object person is the registered user. When the user authentication unit determines that the first object person is not the registered user for more than a predetermined number of times through the first user authentication processing, the object detection sensor is stopped, and when the operation of the touch sensor provided on the door of the vehicle is recognized, the second object person who operates the touch sensor is photographed by the second camera, and the following second user authentication processing is performed: the facial image of the second object person is compared with the facial image of the registered user, so as to determine whether the second object person is the registered user.

[0022] As a fifth method for achieving the above-mentioned purpose, a vehicle control device is cited, wherein the vehicle control device comprises: a surrounding object recognition unit, which recognizes objects existing in the vicinity of the vehicle based on a detection signal of an object detection sensor mounted on the vehicle and detecting objects existing in the vicinity of the vehicle; a human determination unit, which, when the object is recognized by the surrounding object recognition unit, photographs the object by a first camera mounted on the vehicle and having a first resolution, and performs human determination processing based on the image of the object to determine whether the object is a person; and a user authentication unit, which, when the object is recognized by the human determination unit, performs human determination processing based on the image of the object to determine whether the object is a person. When it is determined that the object is a person, a first object person determined to be a person by the person determination unit is photographed by a second camera mounted on the vehicle and having a second resolution higher than the first resolution, and the following first user authentication processing is performed: a facial image of the first object person and a facial image of a registered user of the vehicle are compared to determine whether the first object person is the registered user, and when the user authentication unit determines through the first user authentication processing that the first object person existing around the vehicle is not the registered user after the vehicle stops, the operation of the object detection sensor is stopped.

[0023] As a sixth method for achieving the above-mentioned purpose, a vehicle control device is cited, wherein the vehicle control device comprises: a surrounding object recognition unit, which recognizes objects existing in the vicinity of the vehicle based on a detection signal of an object detection sensor mounted on the vehicle and detecting objects existing in the vicinity of the vehicle; a human determination unit, which, when the surrounding object recognition unit recognizes the object, photographs the object through a first camera mounted on the vehicle and having a first resolution, and performs human determination processing based on the image of the object to determine whether the object is a person; and a user authentication unit, which, when the human determination unit determines that the object is a person, When the object is a person, a second camera mounted on the vehicle and having a second resolution higher than the first resolution is used to photograph a first subject person determined to be a person by the person determination unit, and the following first user authentication processing is performed: a facial image of the first subject person and a facial image of a registered user of the vehicle are compared to determine whether the first subject person is the registered user, and when the user authentication unit recognizes that the registered user has gotten off the vehicle through the image captured by the second camera, the user authentication unit stops the operation of the object detection sensor when the registered user cannot be recognized through the image captured by the second camera thereafter.

[0024] Effects of the Invention

[0025] According to the above-mentioned vehicle control device, the object detection sensor and the first camera are used in combination with the second camera with high resolution. The object detection sensor consumes less power than the first camera and the second camera because it only needs to perform the process of detecting the object. The first camera consumes less power than the second camera due to its low resolution. Therefore, the power consumption of the vehicle-mounted camera can be suppressed, and the object person existing around the vehicle can be authenticated with high precision through the camera image of the vehicle-mounted camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a structural diagram of a vehicle equipped with a vehicle control device.

[0027] Figure 2 It is a structural diagram of a vehicle control device.

[0028] Figure 3 This is the first flowchart of the vehicle entry process.

[0029] Figure 4 This is the second flowchart of the vehicle entry process.

[0030] Description of Reference Numerals

[0031] 1… vehicle, 2… driver's door, 10… vehicle control device, 20… processor, 21… surrounding object recognition unit, 22… human determination unit, 23… user authentication unit, 24… object detection sensor control unit, 25… control unit, 30… memory, 31… control program, 40a… first front camera, 40b… second front camera, 41a… first right camera, 41b… second right camera, 42a… first left camera, 42b… second left camera, 43a… first rear camera, 43b… second rear camera, 44… front human sensing sensor, 45… right human sensing sensor, 46… left human sensing sensor, 47… rear human sensing sensor, 55… touch sensor, U… object person. DETAILED DESCRIPTION

[0032] [1. Vehicle structure]

[0033] Reference Figure 1 The structure of the vehicle 1 equipped with the vehicle control device 10 of the present embodiment is described. The front part of the vehicle 1 has a first front camera 40a and a second front camera 40b for photographing the front of the vehicle 1. The resolution of the second front camera 40b is higher than that of the first front camera 40a. In addition, the power consumption of the second front camera 40b is greater than that of the first front camera 40a.

[0034] In addition, at the front of the vehicle, there are a front human sensing sensor 44 and a front illumination LED 60. The front human sensing sensor 44 detects an object existing in front of the vehicle 1. The front human sensing sensor 44 is a PIR (Passive Infrared Ray) sensor that detects infrared rays emitted from an object and detects objects such as people or small animals. The front human sensing sensor 44 consumes less power than the first front camera 40a. The front illumination LED 60 is used when taking pictures at night based on the first front camera 40a and the second front camera 40b. The front illumination LED 60 illuminates within the shooting range of the first front camera 40a and the second front camera 40b.

[0035] The rear portion of the vehicle 1 includes a first rear camera 43a and a second rear camera 43b for photographing the rear of the vehicle 1. The resolution of the second rear camera 43b is higher than that of the first rear camera 43a. In addition, the power consumption of the second rear camera 43b is greater than that of the first rear camera 43a.

[0036] In addition, the rear portion of the vehicle 1 is provided with a rear human sensing sensor 47 and a rear illumination LED 63. The rear human sensing sensor 47 detects an object existing behind the vehicle 1. The rear human sensing sensor 47 is a PIR sensor, and the power consumption of the rear human sensing sensor 47 is less than that of the first rear camera 43a. The rear illumination LED 63 is used when taking pictures at night based on the first rear camera 43a and the second rear camera 43b. The rear illumination LED 63 illuminates within the shooting range of the first rear camera 43a and the second rear camera 43b.

[0037] The right side of the vehicle 1 includes a first right side camera 41a and a second right side camera 41b for photographing the right side of the vehicle 1. The resolution of the second right side camera 41b is higher than that of the first right side camera 41a. In addition, the power consumption of the second right side camera 41b is greater than that of the first right side camera 41a. In addition, the right side of the vehicle 1 includes a welcome light 50 that illuminates a guide display 51 toward the driver's door 2 onto the road surface near the driver's door 2. In addition, the door handle (not shown) of the driver's door 2 includes a touch sensor 55. The touch sensor 55 may be provided on the window frame of the driver's door 2, or may be provided near the periphery of the first right side camera 41a or the second right side camera 41b.

[0038] In addition, the right side of the vehicle 1 is provided with a right side human sensing sensor 45 and a right side illumination LED 61. The right side human sensing sensor 45 detects an object existing on the right side of the vehicle 1. The right side human sensing sensor 45 is a PIR sensor, and the power consumption of the right side human sensing sensor 45 is less than that of the first right side camera 41a. The right side illumination LED 61 is used when shooting at night based on the first right side camera 41a and the second right side camera 41b. The right side illumination LED 61 illuminates within the shooting range of the first right side camera 41a and the second right side camera 41b.

[0039] The left side of the vehicle 1 has a first left camera 42a and a second left camera 42b for photographing the left side of the vehicle 1. The resolution of the second left camera 42b is higher than that of the first left camera 42a. In addition, the power consumption of the second left camera 42b is greater than that of the first left camera 42a.

[0040] In addition, the left side of the vehicle 1 is provided with a left human sensor 46 and a left illumination LED 62. The left human sensor 46 detects an object existing on the left side of the vehicle 1. The left human sensor 46 is a PIR sensor, and the power consumption of the left human sensor 46 is less than that of the first left camera 42a. The left illumination LED 62 is used when shooting at night based on the first left camera 42a and the second left camera 42b. The left illumination LED 62 illuminates within the shooting range of the first left camera 42a and the second left camera 42b.

[0041] The first front camera 40a, the first right camera 41a, the first left camera 42a, and the first rear camera 43a are equivalent to the first camera of the present disclosure. The resolutions of the first front camera 40a, the first right camera 41a, the first left camera 42a, and the first rear camera 43a are equivalent to the first resolution of the present disclosure. The second front camera 40b, the second right camera 41b, the second left camera 42b, and the second rear camera 43b are equivalent to the second camera of the present disclosure. The resolutions of the second front camera 40b, the second right camera 41b, the second left camera 42b, and the second rear camera 43b are equivalent to the second resolution of the present disclosure.

[0042] Furthermore, the vehicle 1 includes a communication unit 80. The communication unit 80 communicates with a portable terminal used by pedestrians around the vehicle 1, other vehicles, an external server, a roadside device, and the like.

[0043] The vehicle control device 10 identifies the object person U existing around the vehicle 1 based on the detection signals obtained by the front human sensing sensor 44, the right human sensing sensor 45, the left human sensing sensor 46 and the rear human sensing sensor 47, and the captured images obtained by the first front camera 40a, the second front camera 40b, the first right camera 41a, the second right camera 41b, the first left camera 42a, the second left camera 42b, the first rear camera 43a and the second rear camera 43b.

[0044] When the vehicle control device 10 authenticates that the subject person U is a pre-registered user of the vehicle 1 (registered user), the vehicle control device 10 executes entry processing for entering the vehicle 1 , such as lighting the welcome light 50 and unlocking the driver's door 2 .

[0045] [2. Structure of vehicle control device]

[0046] Reference Figure 2The structure of the vehicle control device is described. The captured images obtained by the first front camera 40a, the second front camera 40b, the first right camera 41a, the second right camera 41b, the first left camera 42a, the second left camera 42b, the first rear camera 43a, and the second rear camera 43b are input to the vehicle control device 10. In addition, the detection signals of the objects obtained by the front human sensing sensor 44, the right human sensing sensor 45, the left human sensing sensor 46, and the rear human sensing sensor are input to the vehicle control device 10.

[0047] The lighting and extinguishing of the welcome lamp 50, the front illumination LED 60, the right illumination LED 61, the left illumination LED 62, and the rear illumination LED 63, and the operation of the locking mechanism of each door (not shown) are controlled by the control signal output from the vehicle control device 10. In addition, the vehicle control device 10 communicates with other vehicles and the like via the communication unit 80.

[0048] The vehicle control device 10 is a control unit composed of a processor 20, a memory 30, and an interface circuit (not shown). The processor 20 is equivalent to the computer of the present disclosure. The processor 20 reads and executes a control program 31 stored in the memory 30, thereby functioning as a surrounding object recognition unit 21, a human determination unit 22, a user authentication unit 23, an object detection sensor control unit 24, and a control unit 25.

[0049] The processing performed by the surrounding object recognition unit 21 is equivalent to the surrounding object recognition step in the vehicle control method of the present disclosure, and the processing performed by the human determination unit 22 is equivalent to the human determination step in the vehicle control method of the present disclosure. The processing performed by the user authentication unit 23 is equivalent to the user authentication step of the present disclosure, and the processing performed by the object detection sensor control unit 24 is equivalent to the object detection sensor control step of the present disclosure.

[0050] The surrounding object recognition unit 21 recognizes objects existing around the vehicle 1 based on detection signals obtained by the front human sensing sensor 44, the right human sensing sensor 45, the left human sensing sensor 46, and the rear human sensing sensor 47. The human determination unit 22 captures the object recognized by the surrounding object recognition unit 21 through the first front camera 40a, the first right camera 41a, the first left camera 42a, or the first rear camera 43a. Then, the human determination unit 22 performs the following human determination processing: based on the feature amount of the image of the object captured by the first front camera 40a, the first right camera 41a, the first left camera 42a, or the first rear camera 43a, it is determined whether the object is a person.

[0051] The user authentication unit 23 captures the first target person determined as a person by the person determination unit 22 through the second front camera 40b, the second right camera 41b, the second left camera 42b, or the second rear camera 43b. Then, the user authentication unit 23 performs the following first user authentication processing: the facial image of the first target person captured by the second front camera 40b, the second right camera 41b, the second left camera 42b, or the second rear camera 43b is compared with the facial images of the registered users of the vehicle 1 stored in the memory 30, thereby determining whether the target person is a registered user.

[0052] When the ratio of objects determined as people by the human determination unit 22 among the plurality of objects identified by the surrounding object recognition unit 21 is less than a predetermined value, the object detection sensor control unit 24 performs at least one of the following processes (1) to (3) on the front human sensing sensor 44, the right human sensing sensor 45, the left human sensing sensor 46, and the rear human sensing sensor 47. (1) Reducing the detection sensitivity of the object. (2) Processing to narrow the detection range of the object. (3) Prohibiting operation for a predetermined time.

[0053] When the user authentication unit 23 determines that the subject person is a registered user, the control unit 25 executes entry processing such as unlocking the door of the vehicle 1 corresponding to the position of the user to enter the vehicle 1. Figure 1 In the example shown, since the subject U is approaching the driver's door 2 , when determining that the subject U is a registered user, the control unit 25 executes an entry process of lighting the welcome lamp 50 and unlocking the driver's door 2 .

[0054] [3. Vehicle entry processing]

[0055] according to Figure 3-4 The flowchart shown in the figure shows how Figure 1 The vehicle entry process is executed by the vehicle control device 10 when a subject person U who is a registered user of the vehicle 1 approaches the driver's door 2 from the right side of the vehicle 1 .

[0056] Hereinafter, the front human sensing sensor 44, the right human sensing sensor 45, the left human sensing sensor 46, and the rear human sensing sensor 47 are collectively referred to as human sensing sensors. In addition, the first front camera 40a, the first right camera 41a, the first left camera 42a, and the first rear camera 43a are collectively referred to as the first camera. In addition, the second front camera 40b, the second right camera 41b, the second left camera 42b, and the second rear camera 43b are collectively referred to as the second camera. In addition, the front illumination LED 60, the right illumination LED 61, the left illumination LED 62, and the rear illumination LED 63 are collectively referred to as illumination LEDs.

[0057] exist Figure 3In step S1, the surrounding object recognition unit 21 recognizes the illumination around the vehicle 1 based on the image captured by the first camera. Specifically, the surrounding object recognition unit 21 calculates the average value of the gray level of each pixel in one captured image of the first camera. In the next step S2, the surrounding object recognition unit 21 determines whether the average value of the gray level exceeds a specified threshold. Then, the surrounding object recognition unit 21 causes the processing to enter step S20 when the average value of the gray level exceeds the specified threshold (when it can be determined to be nighttime), and causes the processing to enter step S3 when the average value of the gray level is below the specified threshold (when it can be determined to be daytime).

[0058] In addition, the surrounding object recognition unit 21 may determine whether it is nighttime or daytime based on the current time. Alternatively, the vehicle 1 may include an illuminance sensor, and the surrounding object recognition unit 21 may determine whether it is nighttime or daytime based on a detection signal of the illuminance sensor.

[0059] In step S20, the surrounding object recognition unit 21 always turns on (operates) the human sensor. In the next step S21, when the surrounding object recognition unit 21 recognizes an object through the detection signal of the human sensor, the processing proceeds to step S22. On the other hand, when the surrounding object recognition unit 21 does not recognize an object, the processing proceeds to step S23. Figure 4 In this case, the entry process of the vehicle 1 described later is not performed.

[0060] In steps S22 and S23, the human determination unit 22 turns on the first camera and the illumination LED corresponding to the human sensor that detected the object. Figure 1 In the example of , since the object person U is detected by the right human sensing sensor 45, the human determination unit 22 turns on the first right camera 41a and the right illumination LED 61. In the next step S24, the human determination unit 22 photographs the object (here, the object person U) through the first right camera 41a, and the process proceeds to step S6.

[0061] In step S3, the human determination unit 22 always turns on (operates) the first camera and keeps the human sensor turned off. In the next step S4, the human determination unit 22 captures the surroundings of the vehicle 1 through the first camera. In the next step S5, if the human determination unit 22 recognizes an object through the image captured by the first camera, the process proceeds to step S6. If the object is not recognized, the process proceeds to step S7. Figure 4 Step S15. Figure 1 In the example of , the subject person U is recognized as the object through the captured image of the first right camera 41a.

[0062] In step S6, the human determination unit 22 extracts the image portion of the object from the image captured by the first right camera 41a. In the next step S7, the human determination unit 22 determines whether the image portion of the object has human characteristics, thereby determining whether the object is a person. In the next step S8, if the human determination unit 22 determines that the object is a person, the processing enters the next step. Figure 4 Step S9, when it is determined that the object is not a person, the process enters Figure 4 Step S15.

[0063] exist Figure 4 In step S9, the user authentication unit 23 turns on the second right camera 41b, and in step S10, the second right camera 41b captures the subject U. In the next step S11, the user authentication unit 23 extracts the image portion of the subject U from the captured image of the second right camera 41b. In the next step S12, the user authentication unit 23 compares the facial image of the subject U with the facial images of the registered users stored in the memory 30, thereby determining whether the subject U is a registered user.

[0064] In the next step S13, the user authentication unit 23 proceeds to step S14 when determining that the subject U is a registered user, and proceeds to step S15 when determining that the subject U is not a registered user. In step S14, the control unit 25 turns on the welcome lamp 50 and unlocks the driver's door 2 as entry processing of the vehicle 1.

[0065] [4. Other embodiments]

[0066] In the above-mentioned embodiment, it is also possible that, when the user authentication processing of the user authentication unit 23 determines that the object person is not a registered user for more than a predetermined number of times, the user authentication processing is prohibited from being executed until the object determined to be a person by the person determination processing of the person determination unit 22 is no longer recognized by the surrounding object recognition unit 21.

[0067] In the above-mentioned embodiment, when the user authentication unit 23 determines that the subject is not a registered user for more than a predetermined number of times through the user authentication process, the operation of the human sensor is stopped, and when the operation of the touch sensor 55 is recognized, the subject (equivalent to the second subject of the present disclosure) who has operated the touch sensor 55 is photographed by the second right camera 41b, and the image portion of the subject is extracted from the photographed image of the second right camera 41b. Then, the user authentication unit 23 may perform a process of comparing the facial image of the subject with the facial image of the registered user and determining whether the subject is a registered user (second user authentication process). In addition, when the user authentication unit 23 determines that the subject is not a registered user through the second user authentication process, it may send a prescribed notification for notifying the user terminal used by the registered user of the presence of a suspicious person, etc.

[0068] In the above-mentioned embodiment, the user authentication unit 23 may stop the operation of the human sensor when it is determined through the user authentication process that the subject person existing around the vehicle 1 is not a registered user after the vehicle 1 stops. On the other hand, the user authentication unit 23 may continue the operation of the human sensor when it is determined through the user authentication process that the subject person existing around the vehicle 1 is a registered user after the vehicle 1 stops.

[0069] In the above embodiment, when the user authentication unit 23 recognizes that a registered user has gotten off the vehicle 1 through the image captured by the second camera, the human sensor may be stopped when the registered user cannot be recognized through the image captured by the second camera.

[0070] In the above embodiment, a PIR sensor is used as the object detection sensor of the present disclosure, but the object detection sensor of the present disclosure is not limited thereto, and any sensor may be used as long as it can detect the object with less power consumption than the first camera.

[0071] In the above-described embodiment, the object detection sensor control unit 24 is provided to perform processing for controlling the operating conditions of the human detection sensor. However, the object detection sensor control unit 24 may be omitted.

[0072] In the above-mentioned embodiment, the vehicle control method of the present disclosure is executed by the processor 20 (equivalent to the computer of the present disclosure) of the vehicle control device 10, but the vehicle control method of the present disclosure may also be executed by a computer or the like constituting an external server communicating with the vehicle 1. In this case, information about objects existing around the vehicle 1 is sent from the vehicle 1 to the external server, and control information for instructing the lighting or extinguishing of the welcome lamp 50, unlocking of the door of the vehicle 1, etc. is sent from the external server to the vehicle 1. In addition, it may be configured so that the surrounding object recognition unit 21, the human determination unit 22, the user authentication unit 23, the object detection sensor control unit 24, and the control unit 25 are distributed in the vehicle control device 10 and the external server.

[0073] in addition, Figure 1 , Figure 2 This is a schematic diagram showing the structure of the vehicle 1 and the vehicle control device 10 by distinguishing the main processing contents in order to facilitate understanding of the present invention. The structure of the vehicle 1 and the vehicle control device 10 may also be configured by other distinctions. In addition, the processing of each structural element may be performed by one hardware unit or by multiple hardware units. In addition, Figure 3 , Figure 4 The processing of each component of the flowchart shown may be executed by one program or by a plurality of programs.

[0074] [5. Structure supported by the above-mentioned embodiment]

[0075] The above-mentioned embodiment is a specific example of the following structure.

[0076] (Item 1) A vehicle control device, wherein the vehicle control device comprises: a surrounding object recognition unit, which recognizes objects existing in the vicinity of the vehicle based on a detection signal of an object detection sensor installed in the vehicle and detecting objects existing in the vicinity of the vehicle; a human determination unit, which, when the object is recognized by the surrounding object recognition unit, photographs the object by a first camera installed in the vehicle and having a first resolution, and performs a human determination process for determining whether the object is a person based on the image of the object; and a user authentication unit, which, when the object is determined to be a person by the human determination unit, photographs a first subject determined to be a person by the human determination unit by a second camera installed in the vehicle and having a second resolution higher than the first resolution, and performs the following first user authentication process: a facial image of the first subject is compared with a facial image of a registered user of the vehicle to determine whether the first subject is the registered user.

[0077] According to the vehicle control device of item 1, an object detection sensor and a first camera are used in combination with a second camera with a high resolution. The object detection sensor consumes less power than the first camera and the second camera because it only needs to perform processing for detecting objects. The first camera consumes less power than the second camera because of its low resolution. As a result, the power consumption of the vehicle-mounted camera can be suppressed, and the object person existing around the vehicle can be authenticated with high precision through the camera image of the vehicle-mounted camera.

[0078] (Item 2) The vehicle control device according to Item 1, wherein:

[0079] The vehicle control device includes an object detection sensor control unit, which, when the proportion of objects determined as humans by the human determination unit among the plurality of objects identified by the surrounding object recognition unit is below a specified value, executes at least one of a process of reducing the detection sensitivity of the object detection sensor to objects, a process of narrowing the detection range of the object detection sensor, and a process of prohibiting the operation of the object detection sensor within a specified time.

[0080] According to the vehicle control device of item 2, when it is assumed that an object other than a person such as a small animal is wandering around the vehicle 1, that is, when user authentication is not required, the operation of the object detection sensor is limited, thereby suppressing the power consumption for authenticating the registered user.

[0081] (Item 3) A vehicle control device according to Item 1 or Item 2, wherein, if the user authentication unit determines through the first user authentication process that the first object person is not the registered user for more than a predetermined number of times, the user authentication unit prohibits execution of the first user authentication process until no object determined to be a person is extracted from the image captured by the first camera through the human determination process.

[0082] According to the vehicle control device of item 3, when the vehicle 1 is parked in a place where there are many pedestrians and persons other than registered users frequently pass by the vehicle, the first user authentication process is prohibited, thereby suppressing the power consumption of the second camera.

[0083] (Item 4) A vehicle control device according to Item 1 or Item 2, wherein, when the user authentication unit determines through the first user authentication process that the first subject is not the registered user for more than a predetermined number of times, the operation of the object detection sensor is stopped, and when the operation of the touch sensor provided on the door of the vehicle is identified, the second subject who operates the touch sensor is photographed by the second camera, and the second user authentication process is performed as follows: the facial image of the second subject is compared with the facial image of the registered user to determine whether the second subject is the registered user.

[0084] According to the vehicle control device of item 4, when vehicle 1 is parked in a place where there are many pedestrians and persons other than registered users frequently pass by the vehicle, it is possible to reduce power consumption caused by the operation of the object detection sensor and the first camera while ensuring access to the vehicle through a second user authentication process corresponding to the operation of the touch sensor.

[0085] (Item 5) The vehicle control device according to Item 4, wherein the user authentication unit sends a predetermined notification to a user terminal used by the registered user when the user authentication unit determines through the second user authentication process that the second target person is not the registered user.

[0086] According to the vehicle control device of item 5, it is possible to notify the registered user that a target person other than the registered user is about to enter the vehicle.

[0087] (Item 6) A vehicle control device according to any one of Items 1 to 5, wherein the user authentication unit stops the operation of the object detection sensor when it is determined through the first user authentication process after the vehicle stops that the first object person existing around the vehicle is not the registered user.

[0088] According to the vehicle control device of item 6, when the vehicle is expected to stop at a place with many pedestrians because the first target person other than the registered user exists around the vehicle, the operation of the object detection sensor can be stopped to reduce power consumption.

[0089] (Item 7) The vehicle control device according to Item 6, wherein the user authentication unit continues the operation of the object detection sensor when it is determined through the first user authentication process that the first target person existing around the vehicle is the registered user after the vehicle stops.

[0090] According to the vehicle control device of item 7, when there is a registered user around the vehicle, the registered user can enter the vehicle, thereby improving the convenience of use for the registered user.

[0091] (Item 8) A vehicle control device according to any one of Items 1 to 7, wherein, when the user authentication unit recognizes that the registered user has gotten off the vehicle through the image captured by the second camera, the user authentication unit stops the operation of the object detection sensor when the registered user cannot be recognized through the image captured by the second camera thereafter.

[0092] According to the vehicle control device of item 8, the operation of the object detection sensor is stopped when the registered user who has gotten off the vehicle moves away from the vehicle to a certain extent, thereby reducing the power consumption of the object detection sensor.

[0093] (Item 9) A vehicle control method executed by a computer, wherein the vehicle control method includes the following steps: a peripheral object recognition step of recognizing objects existing in the periphery of the vehicle based on a detection signal of an object detection sensor mounted on the vehicle and detecting objects existing in the periphery of the vehicle; a human determination step of, when the object is recognized by the peripheral object recognition step, photographing the object by a first camera mounted on the vehicle and having a first resolution, and performing a human determination process of determining whether the object is a human based on the image of the object; and a user authentication step of, when the object is determined to be a human by the human determination step, photographing a first subject determined to be a human by the human determination step by a second camera mounted on the vehicle and having a second resolution higher than the first resolution, and performing the following first user authentication process: comparing the facial image of the first subject with the facial image of a registered user of the vehicle to determine whether the first subject is the registered user.

[0094] By executing the vehicle control method of item 9 by a computer, the same operational effects as those of the vehicle control device of item 1 can be obtained.

Claims

1. A vehicle control device, the vehicle control device being mounted in a vehicle having a plurality of object detection sensors, a lighting lamp, a first camera and a second camera, wherein: The vehicle control device comprises: a surrounding object recognition unit that recognizes illumination based on the image captured by the first camera, determines whether it is nighttime or daytime, and when it is determined that it is nighttime, always sets each of the object detection sensors to an operating state, and recognizes objects existing around the vehicle based on detection signals of each of the object detection sensors; a human determination unit that, when the surrounding object recognition unit recognizes an object existing in the vicinity of the vehicle based on a detection signal of any of the object detection sensors, turns on the first camera corresponding to the object detection sensor that has detected the object and the illumination light, photographs the object through the turned-on first camera corresponding to the object detection sensor that has detected the object, and performs human determination processing for determining whether the object is a human based on the image of the object; as well as a user authentication unit which, when the human determination unit determines that the object is a human, photographs a first object person determined by the human determination unit as a human through the second camera corresponding to the object detection sensor that detects the object, and performs the following first user authentication processing: comparing a facial image of the first object person with a facial image of a registered user of the vehicle to determine whether the first object person is the registered user, The resolution of each of the second cameras is higher than the resolution of the corresponding first camera, and the power consumption of each of the object detection sensors is lower than the power consumption of the corresponding first camera.

2. The vehicle control device according to claim 1, wherein: The vehicle control device includes an object detection sensor control unit, which performs at least one of a process of reducing the detection sensitivity of an object detection sensor mounted on the vehicle and detecting objects existing around the vehicle, a process of narrowing the detection range of the object detection sensor, and a process of prohibiting the operation of the object detection sensor for a specified time when the proportion of the objects determined as humans by the human determination unit among the plurality of objects identified by the surrounding object recognition unit is below a specified value.

3. The vehicle control device according to claim 1 or 2, wherein: When the user authentication unit determines through the first user authentication process that the first object person is not the registered user for more than a predetermined number of times, the user authentication unit prohibits execution of the first user authentication process until no object determined to be a person is extracted from the image captured by the first camera through the human determination process.

4. The vehicle control device according to claim 2, wherein: When the user authentication unit determines through the first user authentication process that the first subject is not the registered user for more than a specified number of times, the object detection sensor is stopped, and when the operation of the touch sensor provided on the door of the vehicle is recognized, the second subject who operates the touch sensor is photographed by the second camera, and the second user authentication process is performed as follows: a facial image of the second subject is compared with a facial image of the registered user to determine whether the second subject is the registered user.

5. The vehicle control device according to claim 4, wherein: The user authentication unit transmits a predetermined notification to a user terminal used by the registered user when it is determined in the second user authentication process that the second target person is not the registered user.

6. The vehicle control device according to any one of claims 2 and 5, wherein: The user authentication unit stops the operation of the object detection sensor when it is determined through the first user authentication process that the first target person existing around the vehicle is not the registered user after the vehicle stops.

7. The vehicle control device according to claim 6, wherein: When the user authentication unit determines that the first target person existing around the vehicle is the registered user through the first user authentication process after the vehicle stops, the user authentication unit continues the operation of the object detection sensor.

8. The vehicle control device according to any one of claims 2, 5 and 7, wherein: When the user authentication unit recognizes that the registered user has gotten off the vehicle through the image captured by the second camera, the user authentication unit stops the operation of the object detection sensor when the registered user cannot be recognized through the image captured by the second camera thereafter.

9. A vehicle control method, the vehicle control method being used in a vehicle having a plurality of object detection sensors, a lighting lamp, a first camera and a second camera, and being executed by a computer, wherein: The vehicle control method comprises the following steps: A peripheral object recognition step, identifying illumination based on the image captured by the first camera, determining whether it is night or day, and during the period when it is determined that it is night, setting each of the object detection sensors to an operating state at all times, and identifying objects existing around the vehicle based on detection signals of each of the object detection sensors; a human determination step, when an object existing in the vicinity of the vehicle is identified based on a detection signal of any of the object detection sensors in the vicinity object recognition step, turning on the first camera and the illumination light corresponding to the object detection sensor that has detected the object, photographing the object with the turned-on first camera corresponding to the object detection sensor that has detected the object, and performing a human determination process of determining whether the object is a human based on the image of the object; as well as a user authentication step, when the object is determined to be a person by the person determination step, photographing a first object person determined to be a person by the person determination step through the second camera corresponding to the object detection sensor that detected the object, and performing the following first user authentication processing: comparing a facial image of the first object person with a facial image of a registered user of the vehicle, thereby determining whether the first object person is the registered user, The resolution of each of the second cameras is higher than the resolution of the corresponding first camera, and the power consumption of each of the object detection sensors is lower than the power consumption of the corresponding first camera.

10. A vehicle control device, wherein: The vehicle control device comprises: a surrounding object recognition unit that recognizes an object existing around the vehicle based on a detection signal of an object detection sensor mounted on the vehicle and detecting an object existing around the vehicle; a human determination unit that, when the object is recognized by the peripheral object recognition unit, photographs the object using a first camera mounted on the vehicle and having a first resolution, and performs human determination processing for determining whether the object is a human based on the image of the object; as well as a user authentication unit which, when the human determination unit determines that the object is a human, photographs a first object person determined to be a human by the human determination unit through a second camera mounted on the vehicle and having a second resolution higher than the first resolution, and performs the following first user authentication processing: comparing a facial image of the first object person with facial images of registered users of the vehicle to determine whether the first object person is the registered user, When the user authentication unit determines through the first user authentication process that the first object person is not the registered user for more than a predetermined number of times, the user authentication unit prohibits execution of the first user authentication process until no object determined to be a person is extracted from the image captured by the first camera through the human determination process.

11. A vehicle control device, wherein: The vehicle control device comprises: a surrounding object recognition unit that recognizes an object existing around the vehicle based on a detection signal of an object detection sensor mounted on the vehicle and detecting an object existing around the vehicle; a human determination unit that, when the object is recognized by the peripheral object recognition unit, photographs the object using a first camera mounted on the vehicle and having a first resolution, and performs human determination processing for determining whether the object is a human based on the image of the object; as well as a user authentication unit which, when the human determination unit determines that the object is a human, photographs a first object person determined to be a human by the human determination unit through a second camera mounted on the vehicle and having a second resolution higher than the first resolution, and performs the following first user authentication processing: comparing a facial image of the first object person with facial images of registered users of the vehicle to determine whether the first object person is the registered user, When the user authentication unit determines through the first user authentication process that the first subject is not the registered user for more than a specified number of times, the object detection sensor is stopped, and when the operation of the touch sensor provided on the door of the vehicle is recognized, the second subject who operates the touch sensor is photographed by the second camera, and the second user authentication process is performed as follows: a facial image of the second subject is compared with a facial image of the registered user to determine whether the second subject is the registered user.

12. A vehicle control device, wherein: The vehicle control device comprises: a surrounding object recognition unit that recognizes an object existing around the vehicle based on a detection signal of an object detection sensor mounted on the vehicle and detecting an object existing around the vehicle; a human determination unit that, when the object is recognized by the peripheral object recognition unit, photographs the object using a first camera mounted on the vehicle and having a first resolution, and performs human determination processing for determining whether the object is a human based on the image of the object; as well as a user authentication unit which, when the human determination unit determines that the object is a human, photographs a first object person determined to be a human by the human determination unit through a second camera mounted on the vehicle and having a second resolution higher than the first resolution, and performs the following first user authentication processing: comparing a facial image of the first object person with facial images of registered users of the vehicle to determine whether the first object person is the registered user, The user authentication unit stops the operation of the object detection sensor when it is determined through the first user authentication process that the first target person existing around the vehicle is not the registered user after the vehicle stops.

13. A vehicle control device, wherein: The vehicle control device comprises: a surrounding object recognition unit that recognizes an object existing around the vehicle based on a detection signal of an object detection sensor mounted on the vehicle and detecting an object existing around the vehicle; a human determination unit that, when the object is recognized by the peripheral object recognition unit, photographs the object using a first camera mounted on the vehicle and having a first resolution, and performs human determination processing for determining whether the object is a human based on the image of the object; as well as a user authentication unit which, when the human determination unit determines that the object is a human, photographs a first object person determined to be a human by the human determination unit through a second camera mounted on the vehicle and having a second resolution higher than the first resolution, and performs the following first user authentication processing: comparing a facial image of the first object person with facial images of registered users of the vehicle to determine whether the first object person is the registered user, When the user authentication unit recognizes that the registered user has gotten off the vehicle through the image captured by the second camera, the user authentication unit stops the operation of the object detection sensor when the registered user cannot be recognized through the image captured by the second camera thereafter.