Facial recognition device

Abstract

To shorten a processing time taken to establishment of face authentication determination from detection of a person outside a vehicle.SOLUTION: A face authentication device 1 includes: an approach determination unit 1A which determines whether a distance D between a vehicle 2 and a person H outside the vehicle is within an approach distance Dth; an image acquisition unit 1B which sequentially acquires images IM taken by each of multiple cameras 3 when determination of the approach determination unit 1A is established; a face detection unit 1C which scans the acquired image IM and determines whether the image IM is a target image IO including a face image Ih of the person H; and a face authentication unit 1D which determines whether a registered face image Iu and the face image Ih match when determination of the face detection unit 1C is established. The image acquisition unit 1B sets the camera 3 which has a high possibility of capturing the target image IO among the multiple cameras 3 as a priority camera 3P, and preferentially acquires the image IM of the priority camera 3P over the image IM of another camera 3Q. The face detection unit 1C scans a partial prescribed region Rx out of all regions R of the image IM.SELECTED DRAWING: Figure 2

Description

Technical Field

[0002]

[0001] The present invention relates to a face recognition device that performs face recognition determination as to whether a person existing outside a vehicle is a person registered in the vehicle.

Background Art

[0002] Conventionally, there is known a face recognition device that detects a person existing outside a vehicle and performs face recognition determination as to whether a face image of the detected person matches a face image previously registered in the vehicle. For example, Patent Document 1 discloses a device that activates an in-vehicle camera and performs face recognition determination in response to the distance between a target object outside the vehicle acquired by a distance sensor and the vehicle becoming smaller than a predetermined distance threshold.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, some vehicles are equipped with a plurality of cameras in order to widely photograph an area outside the vehicle. In a face recognition device, a plurality of images taken by such a plurality of cameras are acquired, and determination processing (face detection processing) as to whether a face of a person outside the vehicle is captured in each image is performed, and face recognition determination is performed on an image for which the determination processing is successful. However, in the face detection processing, if the entire area of each image is scanned, the processing time may become long. Further, in a face recognition device, when a person outside the vehicle is a registered person, it is preferable to establish face recognition determination before the person reaches the vehicle. However, if the images taken by each camera are simply processed one by one in order, there is a possibility that the person may reach the vehicle during the processing of an image in which the face of the person outside the vehicle is not captured. In this case, the function as a face recognition device is not sufficiently exhibited.

[0005] This invention was conceived in light of the above-mentioned challenges, and one of its objectives is to shorten the processing time from detecting a person outside the vehicle to the completion of facial recognition. Furthermore, this is not the only objective; another objective of this invention is to achieve effects and benefits that cannot be obtained by conventional technologies, resulting from the various configurations shown in the embodiments for carrying out the invention described later. [Means for solving the problem]

[0006] The disclosed facial recognition device can be implemented in the following embodiments (examples of application) and solves at least some of the above-mentioned problems. Each of the embodiments from Embodiment 2 onward is an additional embodiment that can be appropriately selected and each of the embodiments is optional. None of the embodiments from Embodiment 2 onward disclose any embodiments or configurations that are essential to this case.

[0007] Embodiment 1. The disclosed facial recognition device includes: a proximity determination unit that determines whether the distance between a vehicle and a person outside the vehicle is within a predetermined proximity distance; an image acquisition unit that, when the determination of the proximity determination unit is successful, sequentially acquires images captured by each of a plurality of cameras facing different directions outside the vehicle; a face detection unit that scans the images acquired by the image acquisition unit and determines whether the images are target images that include the face image of the person; and a facial recognition unit that, when the determination of the face detection unit is successful, performs a facial recognition determination which is a determination of whether the face image matches a pre-stored registered face image. A base determination unit that determines whether or not the vehicle is located at a base location registered as a base for the vehicle, The image acquisition unit sets the camera most likely to capture the target image among the plurality of cameras as the priority camera, and acquires the image captured by the priority camera with priority over the image captured by the other cameras. The face detection unit scans a predetermined portion of the entire area of ​​the image. If the determination by the location determination unit is successful, the image acquisition unit sets the camera that has captured the most images at the location in the past for which the facial recognition determination was successful as the priority camera.

[0008] Embodiment 2. In Embodiment 1 described above, it is preferable that all of the multiple cameras are cameras equipped with fisheye lenses. In this case, it is preferable that the predetermined area is the area of ​​the entire area excluding both the area in which the vehicle body is captured and the area in which the ground near the vehicle is captured. Embodiment 3. In Embodiment 1 or 2 described above, it is preferable that the predetermined area is an area in which the inter-eye distance of the person is at least equal to or greater than a predetermined value.

[0009] Embodiment 4. In any of embodiments 1 to 3 described above, it is preferable that the image acquisition unit sets the camera whose imaging range includes the approach direction in which the person approaches the vehicle as the priority camera.

[0011] manner 5 The above aspects One of 1-4 In this case, it is preferable that the plurality of cameras include a first camera facing a first side from the driver's side of the vehicle and a second camera facing a second side different from the first side from the passenger side of the vehicle. In this case, it is preferable that the image acquisition unit sets the second camera as the priority camera when the determination of the base determination unit is not met and the vehicle is locked from the second side. [Effects of the Invention]

[0012] According to the disclosed facial recognition device, it is possible to shorten the processing time from detecting a person outside the vehicle to completing facial recognition. [Brief explanation of the drawing]

[0013] [Figure 1] This is a top view showing a vehicle to which a facial recognition device according to the first embodiment is applied, along with the surrounding environment outside the vehicle. [Figure 2] Figure 1 is a block diagram of the facial recognition device. [Figure 3] This figure shows an example of an image captured by a camera mounted on the vehicle shown in Figure 1. [Figure 4]Figure 1 is an example flowchart illustrating the process performed by the facial recognition device. [Figure 5] This is a block diagram of a facial recognition device according to the second embodiment. [Modes for carrying out the invention]

[0014] The facial recognition device as an embodiment will be described with reference to the drawings. The embodiments shown below are merely illustrative, and there is no intention to exclude various modifications or applications of technologies not explicitly shown in the embodiments below. Each component of this embodiment can be modified in various ways without departing from its spirit. Furthermore, components can be selected or combined as needed.

[0015] The facial recognition device of this embodiment is a device applied to a vehicle equipped with multiple cameras that capture images of the outside of the vehicle. When the distance between a person outside the vehicle and the vehicle falls within a predetermined proximity distance, the facial recognition device sequentially performs image acquisition processing, face detection processing, and facial recognition processing. In the image acquisition processing, image IMs captured by each of the multiple cameras are acquired sequentially. In the face detection processing, the acquired image IMs are scanned, and a determination is made (hereinafter referred to as "detection determination") to determine whether the scanned image IM is a target image IO that includes a face image Ih of a person outside the vehicle. Facial recognition processing is performed when the image IM is a target image IO. In the facial recognition processing, a facial recognition determination is made to determine whether the face image Ih included in the target image IO matches a pre-stored registered face image Iu.

[0016] In the face recognition device according to the embodiment, in the face detection process, scanning is performed only on a preset partial predetermined area of the entire area of the image IM. This reduces the time required for the face detection process. Further, in the face recognition device according to the embodiment, in the image acquisition process, one camera with a high possibility of acquiring the target image IO among a plurality of cameras is set as the priority camera, and the image IM captured by the priority camera is acquired preferentially over the images IM captured by other cameras. By preferentially acquiring the image IM captured by the priority camera in this way, the detection determination is established earlier, and the processing time from detecting a person outside the vehicle to establishing the face recognition determination is shortened.

[0017] In the following description, the forward direction of the vehicle is taken as the front (front of the vehicle), and the opposite direction is taken as the rear (rear of the vehicle). Also, the left and right are defined based on the state of facing the front of the vehicle. Note that the left-right direction is orthogonal to the front-rear direction of the vehicle. Hereinafter, the left-right direction is also referred to as the "vehicle width direction", and the front-rear direction of the vehicle is simply referred to as the "front-rear direction". The direction orthogonal to both the vehicle width direction and the front-rear direction is defined as the up-down direction.

[0018] In the embodiment, it is assumed that the vehicle is a right-hand drive vehicle with a driver's seat provided on the right side in the vehicle width direction. That is, in the embodiment, "the first side from the driver's seat side" in the claims is "the right side", and "the second side different from the first side from the passenger seat side" is "the left side".

[0019] [1. First Embodiment] [1-1. Overall Configuration] Referring to FIGS. 1 to 4, the face recognition device 1 according to the first embodiment will be described. As shown in FIG. 1, the face recognition device 1 is applied to a vehicle 2 equipped with a plurality of cameras 3. Note that the face recognition device 1 may be mounted on the vehicle 2 as shown in FIG. 1, or may be provided outside the vehicle 2 and perform the above processing via communication with the vehicle 2.

[0020] In the first embodiment, the face recognition device 1 acquires the approach direction AD in which a person H existing outside the vehicle 2 approaches the vehicle 2, and sets the camera 3 that includes the acquired approach direction AD in the imaging range as the priority camera 3P. Hereinafter, the "person H existing outside the vehicle 2" will be simply referred to as "person H". The vehicle 2 is provided with, for example, a communication device 4 as a device that acquires (calculates) the approach direction AD and the distance D between the person H and the vehicle 2.

[0021] The plurality of cameras 3 are all imaging devices that photograph the outside of the vehicle, and each faces a different direction outside the vehicle. Here, four cameras, a front camera 31, a rear camera 32, a right camera 33 (first camera), and a left camera 34 (second camera), provided for generating an around view image of the vehicle 2 are used as the plurality of cameras 3.

[0022] Each of the cameras 31 to 34 is a fish-eye camera. A fish-eye camera is a camera equipped with a fish-eye lens having a wider angle of view ΔA (for example, an angle of view of about 190° in the horizontal direction and about 150° in the vertical direction) than a standard lens, and can photograph a wider range than a standard camera. In the vehicle 2, by applying fish-eye cameras to the plurality of cameras 3 in this way, it becomes possible to image the entire periphery outside the vehicle.

[0023] Note that the around view image is a composite image corresponding to the view when the vehicle 2 is俯瞰 from above, and is generated by an in-vehicle device (not shown) based on the images captured by each of the cameras 31 to 34. In this way, by utilizing the cameras 3 mounted on the existing vehicle 2, an inexpensive face recognition device 1 can be provided.

[0024] The front camera 31 is positioned, for example, on the front end surface of the vehicle 2 and faces forward. The imaging range of the front camera 31 may include the front of the vehicle 2, as well as the front right and front left of the vehicle 2. The rear camera 32 is positioned, for example, on the rear end surface of the vehicle 2 and faces backward. The imaging range of the rear camera 32 may include the rear of the vehicle 2, as well as the rear right and rear left of the vehicle 2. The right-side camera 33 is positioned, for example, on the right side mirror and faces to the right of the vehicle 2. The imaging range of the right-side camera 33 may include the right side of the vehicle 2, as well as the right side of the vehicle 2. The left-side camera 34 is positioned, for example, on the left side mirror and faces to the left of the vehicle 2. The imaging range of the left-side camera 34 may include the left side of the vehicle 2, as well as the left side of the vehicle 2.

[0025] Communication device 4 is configured to communicate with person H's mobile terminal Hd (e.g., a smartphone or smart key), and acquires the approach direction AD and distance D through communication with the mobile terminal Hd. For example, Bluetooth® communication is used for communication between communication device 4 and the mobile terminal Hd, and information is acquired when communication is established with a pre-paired mobile terminal Hd [for example, a mobile terminal possessed by the person with the registered facial image Iu (hereinafter referred to as "registered person U")].

[0026] The distance D is calculated, for example, using the AOA (Angle of Arrival) method. The TOA (Time of Arrival) method, which is based on the difference in radio wave reflection times, may also be used to calculate the distance D. The communication device 4 may use the AOA method to calculate the angle of person H with respect to an arbitrary position on the vehicle 2 (for example, the position of the center P in the longitudinal direction of the vehicle 2, or a virtual plane passing through this center P and perpendicular to the longitudinal direction). The approach direction AD may be calculated using the calculated distance D and angle. The angle or approach direction AD calculated here may be a numerical value such as "20 degrees counterclockwise from the reference position," or it may be a rough direction such as "to the right of center P" or "in front of center P." Distance detection using the AOA or TOA method, and angle detection using the AOA method are well known, so a detailed explanation will be omitted.

[0027] The facial recognition device 1 is an electronic computer that performs the above-mentioned processing (calculations), and incorporates a microprocessor (central processing unit), memory (main memory), storage device, interface device, etc. Multiple cameras 3 and a communication device 4 are connected to the input side of the facial recognition device 1. An input device (not shown) for registering registered facial images Iu may also be connected to the input side of the facial recognition device 1. Onboard equipment (for example, an actuator that switches between the unlocked and locked states of the doors of the vehicle 2) and a control device for controlling the onboard equipment may be connected to the output side of the facial recognition device 1.

[0028] [1-2. Facial Recognition Devices] As shown in Figure 2, the facial recognition device 1 is equipped with a proximity detection unit 1A, an image acquisition unit 1B, a face detection unit 1C, and a facial recognition unit 1D as functional elements for performing the above-described process. These functional elements may be implemented by electronic circuits (hardware), programmed as software, or some of these functions may be provided as hardware and the others as software.

[0029] The proximity determination unit 1A performs proximity determination to determine whether the distance D is within a predetermined proximity distance Dth. The proximity determination unit 1A receives the distance D from the communication device 4, and if it determines that the received distance D is within the proximity distance Dth (D ≤ Dth), it transmits a signal indicating that proximity determination has been made to the image acquisition unit 1B. The proximity distance Dth is the distance at which it can detect that person H has approached vehicle 2 before arriving at vehicle 2, and is set to, for example, 5 [m]. If the proximity determination unit 1A determines that the received distance D is not within the proximity distance Dth (D > Dth), it determines that person H has not approached vehicle 2 and does not transmit a signal to the image acquisition unit 1B.

[0030] The image acquisition unit 1B performs the image acquisition process described above. When the image acquisition unit 1B receives a signal from the proximity detection unit 1A indicating that proximity detection has been successful, it sequentially acquires the image IM captured by each of the multiple cameras 3. In other words, the image acquisition unit 1B does not acquire the image IM from the multiple cameras 3 simultaneously, but acquires the image IM from each of the multiple cameras 3 in order. The image IM acquired by the image acquisition unit 1B is transmitted to the face detection unit 1C in the order in which it was acquired.

[0031] The image acquisition unit 1B prioritizes acquiring image IM captured by the priority camera 3P over image IM captured by cameras 3 other than the priority camera 3P. Hereinafter, "camera 3 other than the priority camera 3P" will also be referred to as "other camera 3Q". As described above, in the first embodiment, the priority camera 3P is the camera 3 whose imaging range includes the approach direction AD. Therefore, the image acquisition unit 1B acquires the approach direction AD from the communication device 4 and sets the camera 3 whose imaging range includes the acquired approach direction AD as the priority camera 3P.

[0032] For example, as shown in Figure 1, when person H approaches vehicle 2 from the right (when the approach direction AD is to the right), the image acquisition unit 1B sets the right-side camera 33, which includes the right side in its imaging range, as the priority camera 3P and preferentially acquires the image IM from the right-side camera 33. Here, "preferentially" may mean acquiring the image IM from the priority camera 3P "before" the image IM from the other cameras 3Q. Alternatively, it may mean acquiring "more" image IM from the priority camera 3P than from the other cameras 3Q.

[0033] In the former case, the image acquisition unit 1B first acquires the image IM from the priority camera 3P, and then sequentially acquires the image IMs from the other cameras 3Q. For example, if the right camera 33 is set as the priority camera 3P, the image acquisition unit 1B acquires the image IMs in the order of right camera 33, front camera 31, left camera 34, rear camera 32, right camera 33, front camera 31, and so on. In this case, the image acquisition unit 1B only needs to acquire the image IM from the priority camera 3P before the image IMs from the other cameras 3Q, and the order in which the image IMs from the other cameras 3Q are acquired is not limited to the order described above.

[0034] In the latter case, the image acquisition unit 1B repeatedly acquires image IMs, for example, by acquiring the image IM from the priority camera 3P, and then acquiring the image IM from one of the other cameras 3Q. For example, if the right camera 33 is set as the priority camera 3P, the image acquisition unit 1B acquires image IMs in the order of right camera 33, front camera 31, right camera 33, left camera 34, right camera 33, rear camera 32, right camera 33, front camera 31... In this case, the image acquisition unit 1B only needs to acquire at least more image IMs from the priority camera 3P than from the other cameras 3Q, and the frequency of acquiring image IMs from the priority camera 3P and the order in which image IMs from the other cameras 3Q are acquired are not limited to the above example. In either of the two acquisition methods described above, the image IMs from the priority camera 3P are given priority (importance) over the image IMs from the other cameras 3Q.

[0035] The face detection unit 1C performs the face detection process described above. The face detection unit 1C scans each image IM transmitted sequentially from the image acquisition unit 1B and performs a detection determination to determine whether the scanned image IM is a target image IO that includes a face image Ih.

[0036] If detection is successful, that is, if the scanned image IM contains a face image Ih, the face detection unit 1C transmits a signal indicating successful detection to the face recognition unit 1D. At the same time, the face detection unit 1C transmits the image IM that resulted in the successful detection (i.e., the target image IO containing the face image Ih) to the face recognition unit 1D. Alternatively, the face detection unit 1C may transmit only the face image Ih contained in the target image IO to the face recognition unit 1D. If detection is not successful, that is, if the scanned image IM does not contain a face image Ih, the face detection unit 1C performs scanning and detection on the image IM transmitted from the image acquisition unit 1B after the scanned image IM.

[0037] The face detection unit 1C scans a predetermined portion of the entire image area R, specifically a portion Rx, during the scanning of the image IM. In other words, the face detection unit 1C performs detection and determination without scanning the area of ​​the entire image R that is not the predetermined area Rx.

[0038] The predetermined region Rx will be described in detail with reference to Figure 3. Figure 3 is an example of an image IM captured by the right camera 33, and is a diagram showing an example of an image IM that makes detection determination possible. In Figure 3, the blacked-out areas indicate areas not captured by the right camera 33.

[0039] If the right camera 33 is a fisheye camera, as shown in Figure 3, the image IM includes a vehicle body region Ra (the region shown as dotted in Figure 3) in which the vehicle body (right side) of vehicle 2 is captured. The image IM also includes a ground region Rb (the region shown as hatched in Figure 3) in which the ground near vehicle 2 is captured. In the face detection unit 1C, these regions Rb, Rb are excluded from the scanning area as they do not contain the face of person H. In other words, the predetermined region Rx is defined as the region of the entire region R excluding both the vehicle body region Ra and the ground region Rb.

[0040] Furthermore, in this embodiment, the predetermined region Rx is defined as the region where the inter-eye distance X of person H is at least equal to a predetermined value Xth. This region may be predetermined by experiments or simulations using camera 3.

[0041] The inter-eye distance X is the distance between the right and left eyes of a person H in the image IM, and the size of the face image Ih is determined according to the inter-eye distance X. The size of the face image Ih affects the computational load of the subsequent face recognition processing. More specifically, the smaller the size of the face image Ih, the lower the resolution of the face image Ih, which increases the computational load of the face recognition processing. For this reason, it is preferable that the predetermined value Xth be a value that allows for the acquisition of a face image Ih of a size that does not make the computational load of the face recognition processing too heavy. In this embodiment, in consideration of these regions (i.e., the vehicle body region Ra, the ground region Rb, and the region where the inter-eye distance X is greater than or equal to the predetermined value Xth), the predetermined region Rx is set as the upper and central rectangular region of the entire region R of the image IM, as shown in Figure 3.

[0042] Known methods can be used for the scanning method of a predetermined region Rx, the detection determination method, and the acquisition method of the face image Ih by the face detection unit 1C. For example, the face detection unit 1C scans the predetermined region Rx by moving the position of the search window (a search frame of several pixels × several pixels) one pixel at a time. Then, it processes the frame image of each search window through a classifier that has been created in advance by machine learning to determine whether or not the frame image is a face image Ih. If this determination is successful, the face detection unit 1C acquires the frame image as the face image Ih and determines that the image IM is the target image IO that contains the face image Ih (i.e., it determines that the detection determination has been successful).

[0043] The facial recognition unit 1D performs the facial recognition process described above. When the facial recognition unit 1D receives a signal from the facial detection unit 1C indicating that detection has been successful, it performs a facial recognition determination, which is to determine whether the facial image Ih transmitted from the facial detection unit 1C matches the registered facial image Iu. For example, the facial recognition unit 1D extracts facial features from the facial image Ih, compares the extracted facial features with the facial features of the registered facial image Iu, and determines whether they belong to the same person, thereby performing the facial recognition determination.

[0044] [1-3. Flowchart] Figure 4 is an example flowchart illustrating the above-described process performed by the facial recognition device 1. The flowchart in Figure 4 starts, for example, when the proximity detection performed by the proximity detection unit 1A is successful. The flowchart in Figure 4 also ends when the proximity detection performed by the proximity detection unit 1A is unsuccessful, when the mobile terminal Hd is inside the vehicle 2, or when the facial recognition detection performed by the facial recognition unit 1D is successful.

[0045] In step S1, the approach direction AD is acquired. In the following step S2, camera 3 whose imaging range includes the acquired approach direction AD is set as the priority camera 3P. In the following step S3, the image IM is acquired taking into account the priority camera 3P set in step S2, and the process proceeds to step S4. For example, if the process in step S3 is performed for the first time in the state shown in Figure 1, the image IM of the right camera 33, which is the priority camera 3P, is acquired with priority over the other cameras 3Q.

[0046] In step S4, it is determined whether detection was successful based on the image IM acquired in step S3. If detection is determined to be successful in step S4, the process proceeds to step S5. In step S5, it is determined whether face recognition was successful. If face recognition is determined to be successful in step S5, that is, if the face image Ih of person H in the image IM that resulted in the successful detection in step S4 matches the registered face image Iu of registered person U, this flow is terminated. In this way, by prioritizing the processing of image IMs captured by the priority camera 3P, the processing time from the start to the end of this flow can be shortened.

[0047] On the other hand, if it is determined in step S4 that detection is not successful, that is, if the image IM of priority camera 3P does not include the face image Ih of person H, the process returns to step S3. Also, if it is determined in step S5 that face recognition is not successful, the process returns to step S3. The above-mentioned "cases where the image IM of priority camera 3P does not include the face image Ih of person H" include, for example, when person H is not facing towards vehicle 2, or when person H has moved out of the imaging range of priority camera 3P.

[0048] If the process returns to step S3 from step S4 or step S5, step S3 will acquire an image IM taken by a different camera 3 than the one that took the image IM acquired in the previous step S3, and the process will proceed to step S4. For example, if the image IM of the right camera 33 is acquired in the first step S3 as described above, then in the next step S3, an image IM of another camera 3Q (for example, the front camera 31) will be acquired.

[0049] Subsequently, if both the determinations in step S4 and step S5 are found to be successful, this flow is terminated. By processing the image IMs of other cameras 3Q in this way, face recognition can be achieved even if person H makes an unexpected movement (for example, if person H moves out of the imaging range of priority camera 3P). Furthermore, in step S3 from the third time onward, the image IMs captured by priority camera 3P are acquired preferentially, which prevents registered person U from reaching vehicle 2 while the image IMs of other cameras 3Q are being processed.

[0050] [1-4. Action, Effects] (1) In the face recognition device 1 described above, the image acquisition unit 1B sequentially acquires image IMs captured by each of the multiple cameras 3, and the face detection unit 1C processes the image IMs one by one in the order they were acquired by the image acquisition unit 1B. By processing the image IMs captured by the multiple cameras 3 sequentially rather than simultaneously, it is possible to suppress the heavy computational load on the face recognition device 1 and the resulting decrease in processing speed. Furthermore, the face detection unit 1C scans only a predetermined region Rx of the entire region R of the image IM. This further reduces the time required for face detection processing. Therefore, it is possible to shorten the processing time from the detection of a person H to the establishment of face recognition determination.

[0051] Furthermore, the facial recognition device 1 sets one of the multiple cameras 3 that is most likely to acquire the target image IO as the priority camera 3P, and prioritizes acquiring the image IM captured by the priority camera 3P over the image IM captured by the other cameras 3Q. This allows for earlier completion of facial recognition and prevents the registered person U from reaching the vehicle 2 while the image IM of the other cameras 3Q, which are less likely to acquire the target image IO, is being processed.

[0052] (2) In the facial recognition device 1 described above, the predetermined region Rx is defined as the region of the entire region R of the image IM that excludes both the vehicle region Ra and the ground region Rb. By excluding regions where the face of a person H is unlikely to be captured from the scanning region in this way, it is possible to reduce the time required for facial detection processing while suppressing a decrease in the accuracy of the facial detection processing.

[0053] (3) Furthermore, in the facial recognition device 1 described above, the predetermined region Rx is defined as the region where the inter-eye distance X is greater than or equal to a predetermined value Xth. As a result, regions where only facial images Ih with an inter-eye distance X less than the predetermined value Xth are detected are not scanned, thus suppressing an increased computational load on the subsequent facial recognition process. Consequently, the processing time from detecting a person H to the completion of facial recognition can be shortened.

[0054] (4) In the facial recognition device 1 described above, camera 3, which includes the approach direction AD in its imaging range, is set as the priority camera 3P. Therefore, the image IM from camera 3, which has a higher probability of capturing a person H, can be processed preferentially. Thus, the processing time from detecting a person H to the completion of facial recognition can be shortened.

[0055] [2. Second Embodiment] In the facial recognition device 1 described above, a camera 3 that includes the approach direction AD in its imaging range is set as the priority camera 3P. However, the method for setting the priority camera 3P is not limited to the example described above. For example, if the approach direction AD cannot be acquired, the facial recognition device 1 may set the priority camera 3P by another method. The facial recognition device 1' of the second embodiment will be described below with reference to Figure 5. In the following description, the configurations that differ from the first embodiment described above will be mainly described, and for configurations that correspond to the configuration of the first embodiment, a dash (') will be added to the reference numerals of the first embodiment, and redundant explanations will be omitted.

[0056] The facial recognition device 1' of the second embodiment sets a priority camera 3P depending on whether or not vehicle 2' is present at location P1, which is registered as a base for vehicle 2'. Location P1 is not particularly limited as long as it is a location registered for vehicle 2', but it is a location where vehicle 2' is parked frequently (for example, home or workplace).

[0057] In the second embodiment, when a vehicle 2' is present at base location P1, the facial recognition device 1' sets the camera 3 that has captured the most images IM that have successfully performed facial recognition at base location P1 in the past as the priority camera 3P. In other words, when a vehicle 2' is present at base location P1, the facial recognition device 1' in the second embodiment sets the camera 3 that has contributed the most to successful facial recognition in the past as the priority camera 3P.

[0058] Furthermore, if the vehicle 2' is located at a location other than the base location P1, and the vehicle 2' is locked from the left side, which is the passenger side, the facial recognition device 1' sets the left-side camera 34 as the priority camera 3P. An example of a case where the vehicle 2' is locked from the left side at a location other than the base location P1 is when a registered person U assists a passenger (e.g., a child) to get out of the vehicle at a shopping center, and then locks the vehicle 2'. In such a case, the registered person U may assist the passenger to get into the vehicle before getting into the vehicle themselves. In such cases, the facial recognition device 1' of the second embodiment aims to shorten the processing time from detecting a person H to the completion of facial recognition by setting the left-side camera 34 as the priority camera 3P.

[0059] [2-1. Structure] Vehicle 2' is equipped with multiple cameras 3, a communication device 4', a positioning device 5, and a passenger-side door handle button 6 (hereinafter referred to as "passenger-side button 6"). In this embodiment, the communication device 4' does not acquire the approach direction AD, but only acquires (calculates) the distance D.

[0060] The positioning device 5 measures the position information (latitude, longitude, and altitude) of the vehicle 2' current position P based on GNSS (Global Navigation Satellite System) and detection information from a vehicle speed sensor (not shown).

[0061] The passenger-side button 6 is a physical button located on the passenger-side door handle (not shown) and operated manually. When the passenger-side button 6 is operated, the operation signal from the passenger-side button 6 is transmitted to an actuator (not shown) that switches between the unlocked and locked states of the vehicle's door 2'. This switches the door between the unlocked and locked states. The operation signal from the passenger-side button 6 is also transmitted to the facial recognition device 1'.

[0062] The facial recognition device 1' is an electronic computer that performs the processing (calculations) described above. Multiple cameras 3, a communication device 4', a positioning device 5, and a passenger-side button 6 are connected to the input side of the facial recognition device 1'. An input device (not shown) for registering a registered facial image Iu and a base location P1 may also be connected to the input side of the facial recognition device 1'. Onboard equipment (for example, the actuators described above) and a control device for controlling the onboard equipment may be connected to the output side of the facial recognition device 1'.

[0063] The facial recognition device 1' is equipped with a proximity determination unit 1A, an image acquisition unit 1B', a face detection unit 1C, and a facial recognition unit 1D as functional elements for performing the above-described process. Furthermore, the facial recognition device 1' is equipped with a location determination unit 1E and a storage unit 1F.

[0064] The base determination unit 1E performs base determination, which is the determination of whether or not a vehicle 2' is present at base location P1. The base determination unit 1E obtains location information of the current location P of vehicle 2' from the positioning device 5 and performs base determination by determining whether or not the obtained location information is approximately the same as the location information of base location P1. The base determination unit 1E transmits the result of the base determination to the image acquisition unit 1B'. If the above determination is successful, the base determination unit 1E may transmit a signal to the storage unit 1F indicating that a vehicle 2' is present at base location P1.

[0065] The memory unit 1F stores the number of times a camera 3, which captured an image IM (hereinafter also called "positive image IC") that successfully determined facial recognition at base location P1, captured the positive image IC. When the memory unit 1F receives a signal from the base location determination unit 1E indicating the presence of a vehicle 2' at base location P1, and also receives a signal from the facial recognition unit 1D indicating successful facial recognition, it identifies the camera 3 that captured the positive image IC. The memory unit 1F then counts the number of times each of the multiple cameras 3 captured the positive image IC and stores the camera with the highest count as base camera 3B.

[0066] The image acquisition unit 1B' acquires the image of the first embodiment. acquisitionSimilar to unit 1B, image acquisition processing is performed. As described above, in the second embodiment, the priority camera 3P is set depending on whether or not a vehicle 2' is present at base location P1. For this reason, the image acquisition unit 1B' receives the result of base determination from base determination unit 1E and sets the priority camera 3P according to the received result.

[0067] Specifically, when the image acquisition unit 1B' receives a result from the base determination unit 1E indicating that a base determination has been made, it refers to the storage unit 1F and sets the base camera 3B as the priority camera 3P. If the image acquisition unit 1B' receives a result from the base determination unit 1E indicating that a base determination has not been made, it determines whether or not it received an operation signal from the passenger-side button 6 while receiving this result. If the image acquisition unit 1B' determines that it received an operation signal from the passenger-side button 6, it sets the left camera 34 as the priority camera 3P.

[0068] Furthermore, if the image acquisition unit 1B' determines that it has not received an operation signal from the passenger-side button 6, it may acquire images IM in a predetermined order (for example, clockwise from the front camera 31 in a top view) without setting the priority camera 3P. Alternatively, in this case, the image acquisition unit 1B' may determine that there is a high probability that the registered person U is approaching the vehicle 2' from the right side, which is the side of the driver's seat, and set the right-side camera 33 as the priority camera 3P.

[0069] [2-2. Action, Effects] (1) In the facial recognition device 1' described above, if the vehicle 2' is located at base location P1, the base camera 3B is set as the priority camera 3P. In this way, the facial recognition device 1' can set the priority camera 3P even if the approach direction A and D are unknown. Therefore, the processing time from detecting a person H to the completion of facial recognition can be shortened.

[0070] (2) Furthermore, in the facial recognition device 1' described above, if the vehicle 2' is located at a location other than the base location P1 and the vehicle 2' is locked from the passenger side, the left camera 34 is set as the priority camera 3P. By setting the priority camera 3P in this way, taking into account the actions of the registered person U after getting out of the vehicle, it is possible to shorten the processing time from detecting the person H to the completion of facial recognition determination, even if the approach direction AD is unknown. Furthermore, the facial recognition device 1' of the second embodiment can also obtain the same effects and benefits as those (1) to (3) obtained from the facial recognition device 1 of the first embodiment.

[0071] [3. Others] The configuration of the facial recognition devices 1,1' and the processing performed by the facial recognition devices 1,1' described above are examples. For example, the predetermined region Rx may be at least a part of the entire region R of the image IM, and is not limited to the region described above.

[0072] The facial recognition device 1,1' does not need to acquire image IMs from all cameras 3 mounted on the vehicle 2,2'. For example, if it determines from the image IM captured by the rear camera 32 that it is difficult to open or close the back door, the facial recognition device 1,1' does not need to acquire the image IM captured by the rear camera 32.

[0073] Furthermore, the configuration of the vehicle 2,2' to which the facial recognition device 1,1' is applied is also an example. For example, vehicle 2,2' may be equipped with a radar device for detecting objects outside the vehicle, instead of or in addition to the communication device 4,4', as a device for acquiring distance D. In vehicle 2', the device for detecting that vehicle 2' has been locked from the passenger side (second side) does not have to be the passenger side button 6. For example, vehicle 2' may be equipped with a receiver that receives weak radio waves emitted from the smart key of vehicle 2', instead of or in addition to the passenger side button 6. In this case, the receiver may determine whether or not vehicle 2' has been locked from the passenger side (second side) by identifying the direction of transmission of the received radio waves.

[0074] The multiple cameras 3 only need to be at least two and facing different directions, and are not limited to cameras 31-34 for generating the around-view image. Also, cameras 3 do not need to be fisheye cameras. The vehicle may be a left-hand drive vehicle with the driver's seat located on the left side in the vehicle width direction. In this case, "first side from the driver's seat side" in the claims means "left side," and "second side from the passenger seat side, separate from the first side" means "right side." [Industrial applicability]

[0075] This technology is applicable to the manufacturing industry of facial recognition devices applied to vehicles, and also to the manufacturing industry of vehicles equipped with such facial recognition devices. [Explanation of Symbols]

[0076] 1,1′ Facial recognition device 1A Approach judgment part 1B,1B′ Image acquisition unit 1C Face detection unit 1D facial recognition unit 1E Base Determination Unit 2,2' Vehicle 3 cameras 3P Priority Camera 3Q Other cameras 31 Front Camera 32 Rear camera 33 Right-hand camera 34 Left-hand camera AD approach direction D distance Dth approach distance H Person (Person outside the car) IM image IO target image Ih Face Image IU's registered face image R All areas Ra: Vehicle body area (the area where the vehicle body is visible) Rb Ground area (the area showing the ground near the vehicle) Rx predetermined area X Eye distance Xth predetermined value

Claims

1. A proximity determination unit that determines whether the distance between the vehicle and a person outside the vehicle is within a predetermined proximity distance, When the proximity detection unit makes a determination, the image acquisition unit sequentially acquires images captured by each of the multiple cameras facing different directions externally, A face detection unit scans the image acquired by the image acquisition unit and determines whether the image is a target image that includes the face image of the person, If the determination by the face detection unit is successful, the face recognition unit performs a face recognition determination, which is a determination of whether or not the face image matches a pre-stored registered face image. The system includes a base determination unit that determines whether or not the vehicle is located at a base location registered as a base for the vehicle, The image acquisition unit sets the camera most likely to capture the target image among the multiple cameras as the priority camera, and acquires the image captured by the priority camera with priority over the image captured by the other cameras. The face detection unit scans a predetermined portion of the entire image, If the determination by the location determination unit is successful, the image acquisition unit sets the camera that has captured the most images at the location in the past for which the facial recognition determination was successful as the priority camera. A facial recognition device characterized by the following features.

2. All of the aforementioned cameras are equipped with fisheye lenses. The predetermined region is the region of the entire region excluding both the region in which the vehicle's body is visible and the region in which the ground near the vehicle is visible. The facial recognition device according to claim 1, characterized in that...

3. The predetermined region is a region in which the inter-eye distance of the person is at least equal to or greater than a predetermined value. A facial recognition device according to claim 1 or 2, characterized in that...

4. The image acquisition unit sets the camera whose imaging range includes the approach direction in which the person approaches the vehicle as the priority camera. The facial recognition device according to claim 1, characterized in that...

5. The plurality of cameras include a first camera facing a first side from the driver's side of the vehicle, and a second camera facing a second side different from the first side from the passenger side of the vehicle. The image acquisition unit sets the second camera as the priority camera if the determination by the base determination unit is not successful and the vehicle is locked from the second side. The facial recognition device according to claim 1, characterized in that...

Images