Authentication system

Abstract

To provide an authentication system that can perform face authentication, with a simple structure, without being provided with a dedicated on-vehicle camera.SOLUTION: An authentication system 2, in which cameras (side cameras 20c and 20d) are provided on lower parts of side mirrors of a vehicle, can photograph a road surface beside the vicinity of the vehicle when the side mirrors are in use. A mirror driving part 26 stores the side mirrors in positions where the cameras can photograph the face of a person present beside the vehicle. When the side mirrors are stored, an authenticating part 42 performs face authentication on the basis of an image of the face of the person photographed with the cameras.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an authentication system for a vehicle that performs face authentication. [Background technology]

[0002] A configuration is known in which a dedicated camera for facial recognition of people approaching the vehicle is installed on the B-pillar of the vehicle. In order to suppress the increase in cost due to the dedicated camera for facial recognition and the increase in cost due to changes in the body structure, a technology is also known in which a side camera for object recognition is also used for facial recognition.

[0003] Patent Document 1 discloses an unlocking control device that includes an exterior camera that captures images of the outside of the vehicle cabin and an interior camera that captures images of the inside of the vehicle cabin, and controls unlocking of the vehicle's door. When the vehicle is in a running state, this device acquires a first image of the interior of the vehicle cabin captured by the interior camera, the first image including the faces of the vehicle occupants. After switching from a running state to a stopped state, the imaging range of the exterior camera is changed from a first imaging range that captures images used for driving assistance of the vehicle to a second imaging range for capturing images of the faces of people outside the vehicle cabin, and the second imaging range acquires a second image captured by the exterior camera. Then, face authentication is performed by comparing the faces of the occupants included in the first image with the faces of people included in the second image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-82954 A Summary of the Invention [Problem to be solved by the invention]

[0005] Cameras provided on the sides of a vehicle include, for example, side cameras for object recognition and side cameras for periphery monitoring that capture images to enable the driver to check the surroundings of the vehicle.

[0006] Since the side camera for object recognition is focused on a long distance to detect cars and pedestrians at a relatively long distance, faces at a relatively close distance are blurred, making it difficult to use for face recognition. In the technology of Patent Document 1, the exterior camera has a complex configuration because it changes the imaging range and focal length during face recognition.

[0007] Side cameras for perimeter monitoring are focused on a relatively close distance to capture the road surface near the vehicle, but because these cameras are mounted below the side mirror and point toward the road surface, it is difficult to capture images of people's faces.

[0008] An object of the present invention is to provide a technology capable of performing face recognition with a simple configuration, without providing a dedicated vehicle-mounted camera. [Means for solving the problem]

[0009] In order to solve the above problems, an authentication system according to one embodiment of the present invention includes a camera mounted under the side mirror of a vehicle and capable of capturing an image of the road surface to the side of the vehicle near the side mirror when the side mirror is in use, a mirror drive unit that stores the side mirror in a position where the camera can capture an image of the face of a person to the side of the vehicle, and an authentication unit that performs facial authentication based on the image of the person's face captured by the camera when the side mirror is in the stored state. Effect of the Invention

[0010] According to the present invention, it is possible to provide a technology capable of performing face recognition with a simple configuration without providing a dedicated vehicle-mounted camera. [Brief description of the drawings]

[0011] [Figure 1] 1(a) and (b) are side views of a vehicle according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating a functional configuration of an in-vehicle system according to an embodiment. [Diagram 3] 3 is a flowchart showing the process of the authentication system of FIG. 2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] 1(a) and (b) are side views of a vehicle 100 according to an embodiment. Fig. 1(a) shows the vehicle 100 when a side mirror 110 is in a use state, and Fig. 1(b) shows the vehicle 100 when the side mirror 110 is in a stored state. The vehicle 100 may be a vehicle driven by a driver or may be an autonomous vehicle.

[0013] Vehicle 100 is equipped with a side mirror 110 and a side camera 20c. 1(a) and (b) show the left side mirror 110 and the left side camera 20c to show the left side of vehicle 100, but vehicle 100 also includes a right side mirror and a right side camera (not shown). The following mainly describes the configurations and functions of left side mirror 110 and left side camera 20c, but the configurations and functions of the right side mirror and right side camera are the same.

[0014] In the use state, the side mirror 110 protrudes outward in the vehicle width direction from the front door, and the driver can check the rear of the vehicle by viewing the mirror from inside the vehicle. In the stored state, the side mirror 110 is folded. The stored state can also be called a tilted state. In the stored state, the amount of protrusion of the side mirror 110 outward in the vehicle width direction from the front door is smaller than in the use state.

[0015] As shown in FIG. 1(a), the side camera 20c is provided at the bottom of the side mirror 110. The part of the side mirror 110 that is located vertically below the side mirror 110 when the side mirror 110 is in use is defined as the bottom of the side mirror 110. The side camera 20c is fixed to the side mirror 110 and does not need to be tilted relative to the side mirror 110. The side camera 20c does not need to have a focus adjustment mechanism. Therefore, a camera with a simple configuration can be adopted as the side camera 20c. Note that, for the sake of clarity, the side camera 20c protrudes from the bottom of the side mirror 110 in FIGS. 1(a) and 1(b), but the side camera 20c does not need to protrude from the bottom of the side mirror 110.

[0016] The side camera 20c is a camera for monitoring the surroundings, and can capture the road surface on the side near the vehicle 100 within the captureable range R1 when the side mirror 110 is in use. The side camera 20c can capture the road surface including the wheels of the vehicle 100. The optical axis of the side camera 20c at this time is directed substantially vertically downward. The focus of the side camera 20c is set to a relatively short distance so that the road surface can be captured. As will be described later, the image of the left side captured by the side camera 20c when the side mirror 110 is in use is combined with the image of the front, the image of the rear, and the image of the right side of the vehicle 100 captured by other cameras, and is used to generate a virtual bird's-eye image that enables the driver to check the surroundings of the vehicle 100 from a viewpoint above the vehicle 100.

[0017] When the side mirror 110 in use rotates around the rotation axis A1, the side mirror 110 is folded and stored as shown in FIG. 1(b). The rotation axis A1 passes through a support (not shown) that fixes the side mirror 110 to the vehicle body. For example, the rotation axis A1 is inclined toward the front of the vehicle 100 with respect to the vertical direction and is also inclined toward the left of the vehicle 100. Therefore, the side mirror 110 is folded with the support as a fulcrum so that the tip of the side mirror 110 faces diagonally upward and rearward of the vehicle 100, and the lower part of the side mirror 110 rises up from the ground. Therefore, the direction of the optical axis of the side camera 20c changes, the angle between the optical axis of the side camera 20c and the vertical downward direction becomes large, and the angle between the optical axis of the side camera 20c and the ground becomes small. Therefore, the face of an adult person 150 standing near the vehicle door on the side of the vehicle 100 comes within the photographable range R1 of the side camera 20c. Since the side camera 20c is focused on a close distance, the face of the person 150 can be captured clearly without adjusting the focus.

[0018] In the embodiment, face authentication is performed based on the image captured by the side camera 20c at this time. Therefore, the side camera 20c for monitoring the surroundings, which has a simple configuration, can also be used for face authentication without providing a camera dedicated to face authentication. Since the photographable range R1 of the side camera 20c moves by the retracting operation of the side mirror 110 to include the face of the person 150, there is no need to provide a drive mechanism dedicated to face authentication for changing the photographable range R1.

[0019] In contrast, when a typical side mirror is folded, it rotates around a rotation axis that is roughly vertical, and the tip of the mirror is folded so that it faces the rear of the vehicle. In other words, the side mirror is folded while the lower part faces the road surface. Therefore, even when the side mirror is folded, the optical axis of the side camera at the bottom of the side mirror remains roughly vertically downward, and it is not possible to capture the face of an adult person standing to the side of the vehicle.

[0020] The side mirror 110 may be called a flip-up side mirror, and may have a known storage mechanism. As long as the side mirror 110 can be stored in a position where the side camera 20c can capture an image of the face of the person 150 on the side of the vehicle 100, the operation of storing and the orientation of the side mirror 110 in the stored state are not limited to the above-mentioned example. For example, in the storing operation, the side mirror 110 may be rotated around a rotation axis along a substantially vertical direction as in the case of a general side mirror, and then, with the tip of the side mirror 110 facing generally toward the rear of the vehicle, the side mirror 110 may be rotated around a rotation axis along a substantially left-right direction, so that the lower part of the side mirror 110 is raised from the ground and faces the rear of the vehicle.

[0021] Fig. 2 shows a functional configuration of an in-vehicle system 1 according to an embodiment. The in-vehicle system 1 is mounted on the vehicle 100 shown in Fig. 1. The in-vehicle system 1 includes an authentication device 10, a front camera 20a, a rear camera 20b, a side camera 20c, a side camera 20d, an image generating device 22, a display unit 24, a mirror driving unit 26, and a door control unit 28. The authentication device 10, the side camera 20c, the side camera 20d, and the mirror driving unit 26 constitute an authentication system 2.

[0022] The front camera 20a is provided, for example, in the center of the front grill of the vehicle 100, and captures an image in front of the vehicle 100. The rear camera 20b is provided, for example, in the center of the upper end of the rear window of the vehicle 100, and captures an image behind the vehicle 100. As described with reference to FIG. 1(a), the side camera 20c is provided below the left side mirror 110 of the vehicle 100, and captures an image to the left of the vehicle 100. The side camera 20d is provided below the right side mirror of the vehicle 100, and captures an image to the right of the vehicle 100.

[0023] Hereinafter, the front camera 20a, the rear camera 20b, the side camera 20c, and the side camera 20d will be collectively referred to as the camera 20 as appropriate. The camera 20 has a wide-angle lens or a fisheye lens. The camera 20 can also be called a PVM (Panoramic View Monitor) camera or a surroundings monitoring camera. As the camera 20, various known cameras can be used, such as a digital camera with an imaging element such as a CCD (charge coupled device) or a CIS (CMOS image sensor). Each of the multiple cameras 20 transmits data of the captured image to the image generating device 22.

[0024] Image generating device 22 includes a processor, and, for example, when an accessory switch of vehicle 100 is on and the image display mode is PVM mode, executes arithmetic processing and image processing based on image data obtained by multiple cameras 20, and generates a virtual overhead image of vehicle 100 and its surroundings viewed from above. The PVM mode is used, for example, when vehicle 100 is parked. Image generating device 22 supplies the generated overhead image data to display unit 24.

[0025] The display unit 24 includes a display installed inside the vehicle cabin, and displays various images including an overhead image.

[0026] Of the multiple cameras 20, the side camera 20c and the side camera 20d also supply image data to the authentication device 10. For example, when the accessory switch of the vehicle 100 is off, the doors are locked, and the vehicle 100 is parked, the authentication device 10 performs face authentication based on the supplied image data to determine whether or not a person approaching the vehicle 100 is a person registered in advance, and unlocks the doors of the vehicle 100 if the authentication is successful.

[0027] The authentication device 10 comprises an image acquisition unit 40, an authentication unit 42, and a control unit 44. The configuration of the authentication device 10 can be realized in hardware by the CPU, memory, and other LSI of any computer, and in software by a program loaded into memory, but here, functional blocks realized by the cooperation of these are depicted. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various forms by only hardware, only software, or a combination of both.

[0028] The mirror driving unit 26 includes an actuator and a rotation mechanism (not shown), and the actuator operates the rotation mechanism to drive the side mirror 110. The mirror driving unit 26 drives the side mirror 110 from the stored state shown in Fig. 1(b) to the use state shown in Fig. 1(a) and also drives it from the use state to the stored state in response to the operation of an operating unit (not shown) by the driver or the control of the control unit 44. The mirror driving unit 26 includes an actuator and a rotation mechanism for each of the two side mirrors 110 on the left and right.

[0029] The door control unit 28 is, for example, a door ECU. The door control unit 28 locks the doors by controlling a door lock mechanism (not shown) provided on the doors of the vehicle 100 to an activated state. The door control unit 28 also unlocks the doors by driving the door lock mechanism to a deactivated state. The door control unit 28 locks or unlocks the doors in response to an operation of an operation unit (not shown) by the driver or an authentication result by the authentication unit 42. A plurality of operation units may be provided, and for example, one operation unit may be provided in the vehicle cabin and another operation unit may be included in a portable device (not shown) such as an electronic key carried by the driver.

[0030] When the door control unit 28 receives a locking instruction from the operation unit while the vehicle 100 is parked and the driver has exited the vehicle, the door control unit 28 locks the doors and notifies the control unit 44. The fact that the driver has exited the vehicle can be identified using a known technology.

[0031] When the vehicle 100 is parked and the driver has exited the vehicle, and the doors are locked by the door control unit 28, the control unit 44 outputs a mirror folding instruction to the mirror driving unit 26.

[0032] In response to the mirror storage command, the mirror driving unit 26 stores the left side mirror 110 in a position where the left side camera 20c can capture an image of the face of a person on the left side of the vehicle 100. As described with reference to FIG. 1(b), when the side mirror 110 is in the stored state, the face of a person 150 standing on the side of the vehicle 100 is within the angle of view of the side camera 20c. In response to the mirror storage command, the mirror driving unit 26 stores the right side mirror 110 in a position where the right side camera 20d can capture an image of the face of a person on the right side of the vehicle 100.

[0033] The control unit 44 outputs a retraction command to the mirror driving unit 26 to cause the side cameras 20c, 20d to periodically capture images when the side mirror 110 is in the retracted state. The capture frequency of the side cameras 20c, 20d can be appropriately determined by experiment or simulation.

[0034] When the control unit 44 outputs a storage instruction to the mirror driving unit 26, the control unit 44 may cause the side cameras 20c, 20d to capture images only while a wireless signal transmitted from a portable device carried by the driver is being received. In this case, imaging is performed only when the driver carrying the portable device approaches the periphery of the vehicle 100. Alternatively, when the control unit 44 outputs a storage instruction to the mirror driving unit 26, the control unit 44 may cause the side cameras 20c, 20d to capture images for a predetermined period of time when a predetermined operation, such as operating the door handle of the vehicle 100, is performed. This makes it possible to reduce unnecessary imaging operations and power consumption.

[0035] The image acquisition unit 40 acquires an image of a person's face captured by the side cameras 20c, 20d when the side mirror 110 is in the retracted state. The image acquisition unit 40 outputs the acquired image to the authentication unit .

[0036] When the side mirror 110 is in the retracted state, the authentication unit 42 performs face authentication based on the image of the person's face captured by the side cameras 20c, 20d received from the image acquisition unit 40. If the face in the captured image matches a pre-registered face, the authentication unit 42 determines that the authentication is successful. If the face in the captured image does not match a pre-registered face, the authentication unit 42 determines that the authentication is unsuccessful. The authentication unit 42 outputs the authentication result to the door control unit 28. The pre-registered faces may include, in addition to the face of the driver of the vehicle 100, the faces of the driver's family members, for example. A known process can be used for face authentication.

[0037] If the authentication by the authentication unit 42 is successful, the door control unit 28 unlocks the doors, thereby allowing a pre-registered driver or the like to open the doors of the vehicle 100. On the other hand, if the authentication by the authentication unit 42 is unsuccessful, the door control unit 28 keeps the doors locked.

[0038] Next, a description will be given of the overall operation of the authentication system 2 configured as above. Figure 3 is a flowchart showing the processing of the authentication system 2 of Figure 2. The processing of Figure 3 starts when the vehicle 100 is parked.

[0039] The control unit 44 determines whether the doors of the vehicle 100 are locked (S10). If the doors are not locked (N in S10), the control unit 44 returns to the process of S10. If the doors are locked (Y in S10), the mirror driving unit 26 stores the left and right side mirrors 110 (S12), and the side cameras 20c and 20d capture images (S14).

[0040] The authentication unit 42 determines whether the authentication is successful based on the captured image (S16). If the authentication is not successful (N in S16), the process returns to S14. In S14, for example, the side cameras 20c and 20d capture images when a predetermined time has elapsed since the previous image capture. If the authentication is successful (Y in S16), the door control unit 28 unlocks the door (S18), and the authentication device 10 ends the process.

[0041] According to the embodiment, the left and right side mirrors 110 are stored in positions where the side cameras 20c, 20d can capture the face of a person on the side of the vehicle 100, and when the side mirrors 110 are in the stored state, face authentication is performed based on the image of the person's face captured by the side cameras 20c, 20d, so face authentication can be performed using the existing side cameras 20c, 20d for monitoring the surroundings. There is no need to change the structure of the existing side cameras 20c, 20d for monitoring the surroundings. Therefore, face authentication can be performed with a simple configuration without providing a dedicated vehicle-mounted camera. This allows the authentication system 2 to be configured relatively inexpensively.

[0042] The present invention has been described above based on the embodiments. The embodiments are merely examples, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component and each processing step, and that such modifications are also within the scope of the present invention. [Explanation of symbols]

[0043] 2... authentication system, 10... authentication device, 20... camera, 20a... front camera, 20b... rear camera, 20c, 20d... side cameras, 26... mirror drive unit, 28... door control unit, 40... image acquisition unit, 42... authentication unit, 44... control unit, 100... vehicle, 110... side mirror.

Claims

[Claim 1] A camera provided under a side mirror of a vehicle and capable of photographing a road surface on the side near the vehicle when the side mirror is in use; a mirror driving unit that stores the side mirror in a position where the camera can capture an image of a face of a person on the side of the vehicle; an authentication unit that performs face authentication based on an image of a person's face captured by the camera when the side mirror is in a retracted state; An authentication system comprising:

Images