Information processing system, information processing apparatus, information processing method, and computer program

Abstract

To appropriately capture an image of an object using a vehicle.SOLUTION: An information processing system includes first imaging means provided in a vehicle, detection means configured to detect that a predetermined action is performed on a predetermined place to which a target riding in the vehicle directs a line of sight, control means configured to control the first imaging means so as to capture an image of the target when the predetermined action is detected, and authentication means configured to execute an authentication process of the target based on the image of the target captured by the first imaging means.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to the technical fields of an information processing system, an information processing device, an information processing method, and a computer program. [Background technology]

[0002] Known systems of this type perform authentication processing based on biometric information acquired from an image of a target. For example, Patent Document 1 discloses a system for identifying the positions of the target's eyes based on an image of the target's face, capturing an iris image of the target, and performing authentication processing of the target using the captured iris image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2023 / 007714 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of this disclosure is to provide an information processing system, an information processing device, an information processing method, and a computer program that aim to improve upon the techniques disclosed in prior art documents. [Means for solving the problem]

[0005] One aspect of the information processing system disclosed herein comprises a first imaging means provided in a vehicle, a detection means for detecting that a predetermined action has been performed toward a predetermined location toward which a subject riding in the vehicle is looking, a control means for controlling the first imaging means to capture an image of the subject when the predetermined action is detected, and an authentication means for performing an authentication process for the subject based on the image of the subject captured by the first imaging means.

[0006] One aspect of the information processing device disclosed herein includes a detection means for detecting that a predetermined action has been performed toward a predetermined location toward which a subject riding in a vehicle is looking, a control means for controlling a first imaging means provided in the vehicle to capture an image of the subject when the predetermined action is detected, and an authentication means for performing an authentication process for the subject based on the image of the subject captured by the first imaging means.

[0007] One aspect of the information processing method disclosed herein is to have at least one computer detect that a predetermined action has been performed toward a predetermined location toward which a subject riding in the vehicle is looking, and when the predetermined action is detected, control a first imaging means provided in the vehicle to capture an image of the subject, and perform authentication processing of the subject based on the image of the subject captured by the first imaging means.

[0008] One aspect of the computer program of this disclosure causes at least one computer to execute an information processing method that detects that a predetermined action has been performed toward a predetermined location toward which a subject riding in the vehicle is looking, and when the predetermined action is detected, controls a first imaging means provided in the vehicle to capture an image of the subject, and performs authentication processing of the subject based on the image of the subject captured by the first imaging means. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 2 is a block diagram showing a hardware configuration of a first information processing system. [Figure 2] FIG. 2 is a block diagram showing a functional configuration of a first information processing system. [Figure 3] 10 is a flowchart showing the flow of an authentication operation by the first information processing system. [Figure 4] 1 is a schematic diagram illustrating an example of a mirror provided in a vehicle. [Figure 5] 10 is a flowchart showing the flow of an authentication operation by a second information processing system. [Figure 6]FIG. 10 is a block diagram showing the functional configuration of a third information processing system. [Figure 7] 10 is a flowchart showing the flow of an authentication operation by the third information processing system. [Figure 8] FIG. 10 is a block diagram showing the functional configuration of a fourth information processing system. [Figure 9] FIG. 1 is a schematic diagram illustrating an example of a display provided in a vehicle. [Figure 10] 10 is a flowchart showing the flow of an authentication operation by the fourth information processing system. [Figure 11] FIG. 10 is a block diagram showing the functional configuration of a fifth information processing system. [Figure 12] 13 is a flowchart showing the flow of an authentication operation by the fifth information processing system. [Figure 13] FIG. 10 is a block diagram showing the functional configuration of a sixth information processing system. [Figure 14] 10 is a flowchart showing the flow of an authentication operation by the sixth information processing system. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of an information processing system, an information processing device, an information processing method, and a computer program will be described with reference to the drawings.

[0011] First Embodiment The first information processing system will be described with reference to FIGS.

[0012] (Hardware configuration) First, the hardware configuration of the first information processing system will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the first information processing system.

[0013] 1, the information processing system 1 includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, a storage device 14, an input device 15, and an output device 16. The information processing system 1 also includes a camera 20. The processor 11, RAM 12, ROM 13, storage device 14, input device 15, output device 16, and camera 20 are all connected to each other via a data bus 17. The data bus 17 may be an interface other than a data bus (for example, a LAN or a USB).

[0014] The processor 11 loads a computer program. For example, the processor 11 is configured to load a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may load a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may acquire (i.e., load) the computer program from a device (not shown) located outside the information processing system 1 via a network interface. The processor 11 executes the loaded computer program to perform various processes. In particular, in this embodiment, when the processor 11 executes the loaded computer program, functional blocks related to the authentication process performed by the information processing system 1 are realized within the processor 11. In other words, the processor 11 may function as a controller that executes each control in the information processing system 1.

[0015] The processor 11 may be configured as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a quantum processor. The processor 11 may be configured as one of these, or may be configured to use multiple processors in parallel.

[0016] The RAM 12 temporarily stores computer programs executed by the processor 11. The RAM 12 temporarily stores data that the processor 11 temporarily uses while the processor 11 is executing the computer programs. The RAM 12 may be, for example, a dynamic random access memory (DRAM) or a static random access memory (SRAM). Alternatively, other types of volatile memory may be used instead of the RAM 12.

[0017] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store fixed data. The ROM 13 may be, for example, a programmable read-only memory (P-ROM) or an erasable read-only memory (EPROM). Alternatively, the ROM 13 may be replaced by another type of non-volatile memory.

[0018] The storage device 14 stores data that the information processing system 1 stores for a long period of time. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device 14 may store computer programs executed by the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0019] The input device 15 is a device that receives input instructions from a user of the information processing system 1. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may also be, for example, a device that includes a microphone and is capable of voice input.

[0020] The output device 16 is a device that outputs information related to the information processing system 1 to the outside. For example, the output device 16 may be a display device (for example, a display or a monitor) that can display information related to the information processing system 1. The output device 16 may also be a speaker or the like that can output information related to the information processing system 1 as audio.

[0021] The camera 20 is configured to be able to capture an image of a target. The target here is typically a human, but may also be an animal such as a dog or a cat. The camera 20 according to this embodiment is installed in a vehicle. That is, the camera 20 is configured to be able to capture an image of a target that is a vehicle occupant (e.g., a driver or a passenger). A plurality of cameras 20 may be installed in the vehicle. For example, a plurality of cameras 20 may be installed corresponding to the driver's seat, passenger seat, and rear seat of the vehicle. Also, a plurality of cameras 20 may be installed for one target. For example, the camera 20 may be installed as a visible light camera that captures an image of the target's face and a near-infrared camera that captures an image of the target's iris. Specific uses and arrangement examples of the camera 20 will be described in detail later.

[0022] The first information processing system 1 may be configured to include some of the components described in FIG. 1 . For example, the first information processing system 1 may be configured to include only the processor 11, RAM 12, and ROM 13 among the above-mentioned components. For example, the first information processing system 1 may be configured as an information processing device that is a standalone device. In this case, the storage device 14, the input device 15, the output device 16, and the camera 20 may each be provided as a device external to the first information processing system 1. The information processing device may be configured as a device that can control the operation of the storage device 14, the input device 15, the output device 16, and the camera 20. Furthermore, some of the calculation functions of the first information processing system 1 may be realized by an external server, a cloud, or the like. In this way, some functions of the first information processing system 1 may be realized by components provided within the vehicle, and other functions may be realized by components provided outside the vehicle.

[0023] (Functional configuration) Next, the functional configuration of the first information processing system 1 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the functional configuration of the first information processing system.

[0024] 2, the first information processing system 1 is configured to execute authentication processing for a target that uses a vehicle. The first information processing system 1 includes, as components for realizing its functions, a first camera 110, a predetermined action detection unit 120, a control unit 130, and an authentication unit 140. Note that the first camera 110 may be included in the above-mentioned camera 20 (see FIG. 1). Furthermore, each of the predetermined action detection unit 120, the control unit 130, and the authentication unit 140 may be a processing block realized by the above-mentioned processor 11 (see FIG. 1).

[0025] The first camera 110 is configured as a camera capable of capturing an image of a subject using the vehicle. The first camera 110 may be disposed in a position capable of capturing an image of a subject seated in a vehicle seat. For example, the first camera 110 may be disposed near the driver's seat so as to capture an image of the driver of the vehicle. Alternatively, the first camera 110 may be disposed near the passenger seat or rear seat so as to capture an image of a passenger in the vehicle. Alternatively, the first camera 110 may be disposed in a position capable of capturing an image of all the seats in the vehicle with a single camera. A plurality of first cameras 110 may be provided corresponding to the respective seats in the vehicle. The first camera 110 is configured to capture an image capable of acquiring biometric information of the subject. For example, the first camera 110 is configured to capture a face image from which facial features of the subject can be extracted, an iris image from which iris features of the subject can be extracted, or the like. The image of the subject captured by the first camera 110 is configured to be output to the authentication unit 140. The operation of the first camera 110 can be controlled by the control unit 130.

[0026] The predetermined action detection unit 120 is configured to detect a predetermined action by a subject toward a predetermined location. The predetermined location here is a location in the vehicle where the subject intentionally directs their gaze. For example, the predetermined location may be any of various mirrors (e.g., a rearview mirror or a side mirror) provided in the vehicle. Alternatively, the predetermined location may be a display provided in the vehicle (e.g., a front monitor provided around the dashboard or a rear monitor provided on the rear seat side). The predetermined action may be any action that the subject can perform toward the predetermined location. For example, the predetermined action may be the subject touching a mirror or a display. Alternatively, the predetermined action may be the subject directing their gaze toward a mirror or a display. The predetermined action detection unit 120 may be configured to detect the predetermined action using, for example, various sensors. Alternatively, the predetermined action detection unit 120 may be configured to detect the predetermined action based on an image captured by the first camera 110 or another camera. The detection result by the predetermined action detection unit 120 is configured to be output to the control unit 130.

[0027] The control unit 130 is configured to control the first camera 110 when the predetermined action detection unit 120 detects a predetermined action at a predetermined location. For example, the control unit 130 may be configured to control the first camera 110 so that an image of a target is captured when the predetermined action is detected as a trigger. That is, the first camera 110 may be controlled to capture an image of the target at the timing when the predetermined action is detected. The control unit 130 may also be configured to control the position, orientation, and camera parameters (e.g., exposure, shutter speed, etc.) of the first camera 110 when capturing an image with the first camera 110. The control unit 130 may also be configured to control devices other than the first camera 110 in the vehicle. Control of devices other than the first camera 110 will be described in detail in other embodiments described later.

[0028] The authentication unit 140 is configured to be able to perform authentication processing of the target using an image of the target captured by the first camera 110 (specifically, an image captured by the first camera 110 under the control of the control unit 130). The authentication unit 140 may extract biometric information from the image of the target and perform authentication processing. For example, the authentication unit 140 may extract facial features from a facial image of the target and perform face authentication. Alternatively, the authentication unit 140 may extract iris features from an iris image of the target and perform iris authentication. The authentication unit 140 may perform authentication processing of the target by comparing the biometric information of the target with pre-registered registration information (i.e., biometric information of a registered user). For example, the authentication unit 140 may determine that the target and the registered user are the same person when the degree of match between the biometric information of the target and the registration information exceeds a predetermined value. Furthermore, the authentication unit 140 may determine that the target and the registered user are not the same person when the degree of match between the biometric information of the target and the registration information is less than a predetermined value. The authentication unit 140 may be configured to be able to output the result of the authentication process.

[0029] (Authentication operation) Next, the flow of the authentication operation (i.e., a series of operations when executing target authentication processing) by the first information processing system 1 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of the authentication operation by the first information processing system.

[0030] 3, when the authentication operation by the first information processing system 1 is started, the predetermined movement detection unit 120 first determines whether or not a predetermined movement to a predetermined location has been detected (step S101). If a predetermined movement has not been detected (step S101: NO), the subsequent operations may be omitted. In this case, the process may be started again from step S101 after a predetermined period of time has elapsed.

[0031] On the other hand, if a predetermined action is detected (step S101: YES), control unit 130 controls first camera 110 (step S102). First camera 110 controlled by control unit 130 captures an image of a target and acquires a target image (step S103). The acquired target image is output to authentication unit 140.

[0032] Note that first camera 110 may be configured to constantly capture images within its imaging range, rather than capturing images at timing controlled by control unit 130. In this case, first camera 110 may be configured to output to authentication unit 140 only some of the frame images captured at timing controlled by control unit 130, among a plurality of frame images captured in chronological order. In this case, the image output to authentication unit 140 does not have to be only one frame, and for example, two or more consecutive frames may be output to authentication unit 140.

[0033] Next, the authentication unit 140 performs authentication processing of the target using the target image captured by the first camera 110 (step S104). Then, the authentication unit 140 outputs the result of the authentication processing (step S105). The result of the authentication processing by the authentication unit 140 may be displayed, for example, on a display provided in the vehicle. For example, the authentication unit 140 may display information indicating whether the authentication was successful or not, information about the target identified in the authentication processing, etc., on the display of the vehicle. Furthermore, various processes in the vehicle may be permitted depending on the result of the authentication processing by the authentication unit 140. For example, if the authentication by the authentication unit 140 is successful, processes related to driving of the vehicle (for example, engine start, steering, etc.) may be permitted. On the other hand, if the authentication by the authentication unit 140 fails, processes related to driving of the vehicle may be prohibited.

[0034] (Technical Effects) Next, the technical effects obtained by the first information processing system 1 will be described.

[0035] As described with reference to FIGS. 1 to 3, in the first information processing system 1, when a predetermined action is performed at a predetermined location, the first camera 110 is controlled to capture an image of the target. In this way, it is possible to appropriately capture an image of the target using the vehicle. For example, if the imaging range of the first camera 110 is set to match the target performing the predetermined action, an image can be captured at a timing when the target falls within the imaging range. In this case, the imaging range of the first camera 110 may be set depending on which part of the target is to be imaged (i.e., which part is to be used to perform authentication processing). For example, when the authentication unit 140 performs face authentication or iris authentication, the imaging range of the first camera 110 may be set so that the face of the target performing the predetermined action falls within the imaging range.

[0036] Second Embodiment The second information processing system 1 will be described with reference to Figures 4 and 5. The second information processing system 1 differs in some configurations and operations from the above-described first information processing system 1, but other parts may be similar to the first information processing system 1. Therefore, hereinafter, only the parts that differ from the first embodiment will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0037] (Mirror configuration) First, the configuration of a mirror, which is an example of a predetermined location in the second information processing system 1, will be described with reference to Fig. 4. Fig. 4 is a schematic diagram showing an example of a mirror provided on a vehicle.

[0038] 4, in the second information processing system 1, a mirror provided on a vehicle is set as the predetermined location. For example, in the second information processing system 1, at least one of a rearview mirror 51 and a side mirror 52 of the vehicle may be set as the predetermined location. The rearview mirror 51 is provided with a rearview mirror camera 511 that captures an image in the direction in which the rearview mirror 51 is facing. The rearview mirror camera 511 is configured to move integrally with the rearview mirror 51. The side mirror 52 is provided with a side mirror camera 521 that captures an image in the direction in which the side mirror 52 is facing. The side mirror camera 521 is configured to move integrally with the side mirror 52.

[0039] The mirror set at a predetermined location in the second information processing system 1 may be a general mirror or a component having the same function as a mirror. For example, the mirror may be a back monitor that functions as a rearview mirror 51 by displaying an image of the area behind the vehicle, or a side monitor that functions as a side mirror 52 by displaying an image of the area to the side of the vehicle.

[0040] The predetermined action detection unit 120 in the second information processing system 1 is configured to be able to detect contact with a mirror, which is a predetermined location, as the predetermined action. For example, the predetermined action detection unit 120 may detect an action of the subject touching the rearview mirror 51 or the side mirror 52 with a hand as the predetermined action. More specifically, the predetermined action detection unit 120 may detect an adjustment action of the subject touching the rearview mirror 51 or the side mirror 52 with a hand to adjust the angle of the mirror as the predetermined action. Alternatively, the predetermined action detection unit 120 may detect an action of the subject touching an adjustment switch that adjusts the rearview mirror 51 or the side mirror 52 with a hand as the predetermined action. More specifically, the predetermined action detection unit 120 may detect an action of the subject operating an adjustment switch to adjust the angle of the rearview mirror 51 or the side mirror 52 as the predetermined action.

[0041] (Authentication operation) Next, the flow of authentication operations by the second information processing system 1 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of authentication operations by the second information processing system. Note that in Fig. 5, the same processes as those shown in Fig. 3 are denoted by the same reference numerals.

[0042] 5, when the authentication operation by the second information processing system 1 is started, the predetermined operation detection unit 120 first determines whether or not an adjustment operation for the mirror has been detected (step S201). If an adjustment operation for the mirror has not been detected (step S201: NO), the subsequent operations may be omitted. In this case, the process may be started again from step S201 after a predetermined period of time has elapsed.

[0043] On the other hand, if an adjustment operation on the mirror is detected (step S201: YES), control unit 130 determines whether a predetermined time (e.g., several seconds) has passed since the adjustment operation was completed, or whether the subject has released their hand from the mirror after the adjustment operation (step S202). Then, if it is determined that the predetermined time has passed since the adjustment operation was completed, or that the subject has released their hand from the mirror (step S202: YES), control unit 130 controls first camera 110 (step S102). First camera 110 controlled by control unit 130 captures an image of the subject and acquires a target image (step S103). Note that if it is determined that the predetermined time has not passed since the adjustment operation was completed, or that the subject has not released their hand from the mirror (step S202: NO), control unit 130 repeats the determination in step S202.

[0044] Controlling first camera 110 as described above can prevent an image of the target from being captured while the target is adjusting the mirror (i.e., while the mirror is still moving). While the target is adjusting the mirror, first camera 110 (e.g., rearview mirror camera 511 or side mirror camera 521) mounted on the mirror also moves, and capturing an image in this state may result in blurring. Therefore, if an image of the target is captured while no adjustment is being performed (more specifically, immediately after the adjustment is completed), it is possible to capture an appropriate image without blurring.

[0045] Thereafter, authentication unit 140 executes authentication processing of the target using the target image captured by first camera 110 (step S104), and then outputs the result of the authentication processing (step S105).

[0046] (Technical Effects) Next, the technical effects obtained by the second information processing system 1 will be described.

[0047] As described with reference to FIGS. 4 and 5 , in the second information processing system 1, touching a predetermined position is detected as a predetermined action, and an image of the object that touched the predetermined position is captured. In this manner, the touching of the predetermined position serves as a trigger to appropriately capture an image of the object. For example, a subject adjusting the rearview mirror 51 adjusts the angle while looking at the rearview mirror 51. Therefore, if the rearview mirror camera 511 captures an image of the subject facing the direction of the rearview mirror 51 when the adjustment of the rearview mirror 51 is detected, the face and iris of the subject facing the direction of the rearview mirror 51 can be appropriately captured. Similarly, a subject adjusting the side mirror 52 adjusts the angle while looking at the side mirror 52. Therefore, if the side mirror camera 521 captures an image of the subject facing the direction of the side mirror 52 when the adjustment of the side mirror 52 is detected, the face and iris of the subject facing the direction of the side mirror 52 can be appropriately captured.

[0048] Third Embodiment The third information processing system 1 will be described with reference to Figures 6 and 7. The third information processing system 1 differs in some configurations and operations from the first and second information processing systems 1 described above, but other parts may be similar to the first and second information processing systems 1. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0049] (Functional configuration) First, the functional configuration of the third information processing system 1 will be described with reference to Fig. 6. Fig. 6 is a block diagram showing the functional configuration of the third information processing system. Note that in Fig. 6, the same elements as those shown in Fig. 2 are denoted by the same reference numerals.

[0050] 6, the third information processing system 1 includes, as components for realizing its functions, a first camera 110, a predetermined action detection unit 120, a control unit 130, an authentication unit 140, a second camera 150, and a gaze estimation unit 160. That is, the third information processing device 1 further includes the second camera 150 and the gaze estimation unit 160 in addition to the configuration of the first information processing device already described (see FIG. 2). Note that the second camera 150 may be included in the above-described camera 20 (see FIG. 1). Furthermore, the gaze estimation unit 160 may be a processing block realized by the above-described processor 11 (see FIG. 1).

[0051] The second camera 150 is configured as a camera capable of capturing an image of the face of a subject using the vehicle. The second camera 150 may be disposed in a position capable of capturing an image of a subject seated in a vehicle seat. For example, the second camera 150 may be disposed near the driver's seat so as to capture an image of the face of the vehicle driver. Alternatively, the second camera 150 may be disposed near the passenger seat or rear seat so as to capture an image of the faces of the vehicle passengers. Alternatively, the second camera 150 may be disposed in a position capable of capturing an image of all the seats in the vehicle with a single camera. A plurality of second cameras 150 may be provided corresponding to the respective seats in the vehicle. The second camera 150 is configured to capture an image for estimating the gaze direction of the subject. Therefore, the facial image captured by the second camera 150 may be an image including the area around the subject's eyes rather than an image including the entire face of the subject. The facial image of the subject captured by the second camera 150 is configured to be output to the gaze estimation unit 160.

[0052] While the above-described second camera 150 is configured to be able to capture a facial image of the subject, the first camera 110 in the third information processing system 1 is configured to be able to capture an iris image of the subject. That is, the first camera 110 and the second camera 120 are configured as cameras that capture images of different parts of the subject. However, the imaging range of the first camera 110 and the imaging range of the second camera 150 may overlap each other.

[0053] Furthermore, the locations where the first camera 110 and the second camera 150 are disposed are not particularly limited. For example, the first camera 110 and the second camera 150 may be disposed in the same location or in different locations. More specifically, the first camera 110 and the second camera 150 may both be disposed in predetermined locations (first arrangement configuration). Alternatively, the first camera 110 may be disposed in a predetermined location, while the second camera 150 may be disposed near the predetermined location (second arrangement configuration). Alternatively, the second camera 150 may be disposed in a predetermined location, while the first camera 110 may be disposed near the predetermined location (third arrangement configuration). Alternatively, the second camera 150 may be disposed in a predetermined location, while multiple first cameras 110 may be disposed to correspond to each seat in the vehicle (fourth arrangement configuration). In the fourth arrangement configuration, the second camera 150 may be disposed in a location where the entire interior of the vehicle can be captured within its imaging range, and the multiple first cameras 110 may be disposed in locations where they can capture images of the subjects sitting in each seat from the front.

[0054] In each of the above arrangement configurations, the camera provided at the predetermined location is arranged so as to move integrally with the predetermined location. On the other hand, the camera provided near the predetermined location is arranged so as not to move integrally with the predetermined location. For example, if the predetermined location is the rearview mirror 51 of the vehicle (see FIG. 4), the camera provided at the predetermined location is configured as a rearview mirror camera 511 that moves integrally with the rearview mirror 51. On the other hand, the camera provided near the predetermined location may be provided on another member located in the vicinity of the rearview mirror 51. For example, the camera provided near the predetermined location may be arranged near the steering wheel, dashboard, speedometer, fuel meter, engine start button, front monitor, etc.

[0055] By arranging first camera 110 and second camera 150 as described above, it is possible to properly capture an image of the target. For example, by arranging first camera 110 in a position where the iris of the target is clearly visible and second camera 150 in a position where the face of the target is clearly visible, it is possible to properly capture an image of the target's face and iris.

[0056] The gaze estimation unit 160 is configured to be able to estimate the gaze direction of the target based on the facial image of the target captured by the second camera 150. The gaze estimation unit 160 may estimate the gaze direction of the target based on, for example, the positions of the eyes (for example, the positions of the whites and pupils of the eyes) included in the facial image. Information regarding the gaze direction of the target estimated by the gaze direction estimation unit 160 is configured to be output to the predetermined action detection unit 120.

[0057] The predetermined action detection unit 120 in the third information processing system 1 is configured to be able to detect a predetermined action based on the direction of the gaze of the target estimated by the gaze estimation unit 160. Specifically, the predetermined action detection unit 120 detects, as the predetermined action, an action of directing the gaze of the target to a predetermined location. For example, the predetermined action detection unit 120 may detect, as the predetermined action, an action of the target directing the gaze of the target to the rearview mirror 51 or the side mirror 52, which is a predetermined location.

[0058] (Authentication operation) Next, the flow of the authentication operation by the third information processing system 1 will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the flow of the authentication operation by the third information processing system.

[0059] 7, when the authentication operation by the third information processing system 1 starts, first, second camera 150 captures an image of the face of the target using the vehicle to acquire a facial image of the target (step S301). Then, gaze estimation unit 160 estimates the gaze direction of the target based on the facial image of the target captured by second camera 150 (step S302). Note that if gaze estimation unit 160 cannot correctly estimate the gaze direction, the processes of steps S301 and S302 may be repeatedly executed. That is, the facial image of the target may be repeatedly captured until the gaze direction can be estimated.

[0060] Next, the predetermined action detection unit 120 determines whether the subject is directing their gaze to a predetermined location based on the gaze direction estimated by the gaze estimation unit 160 (step S303). In this way, by using the gaze direction estimated from the face image, it is possible to appropriately detect the action of directing their gaze to a predetermined location. Note that if it is determined that the subject is not directing their gaze to a predetermined location (step S303: NO), the processing from step S301 may be repeated. That is, the face image of the subject may be repeatedly captured to estimate the gaze direction until it is determined that the subject is directing their gaze to a predetermined location.

[0061] On the other hand, if it is determined that the subject is directing their gaze toward a predetermined location (step S303: YES), control unit 130 controls first camera 110 (step S304). First camera 110 controlled by control unit 130 captures an image of the subject and acquires an iris image of the subject (step S305). At this time, first camera 110 may capture the iris image according to the position of the subject's eyes estimated from the face image captured by second camera 150. For example, the angle, etc. of first camera 110 may be controlled so that the position of the subject's eyes estimated from the face image falls within the imaging range. The iris image acquired by first camera 110 is output to authentication unit 140.

[0062] Next, authentication unit 140 performs iris authentication of the target using the iris image captured by first camera 110 (step S306). Authentication unit 140 may perform iris authentication of the target by comparing features related to the target's iris extracted from the iris image of the target with features related to the iris of the registered user extracted from the iris image of the registered user. Thereafter, authentication unit 140 outputs the result of iris authentication (step S307).

[0063] (Technical Effects) Next, the technical effects obtained by the third information processing system 1 will be described.

[0064] As described in FIGS. 6 and 7, in the third information processing system 1, the action of directing the gaze to a predetermined location is detected as the predetermined action, and an image of the target that has directed the gaze to the predetermined location is captured. In this way, the directing of the gaze to a predetermined location can be used as a trigger to appropriately capture an image of the target. For example, if the action of directing the gaze to the rearview mirror 51 is detected, and the target is controlled to be imaged by the rearview mirror camera 511, the iris image of the target facing the direction of the rearview mirror 51 can be captured from a position close to the front. Similarly, if the action of directing the gaze to the side mirror 52 is detected, and the target is controlled to be imaged by the side mirror camera 521, the iris image of the target facing the direction of the side mirror 52 can be captured from a position close to the front.

[0065] <Fourth embodiment> The fourth information processing system 1 will be described with reference to Figures 8 to 10. The fourth information processing system 1 differs in some configurations and operations from the first to third information processing systems 1 described above, but other parts may be similar to the first to third information processing systems 1. Therefore, hereinafter, only the parts that differ from the embodiments already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0066] (Functional configuration) First, the functional configuration of the fourth information processing system 1 will be described with reference to Fig. 8. Fig. 8 is a block diagram showing the functional configuration of the fourth information processing system. Note that in Fig. 8, the same elements as those shown in Fig. 6 are denoted by the same reference numerals.

[0067] 8, the fourth information processing system 1 includes, as components for realizing its functions, a first camera 110, a predetermined action detection unit 120, a control unit 130, an authentication unit 140, a second camera 150, a gaze estimation unit 160, a guidance information output unit 170, and a display 180. That is, the fourth information processing device 1 further includes the guidance information output unit 170 and the display 180 in addition to the configuration of the third information processing device already described (see FIG. 6). The guidance information output unit 170 may be a processing block realized by the above-described processor 11 (see FIG. 1). The display 180 may be included in the above-described output device 16 (see FIG. 1).

[0068] The guidance information output unit 170 is configured to be able to output guidance information that guides the gaze of the target. The guidance information is information that guides the target to direct the gaze of the target toward the display 180. The guidance information output unit 170 guides the gaze of the target by displaying a message, image, or the like corresponding to the guidance information on the display. The guidance information output unit 170 may output the guidance information when the gaze of the target is not directed toward the display 180. In other words, the guidance information output unit 170 may not output the guidance information when the gaze of the target is already directed toward the display 180. Specific display examples of the guidance information will be described in detail later.

[0069] The display 180 is a display device provided in a vehicle. The display 180 may be a front monitor provided nearby. Alternatively, the display 180 may be a rear monitor for passengers in the back seats. Alternatively, the display may be a rear monitor that functions as a rearview mirror, or a side monitor that functions as a side mirror. The display 180 is set as a predetermined location in the fourth information processing system 1. The predetermined action detection unit 120 in the fourth information processing system 1 is configured to be able to detect, as a predetermined action, an action of directing the gaze toward the display 180, which is a predetermined location.

[0070] (Example of guidance information display) First, an example of displaying guidance information in the fourth information processing system 1 will be described with reference to Fig. 9. Fig. 4 is a schematic diagram showing an example of a display provided in a vehicle.

[0071] 9, a rear monitor 53 installed on the rear seat side of the vehicle is set as the predetermined location. The rear monitor 53 is provided with a rear monitor camera 531 that captures an image in the direction in which the rear monitor 53 is facing. The rear monitor camera 531 is configured to move integrally with the rear monitor 53.

[0072] 9 shows a configuration in which two rear monitors 53 are provided, but it is sufficient that at least one rear monitor 53 is provided. Alternatively, only one of the multiple rear monitors 53 may be set as the predetermined location. Furthermore, a display other than the rear monitor 53 provided in the vehicle may be set as the predetermined location.

[0073] The rear monitor 53, which is a predetermined location, is configured to be able to display guidance information that guides the direction of the subject's gaze. In the example shown in FIG. 9, a message saying "Look here" is displayed on the rear monitor 53. If such guidance information is displayed, there is a high possibility that the subject will consciously direct their gaze toward the rear monitor 53. Note that instead of such a message, an image or video that attracts the subject's attention may be displayed. For example, a distinctive avatar may be displayed on the rear monitor 53, and the video may be one in which the avatar issues a message. Alternatively, an image or video that the subject is interested in may be displayed. In this case, attribute information of the subject (e.g., gender, age, etc.) may be acquired to determine the guidance information to be displayed. Furthermore, in addition to displaying the guidance information, audio guidance may be provided. For example, the message "Look here" may be displayed on the rear monitor 53, and an audio message saying "Look at the monitor" may be output from a speaker.

[0074] (Authentication operation) Next, the flow of authentication operations by the fourth information processing system 1 will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the flow of authentication operations by the fourth information processing system. Note that in Fig. 10, the same processes as those shown in Fig. 7 are denoted by the same reference numerals.

[0075] 10, when the authentication operation by the fourth information processing system 1 is started, first, the second camera 150 captures an image of the face of the target using the vehicle to acquire a facial image of the target (step S301). Then, the gaze estimation unit 160 estimates the gaze direction of the target based on the facial image of the target captured by the second camera 150 (step S302).

[0076] Next, the predetermined action detection unit 120 determines whether the target is directing their gaze to a predetermined location based on the gaze direction estimated by the gaze estimation unit 160 (step S303). If it is determined that the target is not directing their gaze to a predetermined location (step S303: NO), the guidance information output unit 170 displays guidance information on the display 180 (step S401). After the guidance information is displayed, the process from step S301 is repeated. That is, until it is determined that the target is directing their gaze to the display 180, which is the predetermined location, the facial image of the target is repeatedly captured to estimate the gaze direction.

[0077] The guidance information output unit 170 may display the guidance information on the display 180 from the beginning, regardless of whether the target is directing its gaze toward the display 180. In this case, the guidance information output unit 170 may display the guidance information at the timing when it is detected that the target has gotten into the vehicle, for example.

[0078] If it is determined that the target is directing its gaze toward a predetermined location as a result of displaying the guidance information (step S303: YES), the control unit 130 controls the first camera 110 (step S304). The first camera 110 controlled by the control unit 130 captures an image of the target and acquires an iris image of the target (step S305). Note that the guidance information output unit 170 may end the display of the guidance information when it is determined that the target is directing its gaze toward a predetermined location. Alternatively, the guidance information output unit 170 may end the display of the guidance information when it has acquired an iris image of the target.

[0079] Next, authentication unit 140 performs iris authentication of the target using the iris image captured by first camera 110 (step S306). At this time, instead of guidance information, display 180 may display information indicating that authentication is in progress. For example, display 180 may display a message saying "Authentication in progress." Thereafter, authentication unit 140 outputs the result of iris authentication (step S307). The authentication result by authentication unit 140 may be displayed on display 180.

[0080] (Technical Effects) Next, the technical effects obtained by the fourth information processing system 1 will be described.

[0081] 8 to 10, in the fourth information processing system 1, guidance information for guiding the subject's line of sight is displayed on a predetermined location, that is, the display 180. In this way, the subject's line of sight is guided toward the display, and it becomes possible to appropriately capture an iris image of the subject.

[0082] Fifth Embodiment The fifth information processing system 1 will be described with reference to Figures 11 and 12. The fifth information processing system 1 differs in some configurations and operations from the above-described first to fourth information processing systems 1, but other parts may be similar to the first to fourth information processing systems 1. Therefore, hereinafter, only the parts that differ from the embodiments already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0083] (Functional configuration) First, the functional configuration of the fifth information processing system 1 will be described with reference to Fig. 11. Fig. 11 is a block diagram showing the functional configuration of the fifth information processing system. In Fig. 11, the same elements as those shown in Fig. 2 are denoted by the same reference numerals.

[0084] 11, the fifth information processing system 1 includes, as components for realizing its functions, a first camera 110, a predetermined action detection unit 120, a control unit 130, an authentication unit 140, and an object identification unit 190. That is, the fifth information processing device 1 further includes the object identification unit 190 in addition to the configuration of the first information processing device already described (see FIG. 2). Note that the object identification unit 190 may be a processing block realized by the above-described processor 11 (see FIG. 1).

[0085] The target identification unit 190 is configured to be able to identify a target using the vehicle. For example, the target identification unit 190 may be configured to identify which of a set of pre-registered target candidates the target currently using the vehicle is. The target identification unit 190 may identify the target based on an image captured by a camera 20 installed in the vehicle. For example, the target identification unit 190 may identify the target by estimating the target's height, gender, age, etc. from the image. Alternatively, the target identification unit 190 may identify the target using various sensors installed in the vehicle. For example, the target identification unit 190 may measure the target's weight using a weight sensor installed in the vehicle seat and identify the target based on the measurement result. Note that in the case of a private car used at home, the number of target candidates is limited. For example, in a family consisting of a father, a mother, and two small children, the target candidates who will sit in the driver's seat are limited to either the father or the mother. Therefore, when identifying the target sitting in the driver's seat, the target identification unit 190 only needs to determine whether the target is the father or the mother. In this way, the process of identifying the target by the target identifying unit 190 is relatively simple. Information on the target identified by the target identifying unit 190 is configured to be output to the control unit .

[0086] The control unit 130 in the fifth information processing system 1 includes a seat position adjustment unit 1301. The seat position adjustment unit 1301 is configured to adjust the seat position of a vehicle seat. For example, the seat position adjustment unit 1301 is configured to adjust the front-to-rear position of the seat, the inclination of the seat, and the like. The seat position adjustment unit 1301 adjusts the seat position to an imaging position suitable for imaging an object. The imaging position is, for example, a position where the face and eyes of the object sitting in the seat are within the imaging range of the first camera 110. Note that the imaging position is set in advance for each object. The seat position adjustment unit 1301 then reads out the imaging position corresponding to the object identified by the object identification unit 190 and adjusts the seat position. For example, the seat position adjustment unit 1301 stores an imaging position for the father and an imaging position for the mother. When the object identification unit 190 identifies the object as the father, the seat position adjustment unit 1301 adjusts the seat position to an imaging position suitable for the father. On the other hand, if the object is identified by the object identifying unit 190 as the mother, the seat position adjusting unit 1301 adjusts the seat position so that the imaging position is suitable for the mother.

[0087] In this way, by adjusting the seat position according to the identified target, it is possible to capture an image of the target more appropriately with first camera 110. For example, even if the eye positions of different targets when sitting in the seat vary greatly, the eye positions of the target can be changed by adjusting the seat position. For example, for a target whose eye position is relatively high (e.g., a tall target), the eye position can be lowered by adjusting the seat to tilt more (i.e., closer to horizontal). On the other hand, for a target whose eye position is relatively low (e.g., a short target), the eye position can be raised by adjusting the seat to tilt less (i.e., closer to vertical). Therefore, even for targets whose eye positions vary, it is possible to fit the eye positions within the imaging range of first camera 110.

[0088] The seat position adjustment unit 1301 may also be configured to adjust the seat position after capturing an image of the target so that the target is in a driving position suitable for driving. The driving position is, for example, a position where the target sitting in the driver's seat can properly view the surroundings of the vehicle. The driving position may also be set in advance for each target, similar to the imaging position. In this case, the seat position adjustment unit 1301 may read out a driving position corresponding to the target identified by the target identification unit 190 and adjust the seat position. Furthermore, for seats other than the driver's seat, driving positions suitable for passengers when the vehicle is traveling may be set. For example, a driving position for a passenger seat or a driving position for a rear seat may be set. In this case, the seat position adjustment unit 1301 may adjust the seat position of the driver's seat to the driving position while adjusting the seat positions of the passenger seat and the rear seat to the driving positions. The driving positions may also be set in advance for each target, similar to the driving position. In this case, the seat position adjustment unit 1301 may read out a driving position corresponding to the target identified by the target identification unit 190 and adjust the seat position. In this way, by changing the seat position again after the image is captured, it is possible to prevent the vehicle from starting to travel with the image capture position (that is, with the seat position unsuitable for traveling the vehicle) remaining.

[0089] (Authentication operation) Next, the flow of authentication operations performed by the fifth information processing system 1 will be described with reference to Fig. 12. Fig. 12 is a flowchart showing the flow of authentication operations performed by the fifth information processing system. Note that in Fig. 12, the same processes as those shown in Fig. 3 are denoted by the same reference numerals.

[0090] 12, when the authentication operation by the fifth information processing system 1 is started, first, the object identification unit 190 identifies the object using the vehicle (step S501). Then, the predetermined action detection unit 120 determines whether or not a predetermined action at a predetermined location has been detected (step S101). Note that if the predetermined action has not been detected (step S101: NO), the subsequent operations may be omitted. In this case, after a predetermined period of time has elapsed, the process may start again from step S101.

[0091] On the other hand, if a predetermined action is detected (step S101: YES), seat position adjustment unit 1301 adjusts the seat position so that the seat is positioned to capture an image according to the target identified by target identification unit 190 (step S502). After adjusting the seat position, control unit 130 controls first camera 110 (step S102). First camera 110 controlled by control unit 130 captures an image of the target and acquires a target image (step S103). The acquired target image is output to authentication unit 140.

[0092] Next, the authentication unit 140 performs authentication processing of the target using the target image captured by the first camera 110 (step S104). Then, the authentication unit 140 outputs the result of the authentication processing (step S105). After the result of the authentication processing is output, the seat position adjustment unit 1301 adjusts the seat position so that it is in a driving position or a traveling position according to the identified target (step S503). The authentication unit 140 may allow the vehicle to travel on the condition that the seat position has been adjusted to the driving position or the traveling position. For example, even if the authentication processing of the target is successful, the vehicle may not be allowed to travel if the seat position is in the imaging position.

[0093] Note that setting the seat position to the driving position or the running position may result in the first camera 110 being unable to capture an appropriate image of the target. For example, for an iris image with a narrow imaging range, setting the seat position to the driving position or the running position may result in the iris being outside the imaging range. In this case, the authentication process after the vehicle starts traveling may be switched from iris authentication using an iris image to face authentication using a face image.

[0094] (Technical Effects) Next, the technical effects obtained by the fifth information processing system 1 will be described.

[0095] 11 and 12, in the fifth information processing system 1, the seat position is controlled when capturing an image of a target. In this way, the first camera 110 is controlled in a state suitable for capturing an image, so that an image of the target can be captured more appropriately.

[0096] Sixth Embodiment The sixth information processing system 1 will be described with reference to Figures 13 and 14. The sixth information processing system 1 differs in some configurations and operations from the first to fifth information processing systems 1 described above, but other parts may be similar to the first to fifth information processing systems 1. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0097] (Functional configuration) First, the functional configuration of the sixth information processing system 1 will be described with reference to Fig. 13. Fig. 13 is a block diagram showing the functional configuration of the sixth information processing system. In Fig. 13, the same elements as those shown in Fig. 11 are denoted by the same reference numerals.

[0098] 13, the sixth information processing system 1 includes, as components for realizing its functions, a first camera 110, a predetermined action detection unit 120, a control unit 130, an authentication unit 140, and an object identification unit 190. In particular, the control unit 130 in the sixth information processing device 1 includes a seat position adjustment unit 1301, a mirror adjustment unit 1302, and an illumination adjustment unit 1303. That is, the control unit 130 in the sixth information processing device 1 is configured to further include a mirror adjustment unit 1302 and an illumination adjustment unit 1303 in addition to the configuration of the fifth information processing device 1 already described (see FIG. 11).

[0099] The mirror adjustment unit 1302 is configured to be able to adjust a mirror provided in the vehicle. The mirror adjustment unit 1302 is capable of automatically adjusting the angle of the mirror, for example. The mirror is provided with a first camera 110. Therefore, when the mirror adjustment unit 1302 adjusts the angle of the mirror, the angle of the first camera 110 provided in the mirror (in other words, the imaging range) also changes. The mirror adjustment unit 1302 adjusts the mirror according to the adjustment amount of the seat position by the seat position adjustment unit 1301. For example, the mirror adjustment unit 1302 adjusts the mirror so that the face of a target sitting in a seat set to the imaging position by the seat position adjustment unit 1301 can be captured at a more appropriate angle. Furthermore, when the seat position adjustment unit 1301 adjusts the seat to the driving position, the mirror adjustment unit 1302 may adjust the mirror so that the angle is appropriate for driving. The adjustment amount of the mirror may be set in advance for each target. That is, the mirror adjustment unit 1302 may adjust the mirror according to the target identified by the target identification unit 190.

[0100] The lighting adjustment unit 1303 is configured to adjust lighting provided in the vehicle. The lighting here refers to lighting used for capturing images with the first camera 110. For example, if the first camera 110 is configured as a near-infrared camera capturing images of the iris of a target, the lighting may be lighting that irradiates near-infrared light toward the target. The lighting adjustment unit 1303 is configured to automatically adjust, for example, the position, angle, on / off, intensity, etc. of the lighting. The lighting adjustment unit 1303 adjusts the lighting according to the amount of seat position adjustment by the seat position adjustment unit 1301. For example, the lighting adjustment unit 1303 adjusts the lighting so that lighting is irradiated at a more appropriate angle and intensity onto the face of the target sitting in the seat set to the capturing position by the seat position adjustment unit 1301. Furthermore, the lighting adjustment unit 1302 may adjust the lighting to create a lighting environment suitable for driving when the seat is adjusted to the driving position by the seat position adjustment unit 1301. The amount of lighting adjustment may be set in advance for each target. That is, the illumination adjustment unit 1303 may adjust the illumination in accordance with the target identified by the target identification unit 190.

[0101] (Authentication operation) Next, the flow of authentication operations by the sixth information processing system 1 will be described with reference to Fig. 14. Fig. 14 is a flowchart showing the flow of authentication operations by the sixth information processing system. Note that in Fig. 14, the same processes as those shown in Fig. 12 are denoted by the same reference numerals.

[0102] 14, when the authentication operation by the sixth information processing system 1 is started, first, the object identification unit 190 identifies the object using the vehicle (step S501). After that, the predetermined action detection unit 120 determines whether or not a predetermined action at a predetermined location has been detected (step S101). Note that if the predetermined action has not been detected (step S101: NO), the subsequent operations may be omitted. In this case, after a predetermined period of time has elapsed, the process may start again from step S101.

[0103] On the other hand, if a predetermined movement is detected (step S101: YES), the seat position adjustment unit 1301 adjusts the seat position so that the imaging position corresponds to the target identified by the target identification unit 190 (step S502). Then, the mirror adjustment unit 1302 adjusts the mirror according to the adjustment amount of the seat position (step S601). Furthermore, the lighting adjustment unit 1303 adjusts the lighting according to the adjustment amount of the seat position (step S602).

[0104] After the seat position, mirrors, and lighting have been adjusted, the control unit 130 controls the first camera 110 (step S102). The first camera 110 controlled by the control unit 130 captures an image of the target and acquires the target image (step S103). The acquired target image is output to the authentication unit 140.

[0105] Next, the authentication unit 140 performs authentication processing of the target using the target image captured by the first camera 110 (step S104). Then, the authentication unit 140 outputs the result of the authentication processing (step S105). After the result of the authentication processing is output, the seat position adjustment unit 1301 adjusts the seat position so that the seat is in a driving position or a traveling position corresponding to the identified target (step S503). At this time, the mirror adjustment unit 1302 may adjust the mirrors to match the driving position adjusted by the seat position adjustment unit 1301. For example, the mirror adjustment unit 1302 may adjust the angle of the rearview mirror 51 to an angle that allows the driver to check behind the vehicle. Furthermore, the mirror adjustment unit 1302 may adjust the angle of the side mirror 52 to an angle that allows the driver to check to the side of the vehicle. Furthermore, the lighting adjustment unit 1302 may adjust the lighting to create a lighting environment suitable for the driving position. For example, the lighting adjustment unit 1303 may adjust the intensity of the lighting to be weaker so as not to interfere with driving. Alternatively, the lighting adjustment unit 1303 may turn off the lighting.

[0106] (Technical Effects) Next, the technical effects obtained by the sixth information processing system 1 will be described.

[0107] As described in FIGS. 13 and 14 , in the sixth information processing system 1, the mirror equipped with the first camera 110 is also adjusted according to the adjustment amount of the seat position. In this way, the first camera 110 is driven integrally with the mirror being adjusted, making it possible to capture an image of the target more appropriately than when only the seat position is adjusted. In other words, it becomes possible to capture an image of the target at an appropriate angle. Furthermore, in the sixth information processing system 1, the lighting is also adjusted according to the adjustment amount of the seat position. In this way, a lighting environment suitable for capturing an image can be realized, making it possible to capture an image of the target more appropriately than when only the seat position is adjusted.

[0108] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, each embodiment includes not only a recording medium on which the above-described program is recorded, but also the program itself.

[0109] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. The scope of each embodiment is not limited to programs recorded on the recording media that execute processes by themselves, but also includes programs that run on an OS in conjunction with other software or expansion board functions to execute processes. Furthermore, the program itself may be stored on a server, and part or all of the program may be downloadable from the server to a user terminal. The program may be provided to users in the form of, for example, SaaS (Software as a Service).

[0110] <Additional Notes> The above-described embodiment may be further described as follows, but is not limited to the following.

[0111] (Appendix 1) The information processing system described in Appendix 1 is an information processing system including a first imaging means provided in a vehicle, a detection means for detecting that a predetermined action has been performed toward a predetermined location toward which a subject riding in the vehicle is looking, a control means for controlling the first imaging means to capture an image of the subject when the predetermined action is detected, and an authentication means for performing authentication processing of the subject based on the image of the subject captured by the first imaging means.

[0112] (Appendix 2) The information processing system described in Appendix 2 is the information processing system described in Appendix 1, in which the detection means detects the action of the object touching the predetermined location as the predetermined action, and the control means controls the first imaging means to capture an image of the object touching the predetermined location.

[0113] (Appendix 3) The information processing system described in Appendix 3 is the information processing system described in Appendix 2, wherein the predetermined location is a mirror on which the first imaging means is provided, the detection means detects the adjustment operation of the mirror by the subject as the predetermined operation, and the control means controls the first imaging means to capture an image of the subject after a predetermined time has elapsed since the adjustment operation of the mirror was performed or at the timing when the subject releases their hand from the mirror.

[0114] (Appendix 4) The information processing system described in Appendix 4 is the information processing system described in Appendix 1, wherein the detection means detects the predetermined action as the action of the subject directing their gaze toward the predetermined location, and the control means controls the first imaging means to capture an image of the subject directing their gaze toward the predetermined location.

[0115] (Appendix 5) The information processing system described in Appendix 5 is the information processing system described in Appendix 4, further comprising a second imaging means capable of capturing an image of the subject's face, and an estimation means for estimating the direction of the subject's gaze based on the facial image of the subject captured by the second imaging means, wherein the detection means uses the estimation result of the estimation means to detect the action of the subject directing their gaze toward the predetermined location as the predetermined action, and when the control means detects the predetermined action, it controls the first imaging means to capture an iris image of the subject.

[0116] (Appendix 6) The information processing system described in Appendix 6 is the information processing system described in Appendix 5, wherein the first imaging means and the second imaging means are arranged in any one of the following arrangements: a first arrangement configuration in which both the first imaging means and the second imaging means are arranged at the predetermined location; a second arrangement configuration in which the first imaging means is arranged at the predetermined location and the second imaging means is arranged near the predetermined location; a third arrangement configuration in which the second imaging means is arranged at the predetermined location and the first imaging means is arranged near the predetermined location; or a fourth arrangement configuration in which the second imaging means is arranged at the predetermined location and a plurality of the first imaging means are arranged at positions corresponding to each of the seats of the vehicle.

[0117] (Appendix 7) The information processing system described in Appendix 7 is the information processing system described in Appendix 4, wherein the specified location is a display provided in the vehicle, and the information processing system further comprises a guidance means for displaying information on the display that guides the subject to direct their gaze toward the display.

[0118] (Appendix 8) The information processing system described in Appendix 8 is the information processing system described in any one of Appendixes 1 to 7, wherein the control means adjusts the seat position of the seat on which the subject sits so that the subject falls within the imaging range of the first imaging means, and then controls the first imaging means to capture an image of the subject.

[0119] (Appendix 9) The information processing system described in Appendix 9 is the information processing system described in Appendix 8, further comprising an identification means for identifying the target, and the control means adjusts the seat position of the seat in which the target is sitting so that the identified target is in an imaging position suitable for imaging the identified target.

[0120] (Appendix 10) The information processing system described in Appendix 10 is the information processing system described in Appendix 9, in which the control means controls the first imaging means to capture an image of the target, and then adjusts the seat position of the seat in which the identified target is sitting so that the target is in a driving position suitable for driving.

[0121] (Appendix 11) The information processing system described in Appendix 11 is the information processing system described in any one of Appendixes 8 to 10, wherein the specified location is a mirror on which the first imaging means is provided, and the control means controls the first imaging means to capture an image of the target after further adjusting the mirror in accordance with the adjustment amount of the seat position.

[0122] (Appendix 12) The information processing system described in Appendix 12 is the information processing system described in any one of Appendixes 8 to 11, wherein the control means controls the first imaging means to capture an image of the target after further adjusting the lighting in accordance with the adjustment amount of the seat position.

[0123] The information processing device described in Appendix 13 includes a detection means for detecting that a predetermined action has been performed toward a predetermined location toward which a subject riding in a vehicle is looking, a control means for controlling a first imaging means provided in the vehicle to capture an image of the subject when the predetermined action is detected, and an authentication means for performing an authentication process for the subject based on the image of the subject captured by the first imaging means.

[0124] (Appendix 14) The information processing method described in Appendix 14 is an information processing method in which at least one computer detects that a predetermined action has been performed on a predetermined location toward which a subject riding in the vehicle is looking, and when the predetermined action is detected, controls a first imaging means provided in the vehicle to capture an image of the subject, and performs authentication processing of the subject based on the image of the subject captured by the first imaging means.

[0125] (Appendix 15) The computer program described in Appendix 15 is a computer program that causes at least one computer to execute an information processing method, which detects that a predetermined action has been performed toward a predetermined location toward which a subject riding in the vehicle is looking, controls a first imaging means provided in the vehicle to capture an image of the subject when the predetermined action is detected, and performs authentication processing of the subject based on the image of the subject captured by the first imaging means.

[0126] (Appendix 16) The recording medium described in Appendix 16 is a recording medium having recorded thereon a computer program that causes at least one computer to execute an information processing method, which includes detecting that a predetermined action has been performed toward a predetermined location toward which a subject riding in the vehicle is looking, controlling a first imaging means provided in the vehicle to capture an image of the subject when the predetermined action is detected, and performing authentication processing of the subject based on the image of the subject captured by the first imaging means.

[0127] (Appendix 17) The information processing device described in Appendix 17 is an information processing device that includes a detection means for detecting that a predetermined action has been performed toward a predetermined location toward which a subject riding in the vehicle is looking, a control means for controlling a first imaging means provided in the vehicle to capture an image of the subject when the predetermined action is detected, and an authentication means for performing authentication processing of the subject based on the image of the subject captured by the first imaging means.

[0128] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing systems, information processing devices, information processing methods, and computer programs that involve such modifications are also included in the technical idea of ​​this disclosure. [Explanation of symbols]

[0129] 10 Information Processing Systems 11 processors 12 RAM 13 ROM 14 Storage device 15 Input Devices 16 Output Devices 17 Data Bus 20 Camera 51 Rearview mirror 511 Rearview camera 52 Side mirror 521 Side mirror camera 53 Rear monitor 531 Rear monitor camera 110 Camera 1 120 Predetermined operation detection unit 130 control section 1301 Seat position adjustment part 1302 Mirror adjustment part 1303 Lighting control unit 140 Authentication Department 150 Second Camera 160 Gaze estimation section 170 Guidance information output unit 180 display 190 Target Identification Department

Claims

1. a first imaging means provided in the vehicle; a detection means for detecting that a predetermined action has been performed toward a predetermined location toward which a subject riding in the vehicle is looking; a control means for controlling the first imaging means to capture an image of the target when the predetermined movement is detected; an authentication means for executing an authentication process for the target based on the image of the target captured by the first imaging means; An information processing system comprising:

2. the detection means detects, as the predetermined motion, a motion of the target touching the predetermined position; the control means controls the first imaging means to capture an image of the object that has come into contact with the predetermined location. The information processing system according to claim 1 .

3. the predetermined location is a mirror on which the first imaging means is provided, the detection means detects an adjustment operation of the mirror by the object as the predetermined operation; the control means controls the first imaging means to capture an image of the target after a predetermined time has elapsed since the mirror adjustment operation was performed or at a timing when the target releases their hand from the mirror. The information processing system according to claim 2 .

4. the detection means detects, as the predetermined action, an action of the subject directing their gaze toward the predetermined location; the control means controls the first imaging means to capture an image of the target whose line of sight is directed toward the predetermined location. The information processing system according to claim 1 .

5. a second imaging means capable of imaging the face of the subject; an estimation means for estimating a gaze direction of the subject based on a face image of the subject captured by the second imaging means; Further provided with the detection means detects, as the predetermined action, an action of the target directing their gaze toward the predetermined location, using the estimation result of the estimation means; the control means controls the first imaging means to capture an iris image of the target when the predetermined movement is detected. The information processing system according to claim 4 .

6. The first imaging means and the second imaging means are arranged in any one of the following arrangements: a first arrangement in which both the first imaging means and the second imaging means are arranged at the predetermined location; a second arrangement in which the first imaging means is arranged at the predetermined location and the second imaging means is arranged near the predetermined location; a third arrangement in which the second imaging means is arranged at the predetermined location and the first imaging means is arranged near the predetermined location; or a fourth arrangement in which the second imaging means is arranged at the predetermined location and a plurality of the first imaging means are arranged at positions corresponding to each seat of the vehicle. The information processing system according to claim 5 .

7. the predetermined location is a display provided in the vehicle, Further provided is a guidance means for displaying information on the display to guide the subject to direct their gaze toward the display. The information processing system according to claim 4 .

8. the control means adjusts a seat position of a seat on which the object is sitting so that the object falls within an imaging range of the first imaging means, and then controls the first imaging means to capture an image of the object. The information processing system according to claim 1 .

9. Further comprising an identification means for identifying the target, the control means adjusts the seat position of the seat in which the target is sitting so as to provide an imaging position suitable for imaging the identified target. The information processing system according to claim 8 .

10. the control means controls the first imaging means to capture an image of the target, and then adjusts a seat position of a seat in which the target sits so that the specified target is in a driving position suitable for driving. The information processing system according to claim 9 .

11. the predetermined location is a mirror on which the first imaging means is provided, the control means controls the first imaging means to capture an image of the target after further adjusting the mirror in accordance with the adjustment amount of the seat position. The information processing system according to any one of claims 8 to 10.

12. the control means controls the first imaging means to capture an image of the target after further adjusting the illumination in accordance with the adjustment amount of the seat position. The information processing system according to any one of claims 8 to 10.

13. a detection means for detecting that a predetermined action has been performed toward a predetermined location to which a subject riding in the vehicle is looking; a control unit that controls a first image capturing unit provided in the vehicle to capture an image of the target when the predetermined motion is detected; an authentication means for executing an authentication process for the target based on the image of the target captured by the first imaging means; An information processing device comprising:

14. by at least one computer, Detecting that a predetermined action has been performed on a predetermined location to which a subject riding in the vehicle is looking, When the predetermined movement is detected, a first imaging means provided in the vehicle is controlled to capture an image of the target; performing an authentication process for the target based on the image of the target captured by the first imaging means; Information processing methods.

15. At least one computer Detecting that a predetermined action has been performed on a predetermined location to which a subject riding in the vehicle is looking, When the predetermined movement is detected, a first imaging means provided in the vehicle is controlled to capture an image of the target; performing an authentication process for the target based on the image of the target captured by the first imaging means; A computer program that executes an information processing method.

Images