Information processing system, information processing device, information processing method, and recording medium

Abstract

An information processing system (10) comprises: an image acquisition means (110) that acquires a captured image of the interior and / or exterior of a moving body (50); a processing means (130) that executes a plurality of different processes on the basis of the image; and an information acquisition means (120) that acquires moving-body-related information indicating a status pertaining to the moving body. The processing means controls an execution mode pertaining to each of the plurality of processes on the basis of the moving-body-related information. According to such an information processing system, a plurality of processes related to a moving body can be executed appropriately.

Description

Information processing system, information processing device, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing system, an information processing device, an information processing method, and a recording medium.

[0002] Systems that execute various processes in moving objects such as vehicles are known. For example, Patent Literature 1 discloses a technology that determines whether to configure an artificial intelligence model based on the state of the vehicle and identifies the type of object, such as a vehicle, bicycle, or pedestrian, using the artificial intelligence model. Patent Literature 2 discloses a technology that recognizes objects contained in an image using a machine learning model. Patent Literature 3 discloses a technology that changes the authentication level depending on the risk of fraudulent use at the time of performing biometric authentication processing.

[0003] JP 2018-190045 A JP 2022-524920 A JP 2007-145200 A

[0004] This disclosure aims to improve upon the techniques disclosed in the prior art documents.

[0005] One aspect of the information processing system disclosed herein comprises an image acquisition means for acquiring images of at least one of the inside or outside of a moving body, a processing means for executing a plurality of different processes based on the images, and an information acquisition means for acquiring moving body-related information indicating the status of the moving body, and the processing means controls the execution mode of each of the plurality of processes based on the moving body-related information.

[0006] One aspect of the information processing device disclosed herein is provided in a moving body and comprises an image acquisition means for acquiring images of at least one of the inside or outside of the moving body, a processing means for executing a plurality of different processes based on the images, and an information acquisition means for acquiring moving body-related information indicating the status of the moving body, and the processing means controls the execution mode for each of the plurality of processes based on the moving body-related information.

[0007] One aspect of the information processing method disclosed herein is an information processing method in which at least one computer acquires images of at least one of the inside or outside of a moving body, performs a plurality of different processes based on the images, and acquires moving body-related information indicating the status of the moving body, and when performing the plurality of processes, controls the execution mode for each of the plurality of processes based on the moving body-related information.

[0008] One aspect of the recording medium of this disclosure is a recording medium having recorded thereon a computer program that causes at least one computer to execute an information processing method, which acquires images of at least one of the inside or outside of a moving body, performs a plurality of different processes based on the images, and acquires moving body-related information indicating the status of the moving body, and when executing the plurality of processes, controls the execution mode for each of the plurality of processes based on the moving body-related information.

[0009] 1 is a block diagram showing a hardware configuration of an information processing system according to a first embodiment. 2 is a block diagram showing a functional configuration of the information processing system according to the first embodiment. 3 is a flowchart showing the operation flow of the information processing system according to the second embodiment. 4 is a side view showing the configuration of a vehicle to which an information processing system according to a second embodiment is applied. 5 is a flowchart showing the operation flow of the information processing system according to the second embodiment. 6 is a block diagram showing the functional configuration of an information processing system according to a third embodiment. 7 is a flowchart showing the operation flow of the information processing system according to the third embodiment. 8 is a flowchart showing the operation flow of the information processing system according to the fourth embodiment. 9 is a flowchart showing the operation flow of the information processing system according to the fifth embodiment. 10 is a flowchart showing the operation flow of the information processing system according to the sixth embodiment. 11 is a flowchart showing the operation flow of the information processing system according to the seventh embodiment. 12 is a flowchart showing the operation flow of the information processing system according to the eighth embodiment. 13 is a flowchart showing the operation flow of the information processing system according to the ninth embodiment. 14 is a flowchart showing the operation flow of the information processing system according to the tenth embodiment. 15 is a flowchart showing the operation flow of the information processing system according to the eleventh embodiment. 16 is a flowchart showing the operation flow of the information processing system according to the twelfth embodiment. 17 is a flowchart showing the operation flow of the information processing system according to the thirteenth embodiment. 18 is a block diagram showing the configuration of a vehicle in which an information processing device according to a fourteenth embodiment is installed.

[0010] Hereinafter, embodiments of an information processing system, an information processing device, an information processing method, and a recording medium will be described with reference to the drawings.

[0011] First Embodiment An information processing system according to a first embodiment will be described with reference to FIGS. 1 to 3. FIG.

[0012] (Hardware Configuration) First, the hardware configuration of the information processing system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the information processing system according to the first embodiment.

[0013] 1 , an information processing system 10 according to the first embodiment includes a processor 11, a RAM (Random Access Memory) 12, and a ROM (Read Only Memory) 13. The information processing system 10 may further include a storage device 14, an input device 15, and an output device 16. The processor 11, RAM 12, ROM 13, storage device 14, input device 15, and output device 16 are connected via a data bus 17.

[0014] The processor 11 loads a computer program. For example, the processor 11 is configured to load a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may load a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may acquire (i.e., load) the computer program from a device (not shown) located outside the information processing system 10 via a network interface. The processor 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16 by executing the loaded computer program. In particular, in this embodiment, when the processor 11 executes the loaded computer program, functional blocks for executing multiple processes based on images captured inside or outside the moving object are realized within the processor 11. In other words, the processor 11 may function as a controller that executes each control in the information processing system 10.

[0015] The processor 11 may be configured as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or a quantum processor. The processor 11 may be configured as one of these, or may be configured to use multiple processors in parallel.

[0016] The RAM 12 temporarily stores computer programs executed by the processor 11. The RAM 12 temporarily stores data that the processor 11 temporarily uses while it is executing the computer programs. The RAM 12 may be, for example, a dynamic random access memory (D-RAM) or a static random access memory (SRAM). Alternatively, other types of volatile memory may be used instead of the RAM 12.

[0017] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store fixed data. The ROM 13 may be, for example, a programmable read-only memory (PROM) or an erasable read-only memory (EPROM). Alternatively, other types of non-volatile memory may be used instead of the ROM 13.

[0018] The storage device 14 stores data that is to be saved long-term by the information processing system 10. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0019] The input device 15 is a device that receives input instructions from a user of the information processing system 10. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may be configured as a mobile terminal such as a smartphone or a tablet. The input device 15 may also be, for example, a device that includes a microphone and is capable of voice input.

[0020] The output device 16 is a device that outputs information related to the information processing system 10 to the outside. For example, the output device 16 may be a display device (e.g., a display) that can display information related to the information processing system 10. The output device 16 may be configured as a mobile terminal such as a smartphone or a tablet. The output device 16 may also be a device that outputs information in a format other than an image. For example, the output device 16 may be a speaker that outputs information related to the information processing system 10 as audio.

[0021] 1 may be provided as an external device of the information processing system 10. For example, the information processing system 10 may be configured to include only the processor 11, RAM 12, and ROM 13 described above, and the other components (i.e., the storage device 14, the input device 15, and the output device 16) may be provided in an external device connected to the information processing system 10. Furthermore, some of the calculation functions of the information processing system 10 may be realized by an external device (e.g., an external server, a cloud, etc.).

[0022] (Functional Configuration) Next, the functional configuration of the information processing system 10 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the information processing system according to the first embodiment.

[0023] 2, the information processing system 10 according to the first embodiment is configured to include, as components for realizing its functions, an image acquisition unit 110, an information acquisition unit 120, and a processing unit 130. Each of the image acquisition unit 110, the information acquisition unit 120, and the processing unit 130 may be a processing block realized by, for example, the above-mentioned processor 11 (see FIG. 1).

[0024] The image acquisition unit 110 is configured to acquire images of at least one of the interior and exterior of a mobile object. The mobile object may include, for example, automobiles, as well as various vehicles such as bicycles, motorcycles, buses, and trains. Alternatively, the mobile object may include ships, airplanes, and other vehicles that are not generally classified as vehicles. The image acquisition unit 110 may be configured to acquire images of the interior or exterior of a vehicle using, for example, a camera installed in the mobile object. Specifically, the image acquisition unit 110 may acquire images of an object aboard the mobile object or the surrounding area from a camera installed inside the mobile object. Alternatively, the image acquisition unit 110 may acquire images of the surrounding area of ​​the mobile object from a camera installed outside the mobile object. Furthermore, if multiple cameras are installed on the mobile object, the image acquisition unit 110 may be configured to acquire images captured by each of the multiple cameras. The image acquisition unit 110 may acquire multiple consecutive images (i.e., video) in chronological order. The images acquired by the image acquisition unit 110 are output to the processing unit 130.

[0025] The information acquisition unit 120 is configured to acquire mobile object-related information. The mobile object-related information is information indicating the status of the mobile object, and may be information indicating the status of the mobile object itself, or information indicating the status of a passenger in the mobile object or the status of the mobile object's surrounding environment. The mobile object-related information may be, for example, information indicating the location of the mobile object, its running state, a stopped time, or the time the passenger left the mobile object. The mobile object-related information may be information regarding the passenger's gender, age, height, weight, clothing, body type, movement, behavior pattern, etc. The mobile object-related information may be information regarding the brightness around the mobile object, the presence or absence of obstacles, the amount of pedestrians, etc. The information acquisition unit 120 may acquire the mobile object-related information using various sensors installed in the mobile object, in-vehicle communication (e.g., CAN standard), GPS (Global Positioning System), beacons, the Internet, Bluetooth, etc. The information acquisition unit 120 may also acquire the mobile object information from input by the passenger in the mobile object (e.g., button operation, etc.). The mobile object related information acquired by the information acquisition unit 120 is output to the processing unit 130 .

[0026] The processing unit 130 is configured to be able to execute a plurality of different processes based on the images acquired by the image acquisition unit 110. The processing unit 130 may be configured to execute a plurality of processes sequentially, or may be configured to execute a plurality of processes simultaneously in parallel. The processes executed by the processing unit 130 are not particularly limited, and may be, for example, processes related to the use of a mobile object. The processing unit 130 may be configured to execute, for example, a process for unlocking the doors of a mobile object, a process for assisting the driving of a mobile object, etc. More specific examples of the processes executed by the processing unit 130 will be described in other embodiments below.

[0027] The processing unit 130 is further configured to control the execution mode of processes based on the mobile-object-related information acquired by the information acquisition unit 120. For example, the processing unit 130 may execute multiple processes in an optimal mode depending on the mobile-object-related information. Specifically, the processing unit 130 may change the type and number of processes to be executed based on the mobile-object-related information. Alternatively, the processing unit 130 may change the content and order of processes to be executed based on the mobile-object-related information. Alternatively, the processing unit 130 may change whether or not a process is executable based on the mobile-object-related information. Alternatively, the processing unit 130 may change various thresholds used in a process based on the mobile-object-related information. Alternatively, the processing unit 130 may change the priority of a process based on the mobile-object-related information. Alternatively, the processing unit 130 may change parameters of devices used in a process. Note that the processing unit 130 may control the execution mode of all or some of the multiple processes to be executed. Examples of control of the execution mode by the processing unit 130 will be described in more detail in other embodiments described below.

[0028] (Operation Flow) Next, the operation flow of the information processing system 10 according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the operation flow of the information processing system according to the first embodiment.

[0029] 3, when the information processing system 10 according to the first embodiment starts operation, the image acquisition unit 110 first acquires an image of at least one of the interior and exterior of the moving object (step S101). The image acquisition unit 110 outputs the acquired image to the processing unit 130.

[0030] Next, the information acquisition unit 120 acquires mobile-object-related information (step S102). The process of step S102 may be executed simultaneously with the process of step S101, or may be executed one after the other. The information acquisition unit 120 outputs the acquired mobile-object-related information to the processing unit 130.

[0031] Next, the processing unit 130 executes a plurality of processes based on the image acquired by the image acquisition unit 110. At this time, the processing unit 130 controls the execution mode of each process based on the moving object-related information acquired by the information acquisition unit 120 (step S103).

[0032] The above-described series of processes may be repeatedly executed. That is, the image acquisition by the image acquisition unit 110 and the acquisition of moving object-related information by the information acquisition unit 120 may be executed multiple times in succession, and each time, the processing unit 130 may execute multiple processes while controlling the execution mode.

[0033] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the first embodiment will be described.

[0034] 1 to 3, in the information processing system 10 according to the first embodiment, a plurality of processes are executed based on images captured inside or outside a moving object. The execution modes of the plurality of processes are controlled based on moving object-related information. In this way, the plurality of processes based on the captured images can be executed in an appropriate execution mode depending on the situation.

[0035] Second Embodiment An information processing system 10 according to a second embodiment will be described with reference to Figures 4 and 5. Note that the second embodiment differs from the first embodiment described above only in some configurations and operations, and other parts may be the same as the first embodiment. Therefore, the following will describe in detail the parts that differ from the first embodiment already described, and will omit explanations of other overlapping parts as appropriate.

[0036] (Vehicle-Mounted Camera) First, a vehicle equipped with an information processing system 10 according to the second embodiment will be described with reference to Fig. 4. Fig. 4 is a side view showing the configuration of a vehicle to which the information processing system according to the second embodiment is applied.

[0037] 4, the information processing system 10 according to the second embodiment is configured to be mounted on a vehicle 50. All of the components of the information processing system 10 may be mounted on the vehicle 50. Alternatively, the information processing system 10 may have some components mounted on the vehicle 50 and other components located outside the vehicle. In this case, the information processing system 10 may be configured to execute various processes while communicating with the outside of the vehicle.

[0038] A vehicle 50 equipped with the information processing system 10 according to the second embodiment is provided with a first camera 210 and a second camera 220. The first camera 210 and the second camera 220 are cameras that capture images acquired by the image acquisition unit 110.

[0039] The first camera 210 is installed on top of the vehicle 50 and captures images of the area around the vehicle 50. The first camera 210 may be, for example, a camera for detecting objects present in the area around the vehicle 50. In this case, the processing unit 130 may be configured to execute processing for detecting people, obstacles, etc. present in the area around the vehicle 50 from images captured by the first camera 210. Meanwhile, the second camera 220 is installed on each door of the vehicle 50 and captures images of the faces of targets approaching each door. The second camera 220 may be, for example, a camera for executing authentication processing of targets using the vehicle 50 (e.g., a driver or passenger). In this case, the processing unit 130 may be configured to execute processing for detecting the faces of targets from images captured by the second camera 220 and to execute facial authentication processing using the detected faces.

[0040] Although not shown in the drawings, a camera may be provided inside the vehicle 50. In this case, the image acquisition unit 110 may be configured to acquire images captured by a camera provided inside the vehicle 50. The camera provided inside the vehicle 50 may be configured to capture, for example, an image of a driver sitting in the driver's seat of the vehicle 50 or an image of a passenger sitting in the passenger seat or rear seat of the vehicle 50. Alternatively, the camera provided inside the vehicle 50 may be configured to capture an image of luggage or the like loaded on the vehicle 50.

[0041] (Operation Flow) Next, the operation flow of the information processing system 10 according to the second embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the operation flow of the information processing system according to the second embodiment. Note that in Fig. 5, the same processes as those described in Fig. 3 are denoted by the same reference numerals.

[0042] 5, when the operation of the information processing system 10 according to the second embodiment is started, the image acquisition unit 110 first acquires an image of at least one of the interior and exterior of the moving object (step S101), and the information acquisition unit 120 acquires moving object-related information (step S102).

[0043] Next, the processing unit 130 executes a plurality of processes based on the images acquired by the image acquisition unit 110 (i.e., the images captured by the first camera 210 and the second camera 220). At this time, the processing unit 130 controls the execution mode of the plurality of processes by controlling the camera parameters based on the moving object-related information acquired by the information acquisition unit 120 (step S201). For example, the processing unit 130 controls the camera parameters (e.g., exposure, gain, etc.) of the first camera 210 and the second camera 220 so that images suitable for executing the plurality of processes are captured.

[0044] Alternatively, the processing unit 130 may control the imaging ranges of the first camera 210 and the second camera 220 based on the moving object-related information so that a subject suitable for the processing to be executed is captured. For example, the processing unit 130 may control the orientation of the lenses of the first camera 210 and the second camera 220 or the zoom magnification. Alternatively, the processing unit 130 may control the on / off of the first camera 210 and the second camera 220 so that an image suitable for the processing to be executed is captured. For example, the processing unit 130 may perform imaging by the first camera 210 while stopping imaging by the second camera 220. Furthermore, the processing unit 130 may perform imaging by the second camera 220 while stopping imaging by the first camera 210.

[0045] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the second embodiment will be described.

[0046] 4 and 5, in the information processing system 10 according to the second embodiment, the camera parameters for capturing images are controlled based on the moving object-related information. In this way, images suitable for multiple processes can be captured, and therefore, multiple processes can be executed in a more appropriate manner.

[0047] In the following embodiments, an example in which the information processing system 10 is mounted on a vehicle 50 will be described, as described in the second embodiment.

[0048] Third Embodiment An information processing system 10 according to a third embodiment will be described with reference to Figures 6 to 8. Note that the third embodiment differs only in part of the configuration and operation from the first and second embodiments described above, and other parts may be the same as the first and second embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit a description of other overlapping parts as appropriate.

[0049] (Functional Configuration) First, the functional configuration of the information processing system 10 according to the third embodiment will be described with reference to Fig. 6. Fig. 6 is a block diagram showing the functional configuration of the information processing system according to the third embodiment. Note that in Fig. 6, the same elements as those described in Fig. 2 are denoted by the same reference numerals.

[0050] 6, the information processing system 10 according to the third embodiment is configured to include, as components for realizing its functions, an image acquisition unit 110, an information acquisition unit 120, and a processing unit 130. In particular, the processing unit 130 according to the third embodiment includes an object detection unit 131, a face detection unit 132, a face authentication unit 133, and an impersonation determination unit 134.

[0051] The object detection unit 131 is configured to be able to detect an object from an image acquired by the image acquisition unit 110. Since existing technologies can be appropriately adopted as a specific method for detecting an object from an image, detailed description thereof will be omitted here. The object detection unit 131 may be configured to detect only moving objects (e.g., humans, animals, etc.) and not to detect stationary objects. The object detection unit 110 may be configured to detect objects using AI (artificial intelligence). More specifically, the object detection unit 131 may be configured to detect objects using an object detection model including a trained neural network. Hereinafter, the processing performed by the object face detection unit 131 may be referred to as "object detection processing."

[0052] The face detection unit 132 is configured to be able to detect the face of a target (i.e., a target of face recognition, which will be described later) from an image acquired by the image acquisition unit 110. Note that a specific method for detecting a face from an image can be appropriately adopted from existing technologies, and therefore a detailed description thereof will be omitted here. The face detection unit 132 may be configured to be able to detect the face of a human or animal detected by the object detection unit 110. In this case, the face detection unit 132 may perform face detection only when an object is detected by the object detection unit 131. The face detection unit 132 may be configured to detect a face using AI. More specifically, the face detection unit 132 may be configured to detect the face of a target using a face detection model including a trained neural network. Hereinafter, the processing performed by the face detection unit 132 may be referred to as "face detection processing."

[0053] The face authentication unit 133 is configured to perform face authentication using an image acquired by the image acquisition unit 110. The face authentication unit 133 may perform face authentication by comparing a pre-registered face image with the face of a target included in the acquired image. The face authentication unit 133 may perform face authentication based on, for example, facial features extracted from the face image. The face authentication unit 133 may perform face authentication on a face detected by the face detection unit 132. For example, after the face detection unit 132 detects a face from an image, the face authentication unit 133 may perform face authentication using information about the area in the image where the face is detected (face area). The face authentication unit 133 may be configured to perform face authentication using AI. More specifically, the face authentication 133 may be configured to perform face authentication using a face authentication model including a trained neural network. Hereinafter, the processing performed by the face authentication unit 133 may be referred to as "face authentication processing."

[0054] The spoofing determination unit 134 is configured to be able to determine whether or not a target is spoofing using the image acquired by the image acquisition unit 110. Here, "spoofing" refers to the act of another person impersonating the person using, for example, a photograph, video, a 3D mask, or the like. The spoofing determination unit 134 may determine whether or not a target is spoofing by determining whether or not the target is a living organism. For example, the spoofing determination unit 134 may request the target to perform a predetermined action (e.g., moving one's eyes or one's hands) and determine whether or not the target performs the action in accordance with the request. Hereinafter, the processing executed by the spoofing determination unit 134 may be referred to as "spoofing determination processing."

[0055] The processing unit 130 according to the third embodiment is configured to control the priority of each process executed by the object detection unit 131, face detection unit 132, face authentication unit 133, and spoofing determination unit 134. That is, the processing unit 130 is configured to determine the priority and execution level of each of the object detection process, face detection process, face authentication process, and spoofing determination process. The processing unit 130 may control the priority by, for example, changing the allocation of the processor 11 and memory (i.e., RAM 12) used by the object detection unit 131, face detection unit 132, face authentication unit 133, and spoofing determination unit 134. Alternatively, the processing unit 130 may control the priority by executing only some of the object detection process, face detection process, face authentication process, and spoofing determination process. That is, the processing unit 130 may execute only high-priority processes and not execute low-priority processes.

[0056] Note that the processing unit 130 according to the third embodiment does not need to include all of the above-described object detection unit 131, face detection unit 132, face authentication unit 133, and spoofing determination unit 134, but may be configured to include at least two of them. For example, the processing unit 130 may be configured to include two units, the object detection unit 131 and the face detection unit 132. In this case, the processing unit 130 only needs to determine which process, for each of the object detection process and the face detection process, should be prioritized and to what extent. Alternatively, the processing unit 130 may be configured to include two units, the face authentication unit 133 and the spoofing determination unit 134. In this case, the processing unit 130 only needs to determine which process, for each of the face authentication process and the spoofing determination process, should be prioritized and to what extent.

[0057] (Authentication Operation) Next, the flow of the authentication operation (i.e., the operation performed by the object detection unit 131, the face detection unit 132, the face authentication unit 133, and the spoofing determination unit 134) performed by the information processing system 10 according to the third embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the flow of the authentication operation performed in the information processing system according to the third embodiment.

[0058] 7 , when the authentication operation in the information processing system 10 according to the third embodiment is started, the object detection unit 131 first detects an object from the image acquired by the image acquisition unit 110 (step S31). If no object is detected, the subsequent processing may be omitted. If multiple objects are detected, the subsequent processing may be performed for each of the multiple objects.

[0059] Next, the face detection unit 132 detects the face of the target from the image acquired by the image acquisition unit 110 (step S32). Then, the face authentication unit 133 performs face authentication using the face detected by the face detection unit 132 (step S33). After performing face authentication, the face authentication unit 133 may output the authentication result. For example, the face authentication unit 133 may output information indicating that face authentication was successful, information indicating that face authentication was unsuccessful, or information indicating that face authentication could not be performed normally.

[0060] The face authentication unit 133 may further have a function of executing a predetermined process based on the authentication result. For example, the face authentication unit 133 may execute a process of unlocking the doors of the vehicle 50 when the face authentication is successful. Furthermore, the face authentication unit 133 may execute a process of locking the doors of the vehicle 50 or a process of outputting alert information when the face authentication is unsuccessful. Note that the processes described here are merely examples, and various other processes may be executed based on the authentication result of the face authentication unit 130.

[0061] Next, the spoofing determination unit 134 performs spoofing determination on the target based on the image acquired by the image acquisition unit 110 (step S34). After performing the spoofing determination process, the spoofing determination unit 134 may output the determination result. For example, the spoofing determination unit 134 may output information indicating that the spoofing determination process was successful (the target was not spoofed), information indicating that the spoofing determination process failed (the target was spoofed), or information indicating that the spoofing determination process could not be performed normally.

[0062] The determination result by the spoofing determination unit 134 may be used as information for confirming the identity of the person, together with the result of face authentication by the face authentication unit 133 described above. For example, even if the face authentication process is successful, if the spoofing determination process determines that "impersonation has occurred," the person may not be recognized as the person. Furthermore, the determination result by the spoofing determination unit 134 may be used as a condition for executing control other than the face authentication process. For example, if the face authentication process by the face authentication unit 133 is successful, the doors of the vehicle 50 may be unlocked, and if the spoofing determination unit 134 determines that "impersonation has not occurred," the engine of the vehicle 50 may be started.

[0063] Hereinafter, some or all of the processes described in FIG. 7 (i.e., the object detection process, face detection process, face authentication process, and spoofing determination process) may be referred to as "processes related to authentication operations."

[0064] (Operation Flow) Next, the operation flow of the information processing system 10 according to the third embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the operation flow of the information processing system according to the third embodiment. Note that in Fig. 8, the same processes as those described in Fig. 3 are denoted by the same reference numerals.

[0065] 8, when the operation of the information processing system 10 according to the third embodiment is started, the image acquisition unit 110 first acquires an image of at least one of the interior and exterior of the moving object (step S101), and the information acquisition unit 120 acquires moving object-related information (step S102).

[0066] Next, the processing unit 130 sets priorities of the multiple processes to be executed based on the moving object-related information acquired by the information acquisition unit 120 (step S301). That is, the processing unit 130 sets priorities for the object detection process, face detection process, face authentication process, and impersonation determination process.

[0067] Next, the processing unit 130 executes a plurality of processes based on the image acquired by the image acquisition unit 110. At this time, the processing unit 130 executes each process based on the priority set in step S301 (step S302). As a result, the execution manner of the plurality of processes executed by the processing unit 130 is controlled based on the moving object-related information.

[0068] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the third embodiment will be described.

[0069] 6 to 8, in the information processing system 10 according to the third embodiment, the priority of the object detection process, face detection process, face authentication process, and impersonation determination process is changed based on the moving object-related information. In this way, the priority of each process can be changed depending on the situation, and the process related to the authentication operation can be executed in a more appropriate manner.

[0070] Fourth Embodiment An information processing system 10 according to a fourth embodiment will be described with reference to Fig. 9. Note that the fourth embodiment differs from the first to third embodiments described above only in some of its operations, and other parts may be the same as the first to third embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit a description of other overlapping parts as appropriate.

[0071] (Operation Flow) First, the operation flow of the information processing system 10 according to the fourth embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the operation flow of the information processing system according to the fourth embodiment. Note that in Fig. 9, the same processes as those described in Fig. 2 are denoted by the same reference numerals.

[0072] As shown in FIG. 9, when the operation of the information processing system 10 according to the fourth embodiment is started, the image acquisition unit 110 first acquires an image of at least one of the inside and outside of a moving object (step S101).

[0073] Next, the information acquisition unit 120 acquires the moving object-related information. At this time, the information acquisition unit 120 acquires the stopping time of the vehicle 50 (i.e., the elapsed time since the vehicle 50 stopped) as the moving object-related information (step S401). Note that the moving object-related information may include information other than the stopping time of the vehicle 50.

[0074] Next, the processing unit 130 determines whether the stop time of the vehicle 50 is equal to or longer than a first predetermined time based on the acquired moving object-related information (step S402). The "first predetermined time" here is a threshold value for determining whether the stop time of the vehicle 50 is sufficiently long, and may be set to a value that allows, for example, determining that the target has left the vehicle 50 and moved to another location.

[0075] If the stop time of the vehicle 50 is equal to or longer than the first predetermined time (step S402: YES), the processing unit 130 sets the priority of the object detection process higher than the priorities of the face detection process, the face authentication process, and the impersonation determination process (step S403). Note that the processing unit 130 may increase the priority of the object detection process, or may relatively increase the priority of the object detection process by lowering the priorities of processes other than the object detection process.

[0076] On the other hand, if the stopping time of the vehicle 50 is not equal to or longer than the first predetermined time (step S402: NO), the priority of each process is set to a normal value (for example, a preset initial value) (step S404). That is, the priority of the object detection process is not set to a high value as in step S403.

[0077] Next, the processing unit 130 executes a plurality of processes according to the set priorities (step S405). That is, if the priority of the object detection process is set high in step S403, the processing unit 130 executes the object detection process with priority over other processes, whereas if the normal priority is set in step S404, the processing unit 130 executes each process without prioritizing the object detection process.

[0078] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the fourth embodiment will be described.

[0079] As described with reference to FIG. 9 , in the information processing system 10 according to the fourth embodiment, the priority of the object detection process is changed depending on the time the vehicle 50 is stopped. Specifically, if the vehicle 50 is stopped for a long time, the priority of the object detection process is set high. In this way, objects around the vehicle 50 can be efficiently detected when the vehicle 50 is stopped for a long time. For example, in a situation where an object that has left the vehicle 50 returns to the vehicle 50, the returning object can be properly detected as an object. As a result, the object can be detected without being overlooked, and processing after the object is detected (i.e., face detection process, face authentication process, and impersonation determination process) can be properly transitioned to. Note that after the object is detected in the object detection process, the priority of the object detection process, which had been set high until then, may be lowered (i.e., returned to the normal priority).

[0080] Fifth Embodiment An information processing system 10 according to a fifth embodiment will be described with reference to Fig. 10. Note that the fifth embodiment differs from the first to fourth embodiments described above only in some of its operations, and other parts may be the same as the first to fourth embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit a description of other overlapping parts as appropriate.

[0081] (Operation Flow) First, the operation flow of the information processing system 10 according to the fifth embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the operation flow of the information processing system according to the fifth embodiment. In Fig. 10, the same processes as those described in Fig. 3 are denoted by the same reference numerals. In the fifth embodiment, the face authentication process and the impersonation determination process are executed as processes for unlocking the doors of the vehicle 50.

[0082] As shown in FIG. 10, when the operation of the information processing system 10 according to the fifth embodiment is started, the image acquisition unit 110 first acquires an image of at least one of the inside and outside of a moving object (step S101).

[0083] Next, the information acquisition unit 120 acquires moving object-related information. At this time, the information acquisition unit 120 acquires the time when the target left the vehicle 50 (i.e., the time elapsed since the target got off the vehicle) as the moving object-related information (step S501). Note that the moving object-related information may include information other than the time when the target left the vehicle 50.

[0084] Next, the processing unit 130 determines whether the time the target left is within a second predetermined time based on the acquired mobile-object-related information (step S502). The "second predetermined time" here is a threshold value for determining whether the time the target left is very short, and may be set to a value that determines, for example, that the target who got off the vehicle immediately turned around and returned to the vehicle 50 (for example, realized that they had left something behind and returned immediately).

[0085] If the time the target has left is within the second predetermined time (step S502: YES), the processing unit 130 unlocks the doors on the condition that the facial recognition process is successful (step S503). That is, the processing unit 130 unlocks the doors of the vehicle 50 if the facial recognition process can confirm that the target is a passenger of the vehicle 50.

[0086] On the other hand, if the time during which the target has left is not within the second predetermined time (step S502: NO), the processing unit 130 unlocks the doors on the condition that both the face authentication process and the impersonation determination process are successful (step S504). That is, the processing unit 130 unlocks the doors of the vehicle 50 when the face authentication process can confirm that the target is a passenger of the vehicle 50 and the impersonation determination can confirm that the target is not impersonating someone else.

[0087] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the fifth embodiment will be described.

[0088] As described in Fig. 10 , in the information processing system 10 according to the fifth embodiment, the door unlocking conditions (i.e., the conditions imposed on personal authentication) are changed depending on the time the subject has been away from the vehicle 50. In this way, when the subject leaves the vehicle 50 and returns soon after, the door unlocking process is executed under relatively loose conditions, which reduces the burden on the subject and the processing load. Also, when the subject leaves the vehicle 50 and returns after a while, the door unlocking process is executed under relatively strict conditions, which prevents the door from being unlocked by a stranger.

[0089] Sixth Embodiment An information processing system 10 according to a sixth embodiment will be described with reference to Fig. 11 . Note that the sixth embodiment differs only in some operations from the first to fifth embodiments described above, and other parts may be the same as the first to fifth embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit a description of other overlapping parts as appropriate.

[0090] (Operation Flow) First, the operation flow of the information processing system 10 according to the sixth embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the operation flow of the information processing system according to the sixth embodiment. Note that in Fig. 11, the same processes as those described in Fig. 3 are denoted by the same reference numerals.

[0091] As shown in FIG. 11, when the operation of the information processing system 10 according to the sixth embodiment is started, the image acquisition unit 110 first acquires an image of at least one of the inside and outside of a moving object (step S101).

[0092] Next, the information acquisition unit 120 acquires moving object-related information. At this time, the information acquisition unit 120 acquires information indicating whether or not the subject is driving the vehicle 50 as the moving object-related information (step S601). Note that the moving object-related information may include information other than information indicating whether or not the subject is driving the vehicle 50.

[0093] Next, the processing unit 130 determines whether the subject is driving the vehicle 50 based on the acquired moving object-related information (step S602). If the vehicle 50 is being driven (step 602: YES), the processing unit 130 executes an object detection process to detect an object that may affect driving (e.g., an obstacle), and executes a face authentication process to authenticate the face of the driver (step S603).

[0094] On the other hand, if the vehicle 50 is not in operation (step S602: NO), the processing unit 130 executes a plurality of processes as usual (step S603). For example, the processing unit 130 may execute a process of unlocking the doors based on the results of the face authentication process or the spoofing determination process, as described in the fifth embodiment.

[0095] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the sixth embodiment will be described.

[0096] As described in FIG. 11 , in the information processing system 10 according to the sixth embodiment, when a subject is driving a vehicle 50, a process for detecting an object that affects driving and a process for authenticating the face of the driver are executed. This makes it possible to appropriately execute a process for assisting the subject in driving and a process that is executed after authenticating the subject while driving. Note that, when the vehicle 50 is being driven, it is unlikely that a situation will arise in which a process for recognizing an object outside the vehicle 50 and unlocking the doors, as described in the fifth embodiment, is required. Therefore, while driving, it is possible to more appropriately execute multiple processes by increasing the priority of processes related to driving the vehicle 50 and the interior of the vehicle 50 and lowering the priority of unnecessary processes, such as a process for unlocking the doors.

[0097] Seventh Embodiment An information processing system 10 according to a seventh embodiment will be described with reference to Fig. 12. Note that the seventh embodiment differs from the first to sixth embodiments described above only in some of its operations, and other parts may be the same as the first to sixth embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit a description of other overlapping parts as appropriate.

[0098] (Operation Flow) First, the operation flow of the information processing system 10 according to the seventh embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart showing the operation flow of the information processing system according to the seventh embodiment. Note that in Fig. 12, the same processes as those described in Fig. 3 are denoted by the same reference numerals.

[0099] As shown in FIG. 12, when the operation of the information processing system 10 according to the seventh embodiment is started, the image acquisition unit 110 first acquires an image of at least one of the inside and outside of a moving object (step S101).

[0100] Next, the information acquisition unit 120 acquires mobile object-related information. At this time, the information acquisition unit 120 acquires the location where the vehicle 50 is stopped as the mobile object-related information (step S701). The location where the vehicle 50 is stopped may include latitude and longitude information, map information, information about surrounding facilities and the environment, etc. Note that the mobile object-related information may also include information other than the location where the vehicle 50 is stopped.

[0101] Next, the processing unit 130 determines, based on the acquired mobile object-related information, whether the location where the vehicle 50 is parked is a location where it is predicted that the subject will return to the vehicle 50 with luggage (hereinafter referred to as a "baggage loading predicted location") (step S702). An example of a luggage loading predicted location is a store parking lot. When the vehicle 50 is parked in a store parking lot, it is predicted that the subject who has shopped at the store is likely to return to the vehicle 50 with luggage.

[0102] If the location where the vehicle 50 is parked is a luggage loading predicted location (step 702: YES), the processing unit 130 unlocks the driver's seat door lock and the trunk door lock, on the condition that the facial authentication of the subject is successful (step S703). That is, the processing unit 130 unlocks the driver's seat door lock where the subject will get in, as well as the trunk door lock where the subject will load luggage. Note that if it is predicted that the subject will load luggage in the rear seat, the processing unit 130 may also unlock the rear seat door lock of the vehicle 50. Where the subject will load luggage may be predicted, for example, from the subject's past behavioral patterns, etc.

[0103] On the other hand, if the location where the vehicle 50 is parked is not a predicted luggage loading location (step S702: NO), the processing unit 130 unlocks only the driver's door on the condition that the facial authentication of the subject is successful (step S703). In other words, the processing unit 130 determines that the subject will not return with luggage, and does not unlock the doors of the trunk or other areas where luggage is loaded.

[0104] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the seventh embodiment will be described.

[0105] 12 , in the information processing system 10 according to the seventh embodiment, when the vehicle 50 is parked at a location where the subject is expected to return with luggage, the door lock corresponding to the location where the luggage will be placed is unlocked. In this way, the luggage brought by the subject can be easily loaded into the vehicle 50. This effect is more pronounced in a situation where, for example, the subject's hands are full of luggage and it is difficult to operate the device with their hands.

[0106] Eighth Embodiment An information processing system 10 according to an eighth embodiment will be described with reference to Fig. 13. Note that the eighth embodiment differs only in some of the operations from the first to seventh embodiments described above, and other parts may be the same as the first to seventh embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit a description of other overlapping parts as appropriate.

[0107] (Operation Flow) First, the operation flow of the information processing system 10 according to the eighth embodiment will be described with reference to Fig. 13. Fig. 13 is a flowchart showing the operation flow of the information processing system according to the eighth embodiment. Note that in Fig. 13, the same processes as those described in Fig. 3 are denoted by the same reference numerals.

[0108] As shown in FIG. 13, when the operation of the information processing system 10 according to the eighth embodiment is started, the image acquisition unit 110 first acquires an image of at least one of the inside and outside of the moving body (step S101).

[0109] Next, the information acquisition unit 120 acquires mobile object-related information. At this time, the information acquisition unit 120 acquires the location where the vehicle 50 is stopped as the mobile object-related information (step S801). The location where the vehicle 50 is stopped may include latitude and longitude information, map information, information about surrounding facilities and the environment, etc. Note that the mobile object-related information may include information other than the location where the vehicle 50 is stopped.

[0110] Next, the processing unit 130 determines whether the location where the vehicle 50 is stopped is occupied by a passenger of the vehicle 50 (in other words, a passenger other than the driver) based on the acquired moving object-related information (step S802). Note that information about the passenger of the vehicle 50 and information about the location used by the passenger may be registered in advance. For example, the owner of the vehicle 50 may register his / her "child" as a passenger. In this case, "school" may be registered as the location used by the passenger.

[0111] If the location where the vehicle 50 is parked is a location used by a passenger (step S802: YES), the processing unit 130 increases the priority of the process related to the authentication operation for seats other than the driver's seat (e.g., the passenger seat and the rear seats) (step S803). That is, the processing unit 130 increases the priority of the process related to the authentication operation for seats used by a passenger.

[0112] If the location where the vehicle 50 is parked is not a location used by a passenger (step 802: NO), the processing unit 130 omits the process of step S803. In this case, the priority of the process related to the authentication operation for seats other than the driver's seat is not increased.

[0113] Thereafter, the processing unit 130 executes a process related to the authentication operation in the set priority order, and unlocks the door lock of the corresponding door if the authentication is successful (step S804). For example, the processing unit 130 may execute a process related to the authentication operation by turning off the second camera 220 (see FIG. 4) installed on the driver's door and turning on the second camera 220 installed on the rear door (see FIG. 4).

[0114] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the eighth embodiment will be described.

[0115] 13 , in the information processing system 10 according to the eighth embodiment, when the vehicle 50 is parked in a location where a passenger will be using the vehicle, the priority of the process for unlocking the doors of seats other than the driver's seat is changed to be higher. In this way, it is possible to appropriately perform the process related to the authentication operation for the passenger of the vehicle 50 and unlock the door of the seat where the passenger will be sitting.

[0116] Ninth Embodiment An information processing system 10 according to a ninth embodiment will be described with reference to Fig. 14. Note that the ninth embodiment differs from the first to eighth embodiments described above only in some of its operations, and other parts may be the same as the first to eighth embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit a description of other overlapping parts as appropriate.

[0117] (Operation Flow) First, the operation flow of the information processing system 10 according to the ninth embodiment will be described with reference to Fig. 14. Fig. 14 is a flowchart showing the operation flow of the information processing system according to the ninth embodiment. Note that in Fig. 14, the same processes as those described in Fig. 3 are denoted by the same reference numerals.

[0118] As shown in FIG. 14, when the operation of the information processing system 10 according to the ninth embodiment is started, the image acquisition unit 110 first acquires an image of at least one of the inside and outside of the moving body (step S101).

[0119] Next, the information acquisition unit 120 acquires mobile object-related information. At this time, the information acquisition unit 120 acquires information regarding pedestrian traffic around the location where the vehicle 50 is parked as the mobile object-related information (step S901). The information regarding pedestrian traffic may be acquired, for example, from map information. For example, if the location where the vehicle 50 is parked is in a busy downtown area, information indicating that there is a lot of pedestrian traffic may be acquired. Furthermore, the information regarding pedestrian traffic may be acquired taking into account the time of day. For example, information indicating that there is a lot of pedestrian traffic around a school in the morning (school arrival time) may be acquired. Note that the mobile object-related information may include information other than information regarding pedestrian traffic.

[0120] Next, the processing unit 130 determines whether the number of people passing through the area where the vehicle 50 is parked is greater than a predetermined threshold based on the acquired moving object-related information (step S902). The "predetermined threshold" here is a threshold for determining whether the number of people passing through the area (i.e., the number of detection targets) is so large that it can be determined that the load on the object detection process will become extremely large.

[0121] If the number of people passing by is greater than the predetermined threshold (step S902: YES), the processing unit 130 sets a detection threshold so that objects are difficult to detect in the object detection process (step S903). The processing unit 130 also sets a threshold so that face recognition processing is performed on objects approaching the vehicle (step S904). In other words, the processing unit 130 sets a threshold so that face recognition processing is not performed on detected objects that are simply passing by the vehicle 50 or that are moving away from the vehicle 50.

[0122] If the number of people passing by is less than the predetermined threshold (step 902: NO), the processing unit 130 omits the processes of steps S903 and S904. In this case, the detection threshold for the object detection process is not changed, and the processing unit 130 is set to perform face recognition process for targets other than those approaching the vehicle.

[0123] Thereafter, the processing unit 130 executes a process related to the authentication operation based on the set content (step S905). For example, if the number of people passing by is greater than a predetermined threshold and the processes of steps S903 and S904 are executed, the object detection process is executed in a state where it is difficult to detect objects. In addition, face recognition process is executed for targets approaching the vehicle 50.

[0124] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the ninth embodiment will be described.

[0125] As described with reference to FIG. 14 , in the information processing system 10 according to the ninth embodiment, when there is a lot of pedestrian traffic in the area where the vehicle 50 is parked, the object detection process is made less likely to detect objects, and the target of the facial recognition process is limited to people approaching the vehicle 50. This prevents an unnecessarily large number of objects from being detected or the facial recognition process from being performed on many objects, thereby suppressing an increase in the processing load. While suppressing the processing load can reduce power consumption, for example, when there is sufficient power (for example, when an electric vehicle (EV) is being charged), the change in the execution mode described above may not be necessary. In other words, when it is determined that suppressing the processing load is unnecessary, objects may be detected using a normal threshold value even in an area where there is a large number of pedestrian traffic, and the authentication process may be performed on objects other than those approaching the vehicle 50.

[0126] Tenth Embodiment An information processing system 10 according to a tenth embodiment will be described with reference to Fig. 15. Note that the tenth embodiment differs from the first to ninth embodiments described above only in some of its operations, and other parts may be the same as the first to ninth embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit a description of other overlapping parts as appropriate.

[0127] (Operation Flow) First, the operation flow of the information processing system 10 according to the tenth embodiment will be described with reference to Fig. 15. Fig. 15 is a flowchart showing the operation flow of the information processing system according to the tenth embodiment. Note that in Fig. 15, the same processes as those described in Fig. 3 are denoted by the same reference numerals.

[0128] As shown in FIG. 15, when the operation of the information processing system 10 according to the tenth embodiment is started, the image acquisition unit 110 first acquires an image of at least one of the inside and outside of a moving object (step S101).

[0129] Next, the information acquisition unit 120 acquires mobile-object-related information. At this time, the information acquisition unit 120 acquires the detection status of the terminal held by the passenger of the vehicle 50 as the mobile-object-related information (step S1001). The terminal held by the passenger may be a communication-enabled terminal such as a smartphone or a smartwatch. The terminal held by the passenger may also be detected using Bluetooth, for example. However, the terminal held by the passenger may also be detected using a method other than Bluetooth. For example, the information acquisition unit 120 may measure the distance between the vehicle 50 and the terminal held by the passenger in real time, and detect the terminal when the distance between them becomes within a predetermined distance.

[0130] Next, the processing unit 130 determines whether or not the terminal held by the passenger has been detected based on the acquired moving object-related information (step S1002). If the terminal held by the passenger has been detected (step 1002: YES), the processing unit 130 starts processing related to the authentication operation (step S1003). For example, the processing unit 130 starts capturing an image to be used in the object detection process.

[0131] On the other hand, if the terminal held by the passenger is not detected (step 1002: NO), the processing unit 130 does not start the process related to the authentication operation. For example, the processing unit 130 keeps stopping the capture of images to be used in the object detection process. In this case, the process from step S101 may be repeatedly executed.

[0132] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the tenth embodiment will be described.

[0133] 15 , in the information processing system 10 according to the tenth embodiment, processing related to an authentication operation is executed when a terminal held by a passenger in the vehicle 50 is detected. In this way, processing related to an authentication operation is not executed when a terminal held by the passenger is not detected (for example, when the passenger is far away from the vehicle 50), making it possible to suppress unnecessary execution of processing.

[0134] Eleventh Embodiment An information processing system 10 according to an eleventh embodiment will be described with reference to Fig. 16. Note that the eleventh embodiment differs only in some of the operations from the first to tenth embodiments described above, and other parts may be the same as the first to tenth embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit a description of other overlapping parts as appropriate.

[0135] (Operation Flow) First, the operation flow of the information processing system 10 according to the 11th embodiment will be described with reference to Fig. 16. Fig. 16 is a flowchart showing the operation flow of the information processing system according to the 11th embodiment. Note that in Fig. 16, the same processes as those described in Fig. 3 are denoted by the same reference numerals.

[0136] As shown in FIG. 16, when the operation of the information processing system 10 according to the eleventh embodiment is started, the image acquisition unit 110 first acquires an image of at least one of the inside and outside of a moving body (step S101).

[0137] Next, the information acquisition unit 120 acquires moving-object-related information. At this time, the information acquisition unit 120 acquires information about the target's body type or clothing as moving-object-related information (step S1101). The information about the target's body type or clothing may be acquired from an image of the target. The information about the target's body type may include, for example, information about the target's expected height or weight, or information indicating a type such as "slim," "average," or "muscular." The information about the target's clothing may include, for example, information indicating the color of the clothing, or information indicating the type of clothing (e.g., shirt, hoodie, skirt, coat, etc.). Note that the moving-object-related information may include information other than the information about the target's body type and clothing.

[0138] Next, the processing unit 130 determines whether the target is a passenger of the vehicle 50 based on the acquired moving-object-related information (step S1102). The processing unit 130 may determine whether the target is a passenger of the vehicle 50 by, for example, comparing the body type or clothing of the passenger stored in advance with the body type or clothing of the target indicated by the moving-object-related information.

[0139] If the target is determined to be a passenger (step S1102: YES), the processing unit 130 executes facial recognition processing for the target (step S1103). In this case, since it has been determined that the target is highly likely to be a passenger based on their body type or clothing, the threshold for facial recognition may be lowered (i.e., the target may be more easily recognized).

[0140] On the other hand, if the target is not determined to be a passenger (step S1102: NO), the processing unit 130 does not execute the facial recognition process for the target. That is, even if the target attempts to execute the facial recognition process, the facial recognition process is not executed (or the authentication is treated as a failure). In this case, if the target approaches the vehicle 50 and behaves suspiciously, not only may the facial recognition process not be executed but also alert information may be output.

[0141] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the eleventh embodiment will be described.

[0142] 16 , in the information processing system 10 according to the eleventh embodiment, it is determined whether a target is a passenger of the vehicle 50 based on the target's clothing or body type, and if the target is determined to be a passenger, facial recognition processing is executed. This prevents the facial recognition processing from being executed on a target that is clearly not a passenger (for example, a target whose clothing or body type is significantly different from that of a passenger). This prevents unnecessary processing from being executed, and also reduces the chances of erroneously recognizing a different person as a passenger through facial recognition.

[0143] <Twelfth Embodiment> An information processing system 10 according to a twelfth embodiment will be described with reference to Fig. 17. Note that the twelfth embodiment differs only in some operations from the first to eleventh embodiments described above, and other parts may be the same as the first to eleventh embodiments. Therefore, hereinafter, only the parts that differ from the embodiments already described will be described in detail, and descriptions of other overlapping parts will be omitted as appropriate.

[0144] (Operation Flow) First, the operation flow of the information processing system 10 according to the twelfth embodiment will be described with reference to Fig. 17. Fig. 17 is a flowchart showing the operation flow of the information processing system according to the twelfth embodiment. Note that in Fig. 17, the same processes as those described in Fig. 3 are denoted by the same reference numerals.

[0145] As shown in FIG. 17, when the operation of the information processing system 10 according to the twelfth embodiment is started, the image acquisition unit 110 first acquires an image of at least one of the inside and outside of the moving body (step S101).

[0146] Next, the information acquisition unit 120 acquires moving object-related information. At this time, the information acquisition unit 120 acquires information regarding the movement of the target as moving object-related information (step S1201). The movement of the target may be a gesture made by the target, or the movement of the target's line of sight or the movement of the target's face. The information regarding the movement of the target may be acquired from an image (video) of the target. Note that the moving object-related information may include information other than the information regarding the movement of the target.

[0147] Next, the processing unit 130 determines whether the target's movement matches a predetermined registration pattern based on the acquired moving object-related information (step S1202). The "registration pattern" here refers to a movement that is pre-stored as a starting condition for face recognition processing. Examples of a registration pattern include directing one's gaze toward a door and moving one's face up, down, left, or right. A registration pattern may be a combination of multiple movements. Alternatively, a registration pattern may require completion of a movement within a predetermined time (e.g., turning right and left within three seconds).

[0148] If the target's movement matches the registered pattern (step S1202: YES), the processing unit 130 determines that the target is likely to be a passenger in the vehicle 50 (knows the registered pattern) and performs face recognition processing (step S1203).

[0149] On the other hand, if the target's movement does not match the registered pattern (step S1202: NO), the processing unit 130 determines that the target is likely not a passenger in the vehicle 50 (does not know the registered pattern) and does not perform facial recognition processing (step S1204).

[0150] In the above example, the execution mode (executability) of the face recognition process is changed, but the execution mode of the object detection process may be changed in addition to or instead of the face recognition process. For example, the processing unit 130 may detect an object whose movement matches a registered pattern and may not detect an object whose movement does not match the registered pattern.

[0151] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the twelfth embodiment will be described.

[0152] 17 , in the information processing system 10 according to the twelfth embodiment, whether or not to execute the facial recognition process is determined based on the movement of the target. In this way, the target on which the facial recognition process is executed can be limited based on the movement. Therefore, it is possible to prevent the facial recognition process from being executed unnecessarily and to reduce the chance of erroneously recognizing a different person as a passenger through facial recognition.

[0153] <Thirteenth Embodiment> An information processing system 10 according to a thirteenth embodiment will be described with reference to Fig. 18. Note that the thirteenth embodiment differs only in some operations from the first to twelfth embodiments described above, and other parts may be the same as the first to twelfth embodiments. Therefore, hereinafter, only the parts that differ from the embodiments already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0154] (Operation Flow) First, the operation flow of the information processing system 10 according to the thirteenth embodiment will be described with reference to Fig. 18. Fig. 18 is a flowchart showing the operation flow of the information processing system according to the thirteenth embodiment. In Fig. 18, the same processes as those described in Fig. 3 are denoted by the same reference numerals. In addition, the process in Fig. 18 is premised on being executed in a state where the target is away from the vehicle 50.

[0155] As shown in FIG. 18, when the operation of the information processing system 10 according to the thirteenth embodiment is started, the image acquisition unit 110 first acquires an image of at least one of the inside and outside of the moving body (step S101).

[0156] Next, the information acquisition unit 120 acquires mobile-object-related information. At this time, the information acquisition unit 120 acquires information regarding the behavioral pattern of the target as the mobile-object-related information (step S1301). The behavioral pattern of the target may be information linking information indicating the target's behavior with date and time information. For example, the information regarding the target's behavioral pattern may include information indicating that the target uses the vehicle 50 from 8:00 a.m. on weekdays. Alternatively, the behavioral pattern of the target may be information linking information indicating the target's behavior with location information. For example, the information regarding the target's behavioral pattern may include information indicating that if the target parks the vehicle 50 at a specified store, it will return in about an hour. The information regarding the target's behavioral pattern may be learned as needed by storing the target's past behavior. Note that the mobile-object-related information may include information other than the information regarding the target's behavioral pattern.

[0157] Next, the processing unit 130 predicts the time when the target will return to the vehicle 50 based on the acquired mobile-object-related information (step S1302). That is, the processing unit 130 predicts the time when the target will return to the vehicle 50 based on the target's behavioral pattern. Then, the processing unit 130 determines whether the time remaining until the target will return is within a predetermined time (step S1303). The "predetermined time" here is a threshold value set for determining the timing for changing the priority of the authentication operation processing. If it is determined that the accuracy of the prediction of the time when the target will return is high, the predetermined time may be set to a relatively short time (e.g., approximately 1 to 3 minutes). On the other hand, if it is determined that the accuracy of the prediction of the time when the target will return is low, the predetermined time may be set to a longer time (e.g., approximately 30 minutes to 1 hour).

[0158] If it is determined that the target will return within the predetermined time (step S1303: YES), the processing unit 130 determines that the target is likely to return soon and increases the priority of the processing related to the authentication operation (step S1304). For example, the processing unit 130 turns on a camera that takes an image of the target or increases resources allocated to the processing related to the authentication operation.

[0159] On the other hand, if it is determined that it will take more than the predetermined time for the target to return (step S1303: NO), the processing unit 130 determines that it is highly likely that the target will not return for a while, and reduces the priority of the processing related to the authentication operation (step S1305). For example, the processing unit 130 may turn off the camera that takes images of the target, or reduce the resources allocated to the processing related to the authentication operation.

[0160] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the thirteenth embodiment will be described.

[0161] As described with reference to FIG. 18 , the information processing system 10 according to the thirteenth embodiment uses the behavioral patterns of the passengers of the vehicle 50 to predict the time when the subject will return to the vehicle 50 (i.e., the time when the subject will board the vehicle 50). Then, the priority of the authentication operation process is relatively low until a predetermined time before the predicted time. This prevents the authentication operation process from being executed during a time period when the subject is unlikely to use the vehicle 50, or prevents an unnecessary increase in resources allocated to the authentication operation process. Therefore, it is possible to efficiently prevent an increase in the processing load.

[0162] <Fourteenth Embodiment> An information processing device according to the fourteenth embodiment will be described with reference to Fig. 19. Note that the information processing device according to the fourteenth embodiment is configured by configuring the information processing systems 10 described in the first to thirteenth embodiments as a single device, and its configuration and operation may be similar to those of the information processing system 10 described above. For this reason, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0163] (Functional Configuration) First, the functional configuration of the information processing device according to the fourteenth embodiment will be described with reference to Fig. 19. Fig. 19 is a block diagram showing the functional configuration of the information processing device according to the fourteenth embodiment. Note that in Fig. 19, the same elements as those shown in Fig. 2 are denoted by the same reference numerals.

[0164] As shown in Fig. 19 , the information processing device 20 according to the fourteenth embodiment is configured as a device mounted on a vehicle 50. The information processing device 20 is configured to include, as components for realizing its functions, an image acquisition unit 110, an information acquisition unit 120, and a processing unit 130. That is, the information processing device 20 according to the fourteenth embodiment includes the same components as the information processing system 10 according to the first embodiment (see Fig. 2 ). The information processing device 20 may be configured, for example, by an ECU (Electronic Control Unit) provided in the vehicle 50.

[0165] (Technical Effects) Next, technical effects obtained by the information processing device 20 according to the fourteenth embodiment will be described.

[0166] In the information processing device 20 according to the fourteenth embodiment, a plurality of processes are executed based on images captured inside or outside the vehicle 50. The execution modes of the plurality of processes are controlled based on the moving object-related information. In this way, the plurality of processes executed in the vehicle 50 can be executed in an appropriate execution mode depending on the situation.

[0167] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, each embodiment includes not only a recording medium on which the above-described program is recorded, but also the program itself.

[0168] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. Furthermore, the scope of each embodiment is not limited to programs that execute processes by themselves, but also includes programs that execute processes by operating on an OS in conjunction with other software or expansion board functions. Furthermore, the program itself may be stored on a server, and part or all of the program may be downloadable from the server to a user terminal. The program may be provided to the user in, for example, a SaaS (Software as a Service) format.

[0169] <Supplementary Notes> The above-described embodiment may be further described as in the following supplementary notes, but is not limited to the following.

[0170] (Appendix 1) The information processing system described in Appendix 1 is an information processing system comprising an image acquisition means for acquiring images of at least one of the inside or outside of a moving body, a processing means for executing a plurality of different processes based on the images, and an information acquisition means for acquiring moving body-related information indicating the status of the moving body, wherein the processing means controls the execution mode of each of the plurality of processes based on the moving body-related information.

[0171] (Appendix 2) The information processing system described in Appendix 2 is the information processing system described in Appendix 1, in which the moving object is equipped with at least one camera that captures the images, and the processing means controls parameters of the at least one camera based on the moving object-related information.

[0172] (Supplementary Note 3) The information processing system described in Supplementary Note 3 is the information processing system described in Supplementary Note 1 or 2, wherein the plurality of processes include at least two processes out of an object detection process that detects objects present in the vicinity of the moving body, a face detection process that detects the face of a target present inside or around the moving body, a face authentication process that performs authentication using the face of the target, and an impersonation determination process that determines whether the target is being impersonated, and the processing means changes the priority of the object detection process, the face detection process, the face authentication process, and the impersonation determination process based on the moving body-related information.

[0173] (Appendix 4) The information processing system described in Appendix 4 is the information processing system described in Appendix 3, wherein the information acquisition means acquires the stoppage time of the moving body as the moving body-related information, and the processing means, when the moving body has been stopped for a first predetermined time or longer, sets the priority of the object detection processing higher than the priority of the face detection processing, the face authentication processing, and the impersonation determination processing.

[0174] (Appendix 5) The information processing system described in Appendix 5 is the information processing system described in Appendix 3 or 4, wherein the information acquisition means acquires the time when the object leaves the moving body as the moving body-related information, and the processing means unlocks the door lock of the moving body on the condition that the face authentication process is successful if the time when the object leaves is within a second predetermined time, and unlocks the door lock of the moving body on the condition that both the face authentication process and the impersonation determination process are successful if the time when the object leaves is the second predetermined time or longer.

[0175] (Supplementary Note 6) The information processing system described in Supplementary Note 6 is the information processing system described in any one of Supplements 3 to 5, wherein the information acquisition means acquires, as the mobile-body-related information, information indicating whether or not the subject is driving the mobile body, and when the subject is driving the mobile body, the processing means executes the object detection process as a process of detecting an object that affects the driving, and executes the face recognition process as a process of authenticating the face of the subject driving the mobile body.

[0176] (Appendix 7) The information processing system described in Appendix 7 is the information processing system described in any one of Appendices 3 to 6, wherein the information acquisition means acquires a location where the moving body is stopped as the moving body-related information, and the processing means, if the location where the moving body is stopped is a location where the subject is expected to return to the moving body with luggage, unlocks a door lock corresponding to a seat where the subject will sit and a door lock corresponding to a location where the subject will place their luggage, on condition that the facial recognition process is successful.

[0177] (Appendix 8) The information processing system described in Appendix 8 is the information processing system described in any one of Appendices 3 to 7, wherein the information acquisition means acquires a location where the moving body is stopped as the moving body-related information, and the processing means increases the priority of the object detection process, the face detection process, the face authentication process, and the impersonation determination process for unlocking door locks corresponding to seats other than the driver's seat of the moving body when the location where the moving body is stopped is a location used by a passenger other than the driver of the moving body.

[0178] (Appendix 9) The information processing system described in Appendix 9 is the information processing system described in any one of Appendices 3 to 8, wherein the information acquisition means acquires information regarding pedestrian traffic around a location where the moving body is stopped as the moving body-related information, and the processing means, when the number of pedestrian traffic around the location where the moving body is stopped is greater than a predetermined threshold, changes the threshold so that an object becomes more difficult to detect in the object detection process, and performs the face recognition process on an object approaching the moving body.

[0179] (Supplementary Note 10) The information processing system described in Supplementary Note 10 is the information processing system described in any one of Supplements 3 to 9, wherein the information acquisition means acquires whether or not a terminal held by a passenger of the moving body has been detected as the moving body-related information, and the processing means executes the object detection processing, the face detection processing, the face authentication processing, and the impersonation determination processing when the terminal is detected.

[0180] (Appendix 11) The information processing system described in Appendix 11 is the information processing system described in any one of Appendices 3 to 10, wherein the information acquisition means acquires at least one of a body shape or clothing of a target present in the vicinity of the moving body as the moving body-related information, and the processing means executes the face recognition processing when it determines that the target is a passenger of the moving body based on at least one of the body shape or clothing.

[0181] (Appendix 12) The information processing system described in Appendix 12 is the information processing system described in any one of Appendices 3 to 11, wherein the information acquisition means acquires information regarding the movement of a target present in the vicinity of the moving body as the moving body-related information, and the processing means determines whether or not to execute the face recognition processing based on the movement of the target.

[0182] (Supplementary Note 13) The information processing system described in Supplementary Note 13 is the information processing system described in any one of Supplements 3 to 12, wherein the information acquisition means acquires information relating to the behavioral patterns of a passenger of the mobile body as the mobile body-related information, and the processing means predicts the time when the passenger will return to the mobile body from the behavioral pattern, and lowers the priority of the object detection processing, the face detection processing, the face authentication processing, and the impersonation determination processing until a predetermined time before the return time.

[0183] (Appendix 14) The information processing device described in Appendix 14 is provided in a moving body and comprises an image acquisition means for acquiring images of at least one of the inside or outside of the moving body, a processing means for executing a plurality of different processes based on the images, and an information acquisition means for acquiring moving body-related information indicating the status of the moving body, and the processing means is an information processing device that controls the execution mode of each of the plurality of processes based on the moving body-related information.

[0184] (Appendix 15) The information processing method described in Appendix 15 is an information processing method in which at least one computer acquires images of at least one of the inside or outside of a moving body, performs multiple different processes based on the images, and acquires moving body-related information indicating the status of the moving body, and when performing the multiple processes, controls the execution mode of each of the multiple processes based on the moving body-related information.

[0185] (Appendix 16) The recording medium described in Appendix 16 is a recording medium having recorded thereon a computer program that causes at least one computer to execute an information processing method that acquires images of at least one of the inside or outside of a moving body, executes a plurality of different processes based on the images, and acquires moving body-related information that indicates the status of the moving body, and when executing the plurality of processes, controls the execution mode for each of the plurality of processes based on the moving body-related information.

[0186] (Appendix 17) The computer program described in Appendix 17 is a computer program that causes at least one computer to execute an information processing method, which acquires images of at least one of the inside and outside of a moving body, executes a plurality of different processes based on the images, and acquires moving body-related information that indicates the status of the moving body, and when executing the plurality of processes, controls the execution mode of each of the plurality of processes based on the moving body-related information.

[0187] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing systems, information processing devices, information processing methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure.

[0188] REFERENCE SIGNS LIST 10 Information processing system 11 Processor 20 Information processing device 50 Vehicle 110 Image acquisition unit 120 Information acquisition unit 130 Processing unit 131 Object detection unit 132 Face detection unit 133 Face authentication unit 134 Spoofing determination unit 210 First camera 220 Second camera

Claims

1. an image acquisition means for acquiring an image of at least one of the interior and exterior of a moving body; a processing means for executing a plurality of different processes based on the image; information acquisition means for acquiring moving object-related information indicating a status of the moving object; Equipped with the processing means controls an execution mode of each of the plurality of processes based on the moving object-related information. Information processing system.

2. the moving object is equipped with at least one camera that captures the image; the processing means controls parameters of the at least one camera based on the moving object-related information. The information processing system according to claim 1 .

3. the plurality of processes include at least two processes among an object detection process that detects an object present around the moving body, a face detection process that detects a face of a target present inside or around the moving body, a face authentication process that performs authentication using the face of the target, and an impersonation determination process that determines whether the target is impersonating another person, the processing means changes the priorities of the object detection process, the face detection process, the face authentication process, and the spoofing determination process based on the moving object-related information.

3. The information processing system according to claim 1 or 2.

4. the information acquisition means acquires a stop time of the moving object as the moving object-related information; the processing means, when the moving object has been stopped for a first predetermined time or longer, sets a priority of the object detection process higher than priorities of the face detection process, the face authentication process, and the spoofing determination process; The information processing system according to claim 3 .

5. the information acquisition means acquires, as the moving-object-related information, a time at which an object leaves the moving object; The processing means unlocks the door lock of the moving body on the condition that the face authentication process is successful when the time the target has left is within a second predetermined time, and unlocks the door lock of the moving body on the condition that both the face authentication process and the impersonation determination process are successful when the time the target has left is equal to or longer than the second predetermined time. The information processing system according to claim 3 .

6. the information acquisition means acquires, as the mobile-object-related information, information indicating whether or not the target is driving the mobile object; When the subject is driving the moving body, the processing means executes the object detection process as a process of detecting an object that may affect the driving, and executes the face authentication process as a process of authenticating the face of the subject who is driving the moving body. The information processing system according to claim 3 .

7. the information acquisition means acquires a location where the moving object is stopped as the moving object-related information; When the location where the moving body is stopped is a location where the target is expected to return to the moving body with luggage, the processing means unlocks a door lock corresponding to the seat where the target will sit and a door lock corresponding to the location where the target will place their luggage, on condition that the face authentication process is successful. The information processing system according to claim 3 .

8. the information acquisition means acquires a location where the moving object is stopped as the moving object-related information; When the location where the moving body is stopped is a location used by a passenger other than the driver of the moving body, the processing means increases the priority of the object detection process, the face detection process, the face authentication process, and the impersonation determination process for unlocking door locks corresponding to seats other than the driver's seat of the moving body. The information processing system according to claim 3 .

9. by at least one computer, Acquire an image of at least one of the interior and exterior of the moving object; performing a plurality of different processes based on the image; acquiring mobile-object-related information indicating a status of the mobile object; An information processing method, comprising: When the plurality of processes are executed, an execution mode of each of the plurality of processes is controlled based on the moving object-related information. Information processing methods.

10. At least one computer Acquire an image of at least one of the interior and exterior of the moving object; performing a plurality of different processes based on the image; acquiring mobile-object-related information indicating a status of the mobile object; A computer program for executing an information processing method, When the plurality of processes are executed, an execution mode of each of the plurality of processes is controlled based on the moving object-related information. Computer program.