Information processing device, information processing method, and recording medium
Patent Information
- Application Number
- JP2023132302
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-08-15
- Publication Date
- 2025-02-28
Smart Images
Figure 2025027519000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to the technical fields of an information processing device, an information processing method, and a recording medium. [Background technology]
[0002] Patent Document 1 describes a technique for confirming the correctness of camera operations by leaving text information without saving images. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2011-077811 A Summary of the Invention [Problem to be solved by the invention]
[0004] An object of the present disclosure is to provide an information processing device, an information processing method, and a recording medium that aim to improve upon the techniques disclosed in the prior art documents. [Means for solving the problem]
[0005] One aspect of the information processing device includes an acquisition means for acquiring facial image information including at least one of luminance information regarding the luminance of a facial image captured of the face of a subject in a specified space, occlusion information regarding whether a facial area in the facial image is occluded or not, and position information regarding the position of the facial area in the facial image, and a storage means for storing the facial image information corresponding to facial images captured after the subject enters and leaves the specified space, with the same identification information attached thereto.
[0006] One aspect of the information processing method acquires facial image information including at least one of luminance information regarding the luminance of a facial image captured of the face of a subject in a specified space, occlusion information regarding whether a facial area in the facial image is occluded or not, and position information regarding the position of the facial area in the facial image, and stores the facial image information corresponding to facial images captured after the subject enters and leaves the specified space, with the same identification information attached.
[0007] One embodiment of the recording medium has a computer program recorded thereon to cause a computer to execute an information processing method for acquiring facial image information including at least one of luminance information regarding the luminance of a facial image captured of the face of a subject in a specified space, occlusion information regarding whether a facial area in the facial image is occluded or not, and position information regarding the position of the facial area in the facial image, and storing the facial image information corresponding to facial images captured after the subject enters and leaves the specified space, with the same identification information attached thereto. [Brief description of the drawings]
[0008] [Figure 1] 1 is a block diagram showing an example of a configuration of an information processing device according to the present disclosure. [Diagram 2] FIG. 1 is a conceptual diagram showing an example of a situation in which an information processing device according to the present disclosure is applied. [Diagram 3] 1 is a block diagram showing an example of a configuration of an information processing device according to the present disclosure. [Figure 4] 11 is a flowchart illustrating an example of an information processing operation of an information processing device according to the present disclosure. [Diagram 5] 1 is a table showing an example of face image information according to the present disclosure. [Figure 6] 11 is a flowchart illustrating an example of an inference operation of an information processing device according to the present disclosure. [Figure 7] 1 is a block diagram showing an example of a configuration of an information processing device according to the present disclosure. [Figure 8] 1 is a table showing an example of summary information according to the present disclosure. [Figure 9]1 is a block diagram showing an example of a configuration of an information processing device according to the present disclosure. [Figure 10] FIG. 13 is a conceptual diagram illustrating an example of a visualization operation of an information processing device according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, embodiments of an information processing device, an information processing method, and a recording medium will be described with reference to the drawings. [1: First embodiment]
[0010] A first embodiment of an information processing device, an information processing method, and a recording medium will be described. Hereinafter, a first embodiment of an information processing device, an information processing method, and a recording medium will be described using an information processing device 1 according to the present disclosure. [1-1: Configuration of information processing device 1]
[0011] 1 is a block diagram showing a configuration of an information processing device 1 according to the present disclosure. As shown in FIG. 1, the information processing device 1 includes an acquisition unit 11 and a storage unit 12.
[0012] The acquisition unit 11 acquires face image information including at least one of luminance information, occlusion information, and position information. The luminance information is information about the luminance of a face image obtained by capturing the face of a target in a predetermined space. The occlusion information is information about whether a face area in the face image is occluded. The position information is information about the position of the face area in the face image.
[0013] The storage unit 12 stores face image information corresponding to face images captured from when the target enters the specified space until when the target leaves the specified space, with the same identification information attached to the face image information. For example, the storage unit 12 may store face image information corresponding to face images captured from when the target starts to detect the same target in the specified space until the target finishes detecting the target. In other words, the storage unit 12 stores face image information corresponding to face images captured while the target is using the specified space, with the same identification information attached to the face image information. In other words, the storage unit 12 stores face image information with the same identification information attached for each unit of use of the specified space. [1-2: Technical effects of information processing device 1]
[0014] The information processing device 1 according to the present disclosure acquires face image information related to a face image and stores the face image information with the same identification information for each unit of use of a predetermined space, so that it is possible to collect information useful for understanding the situation when the face image was taken. In addition, since the information processing device 1 does not store the face image itself but stores face image information related to the face image, there is no risk of violating the privacy of the subject. [2: Second embodiment]
[0015] A second embodiment of an information processing device, an information processing method, and a recording medium will be described. Hereinafter, a second embodiment of an information processing device, an information processing method, and a recording medium will be described using an information processing device 2 according to the present disclosure. [2-1: Example of a situation in which the information processing device 2 is applied]
[0016] FIG. 2 illustrates an example of a scene to which the information processing device 2 is applied. In this embodiment, the predetermined space is a private space 100. FIG. 2 illustrates a toilet as an example of the private space 100. The private space 100 is provided with an entrance 101, a toilet bowl 102, and a camera C. The entrance 101 may be provided so as to be openable and closable so that the target T can enter and exit the private space 100. The camera C may be provided so as to capture the face of the target T when the target T enters and exits the private space 100. The camera C may be controlled to capture an image of the shooting range R when the entry of the target T into the private space 100 is detected. The camera C may be installed so as to capture a situation in which the target T does not have any resistance even if seen by others. For example, the camera C may be installed so as not to capture the target T when using the toilet bowl 102. In this embodiment, the information processing device 2 performs face recognition of the target T in the private space 100 as a process using a face image. In this disclosure, the "image" may be a still image or a video. [2-2: Configuration of information processing device 2]
[0017] Fig. 3 is a block diagram showing the configuration of the information processing device 2. As shown in Fig. 3, the information processing device 2 includes a calculation device 21 and a storage device 22. Furthermore, the information processing device 2 may include a communication device 23, an input device 24, and an output device 25. However, the information processing device 2 does not have to include at least one of the communication device 23, the input device 24, and the output device 25. The calculation device 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.
[0018] The arithmetic device 21 includes at least one of a central processing unit (CPU), a graphics processing unit (GPU), and a field programmable gate array (FPGA). The arithmetic device 21 reads a computer program. For example, the arithmetic device 21 may read a computer program stored in the storage device 22. For example, the arithmetic device 21 may read a computer program stored in a computer-readable and non-transient recording medium using a recording medium reading device (e.g., an input device 24 described later) not shown in the drawings that is provided in the information processing device 2. The arithmetic device 21 may acquire (i.e., download or read) a computer program from a device (not shown) disposed outside the information processing device 2 via a communication device 23 (or another communication device). The arithmetic device 21 executes the read computer program. As a result, a logical function block for executing the operation to be performed by the information processing device 2 is realized in the arithmetic device 21. That is, the arithmetic device 21 can function as a controller for realizing logical functional blocks for executing operations (in other words, processes) to be performed by the information processing device 2. The arithmetic device 21 may output information to devices (not shown), such as other computers and cloud servers, provided outside the information processing device 2, via the communication device 23 (or other communication devices).
[0019] 3 shows an example of a logical functional block realized in the arithmetic device 21 to execute an information processing operation. As shown in FIG. 3, the arithmetic device 21 realizes a face image information acquisition unit 211 which is a specific example of an "acquisition means" described in the appendix to be described later, a storage unit 212 which is a specific example of a "storage means" described in the appendix to be described later, a face image acquisition unit 213, a face recognition unit 214 which is a specific example of a "face recognition means" described in the appendix to be described later, an estimation unit 215 which is a specific example of a "estimation means" described in the appendix to be described later, and an output unit 216 which is a specific example of an "output means" described in the appendix to be described later. However, at least one of the face image acquisition unit 213, the face recognition unit 214, the estimation unit 215, and the output unit 216 does not have to be realized in the arithmetic device 21. The operations of the facial image information acquisition unit 211, the storage unit 212, the facial image acquisition unit 213, the facial recognition unit 214, the estimation unit 215, and the output unit 216 will be described in detail later with reference to FIGS.
[0020] The storage device 22 can store desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic device 21. The storage device 22 may temporarily store data that the arithmetic device 21 temporarily uses when the arithmetic device 21 is executing a computer program. The storage device 22 may store data that the information processing device 2 stores for a long period of time. The storage device 22 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device. In other words, the storage device 22 may include a non-temporary recording medium.
[0021] The communication device 23 is capable of communicating with devices external to the information processing device 2 via a communication network (not shown). The communication device 23 may be a communication interface based on standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), Bluetooth (registered trademark), and USB (Universal Serial Bus).
[0022] The input device 24 is a device that accepts information input to the information processing device 2 from outside the information processing device 2. For example, the input device 24 may include an operation device (for example, at least one of a keyboard, a mouse, and a touch panel) that can be operated by an operator of the information processing device 2. For example, the input device 24 may include a reading device that can read information recorded as data on a recording medium that can be externally attached to the information processing device 2.
[0023] The output device 25 is a device that outputs information to the outside of the information processing device 2. For example, the output device 25 may output information as an image. That is, the output device 25 may include a display device (so-called a display) capable of displaying an image showing the information to be output. For example, the output device 25 may output information as sound. That is, the output device 25 may include an audio device (so-called a speaker) capable of outputting sound. For example, the output device 25 may output information on paper. That is, the output device 25 may include a printing device (so-called a printer) capable of printing desired information on paper. [2-3: Information processing operation performed by information processing device 2]
[0024] The information processing operation performed by the information processing device 2 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an example of the flow of the information processing operation performed by the information processing device 2.
[0025] 4, the face image acquisition unit 213 determines whether or not the target T has entered the private space 100 (step S20). The face image acquisition unit 213 may determine whether or not the target T has entered depending on, for example, whether the entrance 101 is open or closed.
[0026] The face image acquisition unit 213 acquires a face image of the target T captured by the camera C (step S21). The face image acquisition unit 213 may control the camera C to capture an image of the target T who has entered the private space 100. The face image acquisition unit 213 may acquire a still image captured by the camera C, or may acquire one frame of a video.
[0027] The face recognition unit 214 performs face recognition using the face image acquired by the face image acquisition unit 213 (step S22). The face recognition unit 214 may detect a face area from the face image acquired by the face image acquisition unit 213 and compare information on the detected face area with registered information to perform face recognition. The face recognition unit 214 may extract feature points such as the eyes, nose, and mouth from the detected face and compare information on the extracted feature points with registered information to perform face recognition. In this embodiment, the face recognition operation may be an operation equivalent to the face matching operation.
[0028] The face image information acquisition unit 211 acquires face image information from the face image acquired by the face image acquisition unit 213 (step S23). The face image information includes at least one of brightness information, occlusion information, and position information. Each of the brightness information, occlusion information, and position information may be referred to as element information.
[0029] The luminance information is information regarding the luminance of the face image. The luminance information may include information regarding the luminance of the entire face image. That is, the luminance information may include information representing the lighting state of the private space. The luminance information may be information indicating a defect in the lighting itself. Furthermore, the luminance information may include information regarding the luminance of the face region in the face image. The luminance information may include information representing the luminance of the face image and information representing the luminance of the face region of the face image. The luminance information may include information representing how the lighting hits the face. The luminance information may be information indicating the positional relationship between the lighting and the face. The luminance information may be information indicating that the lighting hits the face is backlit.
[0030] The occlusion information is information about whether or not a face region in a face image is occluded. The thing occluding the face region may be, for example, a mask. The thing occluding the face region may also be sunglasses.
[0031] The position information is information regarding the position of a face region in a face image. The position information may include information indicating the position of the target's face. The position of the face may be, for example, the position of the upper left corner of the face region in the face image. The position information may also include information indicating the positions of parts included in the face. The parts included in the face may be, for example, the left eye, the right eye, the nose, and the mouth. The positions of the left eye and the right eye may be the positions of the centers of the pupils. The position of the nose may be the position of the tip of the nose. The position of the mouth may be the positions of the left corner of the mouth and the right corner of the mouth.
[0032] The storage unit 212 stores the facial image information by assigning a management ID to it (step S24). The management ID is identification information for identifying facial image information on a usage unit basis of the private space 100. Therefore, the storage unit 212 stores the facial image information corresponding to the facial images taken from when the subject T entered the private space 100 until when he / she left the private space 100 by assigning the same management ID to the facial image information. A facial image information table may be realized in the storage device 22. In this case, the storage unit 212 may store a record of the facial image information including the management ID in the facial image information table in the storage device 22.
[0033] The face image information acquisition unit 211 acquires face image information in the form of text data. The storage unit 212 stores the face image information in the form of text data. FIG. 5 illustrates an example of a face image information table. As illustrated in FIG. 5, the storage unit 212 may store the face image information in a CSV (Comma Separated Value) format. FIG. 5 illustrates that each record stored in the face image information table includes a management ID, a shooting time, a user ID identified by face recognition, a certainty of the face recognition result, and a plurality of element information. FIG. 5 illustrates a case in which the face image information includes, as element information, the brightness of an image, the brightness of a face, the X coordinate of a face, the Y coordinate of a face, the width of a face, the height of a face, the presence or absence of a mask, the X coordinate of a left eye, the X coordinate of a right eye, the X coordinate of a nose, the X coordinate of a left side of a mouth, the X coordinate of a right side of a mouth, the Y coordinate of a left eye, the Y coordinate of a right eye, the Y coordinate of a nose, the Y coordinate of a left side of a mouth, and the Y coordinate of a right side of a mouth. 5, in the face image information table, one face image may correspond to one record of face image information. The storage unit 212 assigns a different management ID for each use of the private space 100, so that information for each use can be distinguished.
[0034] The face image acquisition unit 213 determines whether or not the target T has left the private space 100 (step S25). If the target T has not left the private space 100 (step S25: No), the process may return to step S21.
[0035] When the target T has left the private space 100 (step S25: Yes), the estimation unit 215 determines whether or not the face recognition of the target T has been successful (step S26).
[0036] If the facial recognition of the target T fails (step S26: No), the estimation unit 215 estimates the cause of the failure based on the facial image information with the same management ID (step S27). The causes of the failure of the facial recognition are often "I: Insufficient illumination of the space", "II: Possibility that the distance between the camera C and the target T is far", "III: Possibility that the target T is wearing a mask", "IV: Possibility that the photograph was taken in backlight", "V: Possibility that the face is out of the angle of view", and "VI: Possibility that the face was photographed not facing forward". Therefore, it may be determined which of these causes applies in order. The cause of the failure of the facial recognition may apply to multiple causes.
[0037] Fig. 6 is a flowchart showing an example of the flow of the operation of analyzing the cause of failure. As shown in Fig. 6, the estimation unit 215 judges the luminance of the entire face image (step S30). If the entire face image is dark, the estimation unit 215 may estimate that the cause of the failure in face recognition (cause I) is insufficient illuminance in the private space 100. In a small space such as a toilet, the illuminance may be insufficient.
[0038] The estimation unit 215 determines the size of the face in the face image (step S31). When performing face recognition, if the face image is small, the feature amount may be insufficient. If face matching is performed with insufficient feature amount, there is a risk of false detection, so if the face size is small, it may be desired to exclude the face from the target of face recognition. If the face in the face image is small, the estimation unit 215 may estimate that the cause of the failure of face recognition (cause II) is that the target T is far from the camera C and the information required for face matching is insufficient.
[0039] In addition, when the target T is assisted by another person, it may be desired to exclude this other person from the target of face recognition. In such a case, the target of face recognition may be limited based on the size of the face. Therefore, the limitation based on the size of the face may cause the failure of face recognition.
[0040] The estimation unit 215 determines whether or not the target T is wearing a mask (step S32). If the target T is wearing a mask, the estimation unit 215 may estimate that the cause of the failure of the face recognition is that the target T is not a target for face recognition (cause III).
[0041] Furthermore, when the subject T is receiving assistance from another person, it may be desirable to exclude this other person from the target of face recognition. Generally, in nursing care and medical care sites, employees often wear masks. In such cases, the target of face recognition may be limited depending on whether or not the person is wearing a mask. For example, a person wearing a mask may be presumed to be an employee, and the person wearing a mask may be excluded from the target of face recognition. For this reason, limiting the target based on the presence or absence of a mask may cause face recognition to fail.
[0042] The estimation unit 215 judges the luminance of the face area of the face image (step S33). If the face area is dark, the estimation unit 215 may estimate that the cause of the failure in face recognition is that the image was taken in a backlit state (cause IV). Depending on the position of the lighting, backlighting may occur.
[0043] The estimation unit 215 determines the coordinates of the face (step S34). When the coordinates of the face are far from the center of the face image, the estimation unit 215 may estimate that the cause of the failure in face recognition (cause V) is that at least a part of the face area is out of the angle of view. The estimation unit 215 may estimate that the cause of the failure in face recognition is that a part of the face area necessary for face recognition is out of the angle of view. In a private space 100 such as a toilet, the angle of view is often limited to the upper body centered on the face, taking into consideration the psychological burden. For this reason, when the subject T uses a wheelchair, the face may be out of the angle of view of the camera.
[0044] The estimation unit 215 determines the orientation of the face in the face image (step S35). The estimation unit 215 may determine the orientation of the face in the face image based on the positional relationship of each part included in the face. When performing face recognition on a moving target T, the angle at which the face is captured may become poor. When the inclination of the face is significant, the estimation unit 215 may estimate that the reason for the failure in face recognition (cause VI) is that the face is not facing forward.
[0045] The output unit 216 outputs the result of the face recognition (step S28). If the face recognition has failed, the output unit 216 also outputs the cause of the face recognition failure estimated in step S27. The output unit 216 may output at least one of the result of the face recognition and the cause of the face recognition failure to a device (not shown), such as another computer or a cloud server, provided outside the information processing device 2, via the communication device 23 (or other communication device). The output unit 216 may control the output device 25 to output at least one of the result of the face recognition and the cause of the face recognition failure. [2-4: Variation]
[0046] The face image information can be used not only to estimate the cause of failure of processing using face images for each utilization unit of the private space 100, but also for more advanced functions. For example, the information processing device 2 may have a mechanism for deriving an appropriate installation position and angle of the camera C from the face image information, and may output information regarding changes to the installation position, angle, etc. Furthermore, the information processing device 2 may autonomously change the installation position, angle, etc. of the camera C. In addition, the information processing device 2 may be configured to have a mechanism for automatically adjusting the brightness of the lighting device and the intake of external light according to the time of shooting, etc., based on the face image information. The information processing device 2 may suppress a decrease in the accuracy of face authentication due to changes in the environment. [2-5: Technical effects of information processing device 2]
[0047] In a system that performs processing using a face image captured by a camera, if the processing fails, it is easy to imagine that the cause of the failure can be inferred using the captured face image itself. On the other hand, since face images are important personal information, in consideration of protecting the privacy of the person appearing in the face image, it is desirable to operate the system without leaving the face image itself. In particular, when using face images captured in private spaces, it is desirable to operate the system without leaving the face image itself. There is also a demand for processing using face images such as face recognition on edge devices with limited computing resources. In this case, too, it is desirable to operate the system without storing the face image itself. However, in order to understand the situation at the site where the image was captured without leaving the face image, a mechanism for understanding the situation is required.
[0048] Incidentally, causes of failure in processing using face images such as face recognition are often any one of the following: "I: Insufficient illumination of the space", "II: Possibility that the distance between camera C and subject T is too far", "III: Possibility that subject T is wearing a mask", "IV: Possibility that the photograph was taken against the light", "V: Possibility that the face is out of the angle of view", and "VI: Possibility that the face was photographed not facing forward". Therefore, the information processing device 2 infers the cause by acquiring and managing information that can identify these main causes.
[0049] The information processing device 2 according to the present disclosure manages face image information, which is text data, for each unit of use of the private space 100, and can infer the cause of failure when face recognition fails based on the face image information. Since the information processing device 2 does not store images for analysis, it is possible to operate the face recognition system without violating privacy. When face recognition is not possible, the cause can be inferred from the face image information, so that measures can be taken immediately when an inquiry is made from a user. Since the information processing device 2 stores face image information, which is text data, it is possible to alleviate the tightness of the storage capacity in the device. Furthermore, even in a device with low processing capacity such as an edge device, it is possible to infer the cause of failure. Furthermore, even when information is stored in a remote server, it is possible to alleviate communication costs and insufficient storage capacity. [3: Third embodiment]
[0050] A third embodiment of an information processing device, an information processing method, and a recording medium will be described. Hereinafter, a third embodiment of an information processing device, an information processing method, and a recording medium will be described using an information processing device 3 according to the present disclosure. [3-1: Configuration of information processing device 3]
[0051] An example of the configuration of the information processing device 3 will be described with reference to Fig. 7. Fig. 7 is a block diagram showing an example of the configuration of the information processing device 3.
[0052] As shown in FIG. 7, the information processing device 3 includes a calculation device 21 and a storage device 22, similar to the information processing device 2. Furthermore, the information processing device 3 may include a communication device 23, an input device 24, and an output device 25, similar to the information processing device 2. However, the information processing device 3 does not need to include at least one of the communication device 23, the input device 24, and the output device 25. The information processing device 3 differs from the information processing device 2 in that a generation unit 317 is realized in the calculation device 21. Other features of the information processing device 3 may be the same as other features of the information processing device 2. For this reason, hereinafter, parts that differ from the embodiments already described will be described in detail, and other overlapping parts will be appropriately omitted. [3-2: Information processing operation performed by information processing device 3]
[0053] The generating unit 317 generates summary information summarizing the situation from when the target T enters the private space 100 until when the target T leaves the private space 100, based on the facial image information having the same management ID. In other words, the generating unit 317 processes the facial image information into summary information. The generating unit 317 may generate summary information useful for estimating the cause of failure when processing using a facial image fails, based on the facial image information.
[0054] A summary information table may be implemented in the storage device 22. In this case, the generation unit 317 may store the summary information generated for each usage unit in the summary information table in the storage device 22. That is, the summary information table contains one record corresponding to the management ID.
[0055] Fig. 8 illustrates an example of a summary information table. As illustrated in Fig. 8, the summary information table may include the management ID for each usage unit, time, authentication result (user ID), confidence level, image brightness (information about cause I), face size (information about cause I), presence or absence of a mask (information about cause III), face brightness (information about cause IV), face position (left / right and up / down) (information about cause V), face orientation (left / right and up / down) (information about cause VI), and the number of shots. By referring to the summary information, the administrator can infer the cause of the face recognition failure without checking the captured images and videos themselves.
[0056] For example, if the "image brightness" is "dark," it can be inferred that cause I is the cause of the failure of face recognition. If the "face size" is "small," it can be inferred that cause II is the cause of the failure of face recognition. If the "mask" is "present," it can be inferred that cause III is the cause of the failure of face recognition. If the "face brightness" is "dark," it can be inferred that cause IV is the cause of the failure of face recognition. If the "face position" is not "center," it can be inferred that cause V is the cause of the failure of face recognition. If the "face orientation" is not "frontal," it can be inferred that cause VI is the cause of the failure of face recognition.
[0057] The information processing device 3 may have a mechanism for directly transmitting summary information generated from face image information stored in the device to system users. By having such a function, the number of maintenance steps for the system can be reduced. [3-3: Technical effects of information processing device 3]
[0058] The information processing device 3 disclosed herein generates summary information summarizing the situation from when the subject T enters the private space 100 until when he / she leaves the private space 100 based on facial image information having the same management ID, and is therefore useful for understanding the situation in the private space 100 for each unit of usage. [4: Fourth embodiment]
[0059] Fourth embodiment of an information processing device, an information processing method, and a recording medium will be described. Hereinafter, a fourth embodiment of an information processing device, an information processing method, and a recording medium will be described using an information processing device 4 according to the present disclosure. [4-1: Configuration of information processing device 4]
[0060] An example of the configuration of the information processing device 4 will be described with reference to Fig. 9. Fig. 9 is a block diagram showing an example of the configuration of the information processing device 4.
[0061] As shown in FIG. 9, the information processing device 4 includes a calculation device 21 and a storage device 22, similar to the information processing device 2 and the information processing device 3. Furthermore, the information processing device 4 may include a communication device 23, an input device 24, and an output device 25, similar to the information processing device 2 and the information processing device 3. However, the information processing device 3 may not include at least one of the communication device 23, the input device 24, and the output device 25. The information processing device 4 differs from the information processing device 2 and the information processing device 3 in that a visualization unit 418 is realized in the calculation device 21. Other features of the information processing device 4 may be the same as other features of the information processing device 2. For this reason, hereinafter, parts that are different from the already described embodiments will be described in detail, and other overlapping parts will be appropriately omitted. [4-2: Information processing operation performed by information processing device 4]
[0062] The visualization unit 418 visualizes time-series changes in at least one of the position of the face of the target T and the position of the part included in the face, based on the face image information. The visualization unit 418 may draw an image as exemplified in FIG. 10, based on information indicating the position of the face of the target T, included in the face image information, and information indicating the position of the part included in the face. That is, the visualization unit 418 may draw a face outline F, based on information indicating the X coordinate, Y coordinate, width, and height of the face, included in the face image information. The visualization unit 418 may also draw a left eye LE, based on the X coordinate of the left eye and the Y coordinate of the left eye, included in the face image information. The visualization unit 418 may also draw a right eye RE, based on the X coordinate of the right eye and the Y coordinate of the right eye, included in the face image information. The visualization unit 418 may also draw a nose N, based on the X coordinate of the nose and the Y coordinate of the nose, included in the face image information. The visualization unit 418 may also draw a mouth M, based on the X coordinates of the left and right sides of the mouth, and the Y coordinates of the left and right sides of the mouth, included in the face image information.
[0063] The visualization unit 418 may draw an image based on the position of the face and the positions of the parts included in the face for each captured face image, and display them side by side. As a result, as illustrated in FIG. 10, the visualization unit 418 can reproduce the time-series changes in the position of the face of the target T and the positions of the parts included in the face. In other words, by using the information visualized by the visualization unit 418, it is possible to infer the movement of the target T and the situation in which the target T uses the private space 100 without storing photos or videos. In addition, it is possible to analyze the cause of a failure in face recognition from a series of movements. [4-3: Technical effects of information processing device 4]
[0064] The information processing device 4 according to the present disclosure visualizes time-series changes in at least one of the position of the face of the target T and the positions of parts included in the face based on face image information related to the face image, and is therefore useful for understanding the situation of the target T in the private space 100. The information processing device 4 can easily understand the situation in the private space 100, such as the movement of the target T, compared to techniques such as skeletal detection and estimation that estimate the movement of the target.
[0065] In a care setting, for example, excretion assistance places a heavy burden on the caregiver. For this reason, support that can reduce the workload is needed. When the information processing device 4 is applied to a care setting, (1) it is possible to identify the subject T, who is a care recipient using the private space 100, by face recognition, and (2) to grasp the condition of the subject T in the private space 100, thereby reducing the burden on the caregiver. The output unit 216 may notify the caregiver of the condition of the subject T in the private space 100.
[0066] In the above embodiment, a case where facial recognition of a target is performed has been described as an example of processing using a facial image, but processing using a facial image is not limited to identifying an individual using a facial image. For example, emotions, age, etc. can be identified using a facial image. Even when emotions, age, etc. are identified using a facial image, the same identification information may be assigned to each unit of use of a specified space and facial image information may be stored, and information useful for understanding the situation when the facial image was captured may be collected. Even when identification of emotions, age, etc. using a facial image fails, the cause of the failure may be inferred based on the stored facial image information. [5: Notes]
[0067] The following supplementary notes are further disclosed regarding the above-described embodiment. [Appendix 1] an acquisition means for acquiring face image information including at least one of luminance information on the luminance of a face image captured of a face of a target in a predetermined space, occlusion information on whether a face area in the face image is occluded, and position information on the position of the face area in the face image; a storage means for storing the facial image information corresponding to the facial images captured from the time the subject enters the predetermined space until the time the subject leaves the predetermined space, with the same identification information attached thereto; An information processing device comprising: [Appendix 2] a guessing means for guessing a cause of failure when a process using the face image fails, based on the face image information to which the same identification information is attached; an output means for outputting information indicating the cause of the failure; The information processing device according to claim 1 . [Appendix 3] a face recognition means for performing face recognition as a process using the face image, The output means outputs at least one of the result of the face recognition and the cause of the failure. The information processing device according to claim 2. [Appendix 4] The face image information is text data. The information processing device according to claim 1. [Appendix 5] The predetermined space is a private space. The information processing device according to claim 1. [Appendix 6] The position information includes at least one of information indicating a position of a face of the subject and information indicating positions of parts included in the face. The information processing device according to claim 1. [Appendix 7] A generating means for generating summary information summarizing a situation from when the subject enters the predetermined space until when the subject leaves the predetermined space, based on the face image information to which the same identification information is attached. The information processing device according to claim 1 . [Appendix 8] A visualization unit is provided for visualizing time series changes in at least one of the position of the target's face and the positions of parts included in the face based on the face image information. The information processing device according to claim 1. [Appendix 9] Acquire face image information including at least one of luminance information on the luminance of a face image obtained by photographing the face of a target in a predetermined space, occlusion information on whether a face area in the face image is occluded, and position information on the position of the face area in the face image; The facial image information corresponding to the facial images captured from the time the subject enters the predetermined space until the time the subject leaves the space is stored with the same identification information attached thereto. Information processing methods. [Appendix 10] On the computer, Acquire face image information including at least one of luminance information on the luminance of a face image obtained by photographing the face of a target in a predetermined space, occlusion information on whether a face area in the face image is occluded, and position information on the position of the face area in the face image; The facial image information corresponding to the facial images captured from the time the subject enters the predetermined space until the time the subject leaves the space is stored with the same identification information attached thereto. A recording medium on which a computer program for executing an information processing method is recorded.
[0068] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-mentioned embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be appropriately combined with other embodiments. [Explanation of symbols]
[0069] 1,2,3,4 Information processing device 11 Acquisition Department 12,212 storage unit 211 Facial image information acquisition unit 213 Face image acquisition unit 214 Face Recognition Unit 215 Guessing part 216 Output section 317 Generation part 418 Visualization section
Claims
1. an acquisition means for acquiring face image information including at least one of luminance information on the luminance of a face image obtained by capturing the face of a target in a predetermined space, occlusion information on whether a face area in the face image is occluded, and position information on the position of the face area in the face image; a storage means for storing the facial image information corresponding to the facial images captured from the time the subject enters the predetermined space until the time the subject leaves the predetermined space, with the same identification information attached thereto; An information processing device comprising:
2. a guessing means for guessing a cause of failure when a process using the face image fails, based on the face image information to which the same identification information is attached; an output means for outputting information indicating the cause of the failure; The information processing device according to claim 1 .
3. a face recognition means for performing face recognition as a process using the face image, The output means outputs at least one of the result of the face recognition and the cause of the failure. The information processing device according to claim 2 .
4. The face image information is text data. The information processing device according to claim 1 .
5. The predetermined space is a private space. The information processing device according to claim 1 .
6. The position information includes at least one of information indicating a position of a face of the subject and information indicating positions of parts included in the face. The information processing device according to claim 1 .
7. A generating means for generating summary information summarizing a situation from when the subject enters the predetermined space until when the subject leaves the predetermined space, based on the face image information to which the same identification information is attached. The information processing device according to claim 1 .
8. A visualization unit is provided for visualizing time series changes in at least one of the position of the target's face and the positions of parts included in the face based on the face image information. The information processing device according to claim 1 .
9. Acquire face image information including at least one of luminance information on the luminance of a face image obtained by photographing the face of a target in a predetermined space, occlusion information on whether a face area in the face image is occluded, and position information on the position of the face area in the face image; The facial image information corresponding to the facial images captured from the time the subject enters the predetermined space until the time the subject leaves the space is stored with the same identification information attached thereto. Information processing methods.
10. On the computer, Acquire face image information including at least one of luminance information on the luminance of a face image obtained by photographing the face of a target in a predetermined space, occlusion information on whether a face area in the face image is occluded, and position information on the position of the face area in the face image; The facial image information corresponding to the facial images captured from the time the subject enters the predetermined space until the time the subject leaves the space is stored with the same identification information attached thereto. A recording medium on which a computer program for executing an information processing method is recorded.
Citation Information
Patent Citations
Image processing apparatus and image processing method
JP2011077811A