Information processing device, information processing method, and program

Abstract

This invention provides an information processing device that can easily ensure security for the output screen. [Solution] The system comprises a screen output unit 101 that outputs information to the screen, an image acquisition unit 103 that acquires an image taken while information is being output to the screen, a determination unit 106 that determines whether the image acquired by the image acquisition unit 103 satisfies security conditions, and a security processing unit 107 that, if the determination unit 106 determines that the image satisfies security conditions, performs security processing to change the screen output by the screen output unit 101.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus or the like that performs processing for ensuring security.

Background Art

[0002] As a conventional technique, one that performs face recognition of a logged-in user has been known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, conventionally, security can be ensured at the time of login, but it has been difficult to maintain security for the output screen, and there has been a problem that security cannot be easily ensured.

[0005] The present invention has been made to solve the above problems, and an object thereof is to provide an information processing apparatus or the like that can easily ensure security for the output screen.

Means for Solving the Problems

[0006] The information processing apparatus of the present invention includes a screen output unit that outputs information to a screen, an image acquisition unit that acquires an image taken when the information is output to the screen, a determination unit that determines whether the image acquired by the image acquisition unit satisfies security conditions, and a security processing unit that performs security processing for changing the screen output by the screen output unit when the determination unit determines that the image satisfies the security conditions.

[0007] This configuration makes it easy to ensure security for the displayed screen.

[0008] Furthermore, in the information processing apparatus of the present invention, the security condition may be that the image contains multiple people.

[0009] This configuration ensures security for the displayed screen when multiple people are viewing it.

[0010] Furthermore, in the information processing apparatus of the present invention, the security condition is that the image contains an unregistered face, and the determination unit detects a face from the image acquired by the image acquisition unit and determines whether the face can be recognized using face recognition information for recognizing one or more user faces stored in the face recognition information storage unit.

[0011] This configuration prevents users who are not pre-registered from viewing the screen, thereby ensuring the security of the displayed screen.

[0012] Furthermore, in the information processing apparatus of the present invention, the security condition is that the file or data output on the screen includes a face that does not have access rights, the screen output unit outputs the file or data, the determination unit detects a face from the image acquired by the image acquisition unit, and performs face recognition on the detected face using face recognition information for recognizing one or more user faces stored in the face recognition information storage unit, and determines whether the detected face is the face of a user who has the right to access the file or data output on the screen, as identified by the access rights information stored in the access rights information storage unit, which identifies a user who has the right to access the file or data.

[0013] This configuration allows users who do not have access to the output image to view the screen, thereby ensuring security regarding the output screen.

[0014] Furthermore, in the information processing device of the present invention, the security condition may be defined as recognizing a predetermined movement.

[0015] This configuration ensures security for the displayed screen by preventing viewing the screen when predetermined movements are detected.

[0016] Furthermore, in the information processing apparatus of the present invention, the security processing may be configured to be a process that hides the output information.

[0017] This configuration allows for security by hiding information displayed on the screen when security conditions are met.

[0018] Furthermore, in the information processing apparatus of the present invention, the security processing may be a process that hides only a portion of the two or more pieces of information that are output.

[0019] This configuration allows for security by hiding some of the information displayed on the screen when security conditions are met.

[0020] Furthermore, in the information processing apparatus of the present invention, the security processing may be configured to lock the screen on which the information is output.

[0021] With this configuration, if the security conditions are met, the displayed screen can be used as the lock screen, and security can be easily ensured for the displayed screen.

Advantages of the Invention

[0022] According to the information processing apparatus and the like of the present invention, security can be easily ensured for the output screen.

Brief Description of the Drawings

[0023] [Figure 1] Block diagram of the information processing apparatus in the embodiment of the present invention [Figure 2] Flowchart for explaining the operation of the information processing apparatus [Figure 3] Diagram showing an example of the information management table of the information processing apparatus [Figure 4] Diagram showing an example of the access right information management table of the information processing apparatus (FIG. 4(a) and FIG. 4(b)) [Figure 5] Diagram showing an example of the face recognition information management table of the information processing apparatus [Figure 6] Diagram showing an example of the display of the information processing apparatus (FIG. 6(a), FIG. 6(b) and FIG. 6(c)) [Figure 7] Diagram showing an example of the image acquired by the image acquisition unit of the information processing apparatus (FIG. 7(a), FIG. 7(b) and FIG. 7(c)) [Figure 8] Diagram showing an example of the image acquired by the image acquisition unit of the information processing apparatus (FIG. 8(a) and FIG. 8(b)) [Figure 9] Diagram showing an example of the display of the information processing apparatus (FIG. 9(a), FIG. 9(b) and FIG. 9(c)) [Figure 10] Diagram showing an example of the appearance of the computer system in the embodiment of the present invention [Figure 11] Diagram showing an example of the configuration of the same computer system

Embodiments for Carrying Out the Invention

[0024] Hereinafter, embodiments of an information processing apparatus and the like will be described with reference to the drawings. In the embodiments, components denoted by the same reference numerals perform the same operations, and thus repeated descriptions may be omitted.

[0025] (Embodiment) Figure 1 is a block diagram of the information processing device 1 in this embodiment.

[0026] The information processing device 1 includes an information storage unit 100, a screen output unit 101, a camera 102, an image acquisition unit 103, an access rights information storage unit 104, a face recognition information storage unit 105, a decision unit 106, and a security processing unit 107.

[0027] The information storage unit 100 stores information that can be displayed on the screen. The information stored in the information storage unit 100 is, for example, a file or data. The information that can be displayed on the screen is, for example, image data, text data, a combination of image data and text data, or a file containing such data. The image data may be still image data or moving image data. The information that can be displayed on the screen may also be, for example, data such as an executable file or program that produces screen output when executed. The information that can be displayed on the screen may also be, for example, 3D data that can be displayed on the screen by rendering or the like. However, the information that can be displayed on the screen is not limited to the above-mentioned information. The storage here may be considered to include temporary storage of the information that can be displayed, transmitted from a server device (not shown). The information storage unit 100 stores, for example, files or data, associated with an information identifier such as a file name or data name.

[0028] The information storage unit 100 is preferably a non-volatile recording medium, but it can also be implemented with a volatile recording medium. The same applies to other storage units such as the access rights information storage unit 104 and the face recognition information storage unit 105.

[0029] The screen output unit 101 outputs information to the screen. For example, the screen output unit 101 outputs information stored in the information storage unit 100 to the screen. The screen output unit 101 outputs, for example, a file or data. The screen output unit 101 outputs information to, for example, a screen such as a monitor (not shown) of the information processing device 1. The screen output unit 101 may output multiple pieces of information to the screen simultaneously. The screen output unit 101 may also output information to multiple screens.

[0030] The output here is usually displayed on a monitor, but it may also be projected onto a projector, transmitted to an external device, or transferred to other processing devices or programs.

[0031] The screen output unit 101 may or may not be considered to include an output device such as a monitor. The screen output unit 101 can be implemented using driver software for the output device, or using driver software for the output device and the output device itself.

[0032] Camera 102 captures video and acquires images. For example, camera 102 captures video and acquires images during the period when information is output to the screen. The information here may be one or more specific pieces of information, or one or more unspecified pieces of information. The images acquired by camera 102 may be still images or moving images. For example, while the screen output unit 101 is outputting a screen, camera 102 may repeatedly take pictures at fixed or indefinite intervals to acquire multiple still images, or it may repeatedly take pictures again each time one takes a picture, thereby acquiring multiple still images.

[0033] Camera 102, for example, captures an area opposite to the screen output by the screen output unit 101. For example, camera 102 captures one or more users who are within an area where they can view the screen output by the screen output unit 101. Camera 102 is usually a camera that uses ambient light to capture images, but camera 102 may be a combination of a camera that uses ambient light to capture images and a camera that captures light other than ambient light (for example, a camera that emits infrared light and captures the reflected light).

[0034] The image acquisition unit 103 acquires images taken while information is being output to the screen. For example, the image acquisition unit 103 acquires images taken during the period in which information is being output to the screen. The images acquired by the image acquisition unit 103 may be still images or moving images. For example, the image acquisition unit 103 acquires multiple still images that are repeatedly taken while the screen output unit 101 is outputting the screen. The images acquired by the image acquisition unit 103 are, for example, images of the area opposite to the screen output by the screen output unit 101.

[0035] The image acquisition unit 103 acquires images captured by, for example, the camera 102. In this case, image acquisition may involve receiving and reading images captured by the camera 102. For example, the image acquisition unit 103 may acquire images captured by the camera 102 via a storage medium such as memory (not shown). If the camera 102 is a combination of a camera that takes pictures using ambient light and a camera that takes pictures of the reflected light after irradiating with infrared light, the image acquisition unit 103 may acquire images captured by both cameras.

[0036] In this embodiment, the following description will focus on the case where the information processing device 1 has a camera 102 and the image acquisition unit 103 acquires an image captured by the camera 102 when information is being output to the screen. However, the information processing device 1 does not necessarily have a camera 102. In this case, for example, the image acquisition unit 103 may acquire an image captured by an external camera (for example, an external camera) of the information processing device 1.

[0037] The access rights information storage unit 104 stores one or more access rights information. Access rights information is information that identifies users who have the right to access information such as files or data stored in the information storage unit 100 or a storage unit (not shown), etc. The right to access files or data can be considered as an access right. Files and data that a user has the right to access can be considered as files and data that the user can access. Access rights information is information that associates, for example, an identifier for information such as files or data with a user identifier of a user who has the right to access the information such as files or data identified by this identifier. The identifier for files or data is a file identifier such as a file name or a code assigned to a file, or a data identifier such as a data name or a code assigned to data. The user identifier is a username, a user account, a user's email address, a user's employee number, a code assigned to a user, etc.

[0038] Furthermore, access rights information may include information specifying the directory where accessible files or data are stored (for example, the path to the folder where accessible files or data are stored) instead of a file or data identifier, or information indicating attributes such as the file extension or type of file or data (for example, image data, text data, data in a specific file format). In this case, files or data within the directory specified by one access rights piece of information, or files or data with the specified extension or type, become files or data accessible to the user identified by the user identifier of that access rights piece of information.

[0039] In this embodiment, the case in which the information processing device 1 has an access rights information storage unit 104 is described. However, the server device or the like (not shown) that the information processing device 1 can access via a network or the like may have an access rights information storage unit 104, and the information processing device 1 may not have an access rights information storage unit 104.

[0040] The face recognition information storage unit 105 stores face recognition information. Face recognition information is information used to recognize the faces of one or more users. Face recognition information is, for example, one or more pieces of information having a user's face image and the corresponding user identifier of the user. Alternatively, face recognition information may be, for example, one or more pieces of information having a user's facial feature quantity and the corresponding user identifier of the user. Furthermore, face recognition information may be information such as a learner acquired by machine learning or the like using one or more pairs of the user's face image or facial feature quantity information and the corresponding user identifier of the user.

[0041] The face recognition information storage unit 105 may store, for example, face recognition information for recognizing the faces of one or more users that have been registered in advance. Alternatively, the face recognition information storage unit 105 may store, for example, face recognition information used to recognize the faces of users who have the right to access one or more files or data. Note that the face recognition information stored in the face recognition information storage unit 105 is not limited to face recognition information for recognizing the faces of users as described above.

[0042] In this embodiment, the case in which the information processing device 1 has a face recognition information storage unit 105 is described. However, the information processing device 1 may also choose to have a server device (not shown) that is accessible to the information processing device 1 via a network, etc., which has the face recognition information storage unit 105, and the information processing device 1 may choose not to have the face recognition information storage unit 105.

[0043] Furthermore, if the determination unit 106, etc., does not use access rights information, the access rights information storage unit 104 may be omitted. Also, if the determination unit 106, etc., does not use face recognition information, the face recognition information storage unit 105 may be omitted.

[0044] The determination unit 106 determines whether the image acquired by the image acquisition unit 103 satisfies the security conditions. The security conditions are, for example, conditions used to determine whether processing to maintain security is necessary. Processing to maintain security refers to, for example, security processing performed by the security processing unit 107, which will be described later. A user who satisfies the security conditions is, for example, a user for whom processing to maintain security is necessary.

[0045] The following describes the processing performed by the determination unit 106, using examples of each of the four security conditions.

[0046] 1) When the security requirement is that the image contains multiple people (hereinafter referred to as the first security requirement) The determination unit 106 determines that the security condition is met if the image acquired by the image acquisition unit 103 contains multiple faces. Conversely, it determines that the security condition is not met if the image does not contain multiple faces. For example, the determination unit 106 performs a face detection process to detect faces in the image acquired by the image acquisition unit 103, and determines that the security condition is met if the number of detected faces is two or more, as multiple faces are present. The face detection process is, for example, a process to detect the smallest rectangular region containing a face in the image. Since the process of detecting faces in an image is publicly known technology, a detailed explanation is omitted. The information necessary for detecting faces may be stored in advance in a storage unit (not shown), for example, and read and used by the determination unit 106 as appropriate.

[0047] 2) When the security condition is that the image contains a face that is not registered (hereinafter, this security condition will be referred to as the second security condition). The determination unit 106 determines that the security condition is met if the image acquired by the image acquisition unit 103 contains faces that are not registered. Conversely, if the image does not contain faces that are not registered, the determination unit 106 determines that the security condition is not met. Faces that are not registered are, for example, faces that cannot be recognized using the pre-registered face recognition information stored in the face recognition information storage unit 105. In this case, the face recognition information stored in the face recognition information storage unit 105 is, for example, face recognition information for recognizing the faces of one or more pre-registered users, or face recognition information used to recognize the faces of users who have the right to access information such as one or more files or data. For example, the determination unit 106 performs a process to detect faces in the image acquired by the image acquisition unit 103, performs face recognition processing on all detected faces using the face recognition information stored in the face recognition information storage unit 105, and determines that the security condition is met if at least some of the detected faces cannot be recognized. Conversely, for example, if all of the detected faces can be recognized, the determination unit 106 determines that the security condition is not met. The face recognition processing here is, for example, the process of recognizing a face as the face of a specific user. The process of recognizing a face can be thought of as the process of recognizing an image that matches a specific face. Here, "matching" can be defined as, for example, a similarity score being above or above a threshold. The process of recognizing a face image can be achieved, for example, by performing image recognition processing such as feature-based matching or region-based matching using face recognition information such as face images or face features. The process of recognizing a face image may also be performed using face recognition information such as a learner obtained through machine learning. Furthermore, the process of recognizing a face is not limited to the above processes.

[0048] 3) If the security condition is that the files or data displayed on the screen include faces that do not have access rights to (hereinafter, this security condition will be referred to as the third security condition). The determination unit 106 determines that the security condition is met if the image acquired by the image acquisition unit 103 contains faces that do not have access rights to the files or data output to the screen by the screen output unit 101. Conversely, if no faces without access rights are included, the determination unit 106 determines that the security condition is not met. For example, the determination unit 106 detects faces from the image acquired by the image acquisition unit 103, performs face recognition on all detected faces using face recognition information stored in the face recognition information storage unit 105, and determines whether the detected faces are the faces of users who have the right to access the files or data output to the screen, as identified by the access rights information stored in the access rights information storage unit 104. The determination unit 106 determines that the security condition is met if at least some of the detected faces are not the faces of users who have the right to access the files or data output to the screen. Conversely, if all of the detected faces are the faces of users who have the right to access the files or data output to the screen, the determination unit 106 determines that the security condition is not met.

[0049] For example, the determination unit 106 performs the above determination process as follows: First, the determination unit 106 detects faces from the image acquired by the image acquisition unit 103 and performs face recognition on all detected faces using the face recognition information stored in the face recognition information storage unit 105. If one or more faces that cannot be recognized are detected, the determination unit 106 determines that the security conditions are met. If no faces that cannot be recognized are detected, the determination unit 106 uses the access rights information stored in the access rights information storage unit 104 to determine whether the users recognized by face recognition are users who have the right to access the files or data displayed on the screen. If at least some of the users recognized by face recognition are users who do not have the right to access the files or data displayed on the screen, the determination unit 106 determines that the security conditions are met. If all users recognized by face recognition are users who have the right to access the files or data displayed on the screen, the determination unit 106 determines that the security conditions are not met. For example, if no faces that cannot be recognized are detected, the determination unit 106 obtains the user identifiers of all users recognized by face recognition as a result of the face recognition process. If the access rights information corresponding to all obtained user identifiers indicates that the user has the right to access the files or data displayed on the screen, the determination unit 106 determines that the security conditions are not met. If the access rights information corresponding to at least some of the user identifiers indicates that the user does not have the right to access the files or data displayed on the screen, the determination unit 106 determines that the security conditions are met. The face recognition information used in this case may be a face image or information on facial features, or it may be information such as a learning model for face recognition obtained by machine learning using this information.

[0050] Alternatively, for example, the determination unit 106 may perform the above determination process as follows. First, the determination unit 106 obtains user identifiers of all users who have access rights to the files or data output to the screen by the screen output unit 101 from the access rights information stored in the access rights information storage unit 104. Next, it obtains face recognition information corresponding to the obtained user identifiers. The face recognition information to be obtained is, for example, a face image or face features. Then, the determination unit 106 detects faces from the image acquired by the image acquisition unit 103 and performs face recognition processing on all detected faces using the face recognition information corresponding to the user identifiers of all users with access rights obtained above. This face recognition processing is the process of recognizing the face corresponding to the obtained face recognition information among the detected faces. If all detected faces are recognized by the face recognition processing, the determination unit 106 determines that the security conditions are not met, and if at least one face is not recognized, the determination unit 106 determines that the security conditions are met.

[0051] The determination unit 106 may perform the above determination process by processing procedures other than those described above.

[0052] Furthermore, although this explanation describes the case where face recognition processing is performed on an image of a face detected by face detection, the determination unit 106 may perform face recognition without first performing face detection. For example, the determination unit 106 may perform face recognition by detecting from the image regions that match the image of a face indicated by face recognition information, or regions of the image where the feature points of face recognition information match.

[0053] 4) When the security condition is to recognize a predetermined movement (hereinafter, this security condition will be referred to as the fourth security condition.) The determination unit 106 detects predetermined movements in the image acquired by the image acquisition unit 103, and determines that the security conditions are met if the predetermined movements are detected. If the predetermined movements are not detected, it determines that the security conditions are not met. The images used here are preferably moving images, but may also be multiple still images taken in succession. The predetermined movements are, for example, actions taken by a user in front of the screen output by the screen output unit 101 to hide or close the screen output by the screen output unit 101. For example, the user may hide the screen displayed by the screen output unit 101 with one or both hands, or they may swing one hand from right to left or left to right relative to the screen to close it. Alternatively, the predetermined movements may be actions taken by a user in front of the screen output by the screen output unit 101 to move away from the screen (for example, standing up). Actions taken to move away from the screen include, for example, a face moving away from the screen and becoming invisible. Furthermore, the predetermined movements may include movements in which a user other than the user in front of the screen enters the screen. For example, a user entering the screen may be a movement in which a person approaches the screen or a movement in which a person enters the frame. Also, if a person is detected within the screen, it may be considered that a person has entered. The determination unit 106 may detect one movement, or two or more. Furthermore, if the determination unit 106 performs face detection and fails to detect one or more faces, it may determine that it has recognized a movement in which a user has left their seat. The movements that the determination unit 106 can detect are not limited to the above movements, as long as they are movements that are considered to require security processing. The determination unit 106 may also detect predetermined movements using a learner acquired using machine learning stored in a storage unit (not shown). Since the process of detecting predetermined movements within an image is publicly known, a detailed explanation is omitted.

[0054] The determination unit 106 may determine whether the security condition is met with respect to only one of the first to fourth security conditions described above for the image acquired by the image acquisition unit 103. Alternatively, the determination unit 106 may determine whether two or more of the first to fourth security conditions are met for the image acquired by the image acquisition unit 103. In this case, for example, the determination unit 106 may determine that the security condition is met if one or more of the first to fourth security conditions are met.

[0055] The security processing unit 107 performs security processing to change the screen output by the screen output unit 101 when the determination unit 106 determines that the image meets the security conditions. The security processing to change the screen is, for example, a process to hide the information output on the screen. The security processing to change the screen performed by the security processing unit 107 may be, for example, a process to change the screen output by outputting an instruction or control signal to the screen output unit 101, etc., or the security processing unit 107 itself may change the screen being output.

[0056] The process of hiding information displayed on the screen may also be the process of closing files or data displayed on the screen. The process of closing files or data may or may not include closing the window in which the file or data is displayed, or minimizing the window size.

[0057] Furthermore, the process of hiding information displayed on the screen may involve hiding only some of the information among the two or more pieces of information being displayed. This "some" information may include, for example, an image or text from the displayed information. It may also include information that has been assigned predetermined tags or other identifiers.

[0058] Furthermore, the process of hiding the information displayed on the screen may also be the process of locking the screen that displays the information. A lock screen is, for example, a screen in which predetermined operations are restricted.

[0059] Furthermore, when the determination unit 106 determines that different security conditions are met, the security processing unit 107 may perform different security processing for each condition, or it may perform the same security processing for each condition.

[0060] Next, an example of the operation of the information processing device 1 will be explained using the flowchart in Figure 2.

[0061] (Step S101) The information processing device 1 determines whether to output one or more pieces of information (for example, a file or data) stored in the information storage unit 100. For example, it determines whether a receiving unit (not shown) of the information processing device 1 has received an operation to output one or more pieces of information stored in the information storage unit 100 (for example, an operation to display). If an operation has been received, it determines to output the information; otherwise, it determines not to output the information. If the information is to be output, the process proceeds to step S102; otherwise, the process returns to step S101.

[0062] (Step S102) The screen output unit 101 reads information corresponding to the operation received in step S101 from the information storage unit 100 and outputs it. For example, the screen output unit 101 configures a screen to display the read information and displays the configured screen on a monitor or the like. The monitor used for display may be an external monitor or the like.

[0063] (Step S103) The image acquisition unit 103 acquires an image. For example, the image acquisition unit 103 takes a picture using the camera 102 and acquires the captured image from the camera 102. The image acquisition unit 103 acquires an image of the area facing the screen on which the screen output unit 101 outputs information.

[0064] (Step S104) The determination unit 106 performs a process to detect faces in the image acquired in step S103.

[0065] (Step S105) The determination unit 106 determines whether one or more faces were detected in step S104. If no faces were detected, the process proceeds to step S106; if faces were detected, the process proceeds to step S108.

[0066] (Step S106) The determination unit 106 determines that it has detected one of the predetermined actions, namely the user leaving their seat, and determines that the fourth security condition is met.

[0067] (Step S107) The security processing unit 107 performs security processing. For example, it performs processing to hide the screen of the file or data output by the screen output unit 101. Then, it returns to step S103.

[0068] (Step S108) The determination unit 106 counts the number of faces detected in step S104 and determines whether two or more faces were detected. If two or more faces were detected, the process proceeds to step S109; otherwise, the process proceeds to step S110.

[0069] (Step S109) The determination unit 106 determines that the first security condition is met because multiple faces have been detected. Then, it returns to step S107.

[0070] (Step S110) The determination unit 106 performs face recognition processing on the face detected in step S104 using the face recognition information stored in the face recognition information storage unit 105.

[0071] (Step S111) The determination unit 106 determines whether a face was recognized in step S110. If it was not recognized, the process proceeds to step S112. If it was recognized, it obtains information (for example, a user identifier) ​​that identifies the user corresponding to the recognized face, and proceeds to step S113.

[0072] (Step S112) The determination unit 106 determines that the second security condition is met because no pre-registered faces that can be recognized using the facial recognition information are detected. Then, the process returns to step S107.

[0073] (Step S113) The determination unit 106 identifies the file or data being output by the screen output unit 101. For example, the determination unit 106 obtains the identifier of the file or data being output by the screen output unit 101 (e.g., file name, data name, etc.).

[0074] (Step S114) The determination unit 106 uses the access rights information stored in the access rights information storage unit 104 to identify users who have the right to access the file or data identified in step S113. For example, the determination unit 106 obtains one or more user identifiers from the access rights information that are associated with the file or data identifier obtained in step S113.

[0075] (Step S115) The determination unit 106 determines whether the user whose face was recognized in step S110 is one of the users identified in step S114 (i.e., one of the identified users). For example, the determination unit 106 detects a user identifier that matches the user identifier obtained in step S111 among the one or more user identifiers obtained in step S114. If a matching user identifier is detected, the determination unit 106 determines that the user is one of the users identified in step S114; otherwise, the determination unit 106 determines that the user is not one of the identified users. If the user is not one of the identified users, the process proceeds to step S116; if the user is one of the identified users, the process proceeds to step S117.

[0076] (Step S116) The determination unit 106 determines that the third security condition is met because the image acquired by the image acquisition unit 103 contains a face that does not have access rights to the file or data output on the screen. Then, the process returns to step S107.

[0077] (Step S117) The determination unit 106 performs a process to detect predetermined movements within the image acquired in step S103. For example, the determination unit 106 performs a process to detect movements such as a user closing the screen or a user other than the recognized user entering the frame of the screen.

[0078] (Step S118) The determination unit 107 determines whether it was able to detect one or more predetermined movements in step S117. If it was detected, the process proceeds to step S119; otherwise, the process proceeds to step S120. Note that if security processing is performed in step S107 to hide files or data, if no movement is detected in this step, the security processing unit 107 may release the security processing and display the files or data that were hidden.

[0079] (Step S119) The determination unit 107 determines that a predetermined action has been detected in the image acquired by the image acquisition unit 103, and that the fourth security condition is met. Then, the process returns to step S107.

[0080] (Step S120) The information processing device 1 determines whether the reception unit (not shown) or the like has received an operation to change the output of the screen output unit 101. An operation to change the output is, for example, an operation to input information such as a string on the screen to be output, or an operation to change the file or data to be output. The operation to change is not limited to these operations. If an operation to change is received, the device proceeds to step S121; otherwise, it returns to step S103.

[0081] (Step S120) The screen output unit 101 changes the output according to the operation received in step S120. Then, the process returns to step S103.

[0082] In the flowchart of Figure 1, the security processing unit 107 performs the same security processing in step S107 regardless of which of the first to fourth security conditions is met. However, the security processing unit 107 may perform different security processing depending on which of the first to fourth security conditions is met.

[0083] Furthermore, in step S107, if a lock screen is displayed as a security process, for example, if the lock screen is displayed immediately before step S120, the reception unit (not shown) may determine whether it has received an operation to unlock this lock screen, and only proceed to step S120 if it has.

[0084] Furthermore, in the flowchart of Figure 2, before outputting information in step S102, the same processing as in steps S103 to S119 may be performed to carry out security processing as necessary.

[0085] In the flowchart in Figure 2, processing is terminated by power-off or processing termination interrupts.

[0086] The specific operation of the information processing device 1 in this embodiment will be described below. Figure 3 is an information management table for managing files, which are information stored in the information storage unit 100. The information management table has an attribute called "file name". The "file name" is assumed to be the file name, which is the identifier of the file stored in the information storage unit 100.

[0087] Figure 4(a) is an access rights information management table that manages the access rights information stored in the access rights information storage unit. The access rights information management table has a pair of attributes: an attribute called "file name" similar to the "file name" in Figure 3, and an attribute called "user ID," which is the user identifier of the user who can access the file identified by the value of this "file name" attribute. Here, "user ID" is assumed to be a code uniquely assigned to the user.

[0088] Figure 5 shows a face recognition information management table that manages the face recognition information stored in the face recognition information storage unit 105. Face recognition information is assumed to be information about the feature points of the user's face. The face recognition information management table has attributes called "User ID," which is the user identifier of the user, and "Face Recognition File Name," which is the file name of the file containing this user's face recognition information.

[0089] First, suppose user A, whose user identifier is "U001", sits in front of camera 102 and uses an input device such as a keyboard or mouse (not shown) to open a file stored in the information storage unit 100 named "Company Information Session Applicant List" on the information processing device 1. When the reception unit (not shown) of the information processing device 1 receives this operation, the screen output unit 101 reads the file named "Company Information Session Applicant List" specified by this operation from the information storage unit 100, constructs a display screen using the read file, and displays this screen.

[0090] Figure 6(a) shows an example of the screen display of a file named "Company Information Session Applicant List" by the screen output unit 101. In this case, the screen output unit 101 is assumed to be displaying the above screen on the monitor 101a of the information processing device 1.

[0091] When the screen output unit 101 outputs a screen, the camera 102 captures the area opposite the monitor 101a. This captures the area including user A's face. The image acquisition unit 103 acquires the image captured by the camera 102.

[0092] Figure 7(a) shows an example of an image acquired by the image acquisition unit 103.

[0093] The determination unit 106 performs face detection on the image shown in Figure 7(a) acquired by the image acquisition unit 103. As a result of the face detection, the region 71 of user A's face is detected, as shown in Figure 7(b). Note that Figure 7(b) is a schematic diagram to explain the result of face detection, and as shown in the figure, it is not necessary to display the detected face region 71, etc., in the image. The same applies to the face region 72 in Figure 7(C), image 81 in Figure 8(a), image 82 in Figure 8(b), etc., which will be described later.

[0094] The determination unit 106 determines that, since it detected one or more faces, it did not detect one of the predetermined actions, namely that the user left their seat. Therefore, it determines that the fourth security condition, which is the detection of this predetermined action, has not been met.

[0095] Next, the determination unit 106 counts the number of faces detected by face detection and determines whether the number of detected faces is two or more. Here, since only one face was detected, it determines that no multiple faces were detected and that the first security condition of multiple faces being detected is not met.

[0096] Next, the determination unit 106 performs face recognition processing on the image of the face region 71 detected by face detection. Specifically, it performs face recognition processing using the information of the user's face feature points, which is face recognition information indicated by the attribute value of the "face recognition file name" that is managed in association with the user identifier in the face recognition information management table shown in Figure 5. Specifically, it detects a face recognition information file that has feature point information that matches the face image of region 71 from among the face recognition information files, which are information of the user's face feature points associated with the user identifier managed in the face recognition information management table shown in Figure 5, and obtains the "user ID", which is the user identifier associated with the detected file, from the face recognition information management table. Matching feature points may also mean that the similarity of the feature points is higher than a predetermined threshold, etc. Here, if it is determined that the feature points of the face recognition information with the file name "F10001" match, the determination unit 106 obtains the attribute value "U001" of the "User ID" of the record (row) with the "Face Recognition File Name" "F10001" in the face recognition information management table shown in Figure 5 as the result of face recognition. In other words, the determination unit 106 has recognized the face detected above as the face of user A, whose user identifier is "U001".

[0097] The determination unit 106 determines that the face was recognized as a specific face (in this case, a face with user identifier "U001"), and therefore the second security condition, that the face of a pre-registered user is not detected, is not met.

[0098] Next, the determination unit 106 obtains the file name "Company Presenter Applicant List" corresponding to the screen output by the screen output unit 101, searches the access rights information management table shown in Figure 4(a) for records (rows) where the "file name" matches the obtained file name "Company Presenter Applicant List", and obtains the user identifiers "U001" and "U002", which are the attribute values ​​of the "User ID" for all detected records. The obtained user identifiers are the user identifiers of one or more users who can access the file whose file name is "Company Presenter Applicant List". Then, the determination unit 106 determines whether there is a user identifier among the obtained user identifiers that matches "U001", which was obtained as a result of recognizing the face image in area 71 above. Here, since there is a matching user identifier, the determination unit 106 determines that the third security condition, that the image acquired by the image acquisition unit 103 does not contain a face that does not have access rights to the file output on the screen, is not met.

[0099] Furthermore, the determination unit 106 performs a process to detect predetermined movements. For example, first, it detects the movement of another user other than the recognized user A entering the image acquired by the image acquisition unit 103. In this case, it is determined that no movement of another user entering was detected. Next, it is determined whether the movement of user A covering the screen with their hand was detected. In this case, it is determined that this covering movement was also not detected. Therefore, the determination unit 106 determines that the fourth security condition, that predetermined movements are detected, is not met.

[0100] In this way, because the determination unit 106 has determined that the first to fourth security conditions are not met, user A can perform actions such as inputting new information on the screen output by the screen output unit 101 or closing the screen and opening a new file.

[0101] Here, if, for example, the image acquisition unit 103 performs face detection on the acquired image and no faces are detected, the judgment unit 106 determines that it has detected the user leaving their seat and that the fourth security condition is met. Then, the security processing unit 107 performs security processing. For example, the security processing unit 107 instructs the screen output unit 101 to hide the screen of the file being output (in this case, displayed), and in response to this instruction, the screen output unit 101 stops displaying the file whose filename is "Company Information Session Applicant List" as shown in Figure 6(a). As a result, as shown in Figure 6(b), the file whose filename is "Company Information Session Applicant List" that was displayed on the monitor 101a of the information processing device 1 is hidden.

[0102] The security processing unit 107 may, as part of its security processing, instruct the screen output unit 101 to display a lock screen, and the screen output unit 101 may, in response to this instruction, display a lock screen as shown in Figure 6(c).

[0103] Furthermore, suppose that the image acquired by the image acquisition unit 103, as shown in Figure 7(c), is an image in which not only User A's face but also faces other than User A are captured, and as a result of face detection by the determination unit 106, in addition to the face region 71 of User A, the region 72 of faces other than User A is also detected. In this case, the number of faces detected by the face detection counted by the determination unit 106 becomes 2, so the determination unit 106 determines that multiple faces have been detected and determines that the first security condition is met. Then, the security processing unit 107 performs the same security processing as above to hide the output file.

[0104] Furthermore, suppose that, as a result of face detection processing on the image acquired by the image acquisition unit 103, only one face is detected as shown in Figure 7(b), and as a result of face recognition processing by the determination unit 106 on this face, no face recognition information, which is information on feature points that match the face image in region 71, is detected, and face recognition fails. In this case, the determination unit 106 determines that no pre-registered faces were detected and determines that the second security condition is met, and the security processing unit 107 performs the same security processing as above to hide the output file.

[0105] Furthermore, let's assume that the access rights information management table, which manages the access rights information stored in the access rights information storage unit, is different from Figure 4(a), as shown in Figure 4(b), and does not have a record (row) that associates the file name "Company Presenter Applicant List" with the user identifier "U001". Then, let's assume that when the image acquisition unit 103 acquires an image, a face detection process is performed and as a result, one face is detected as shown in Figure 7(b). When the judgment unit 106 performs face recognition processing on this face, face recognition information, which is information about feature points that match the image of the detected face, is detected, and "U001" is acquired as the user identifier corresponding to this face recognition information. Then, the determination unit 106 obtains the file name "Company Presenter Applicant List" corresponding to the screen output by the screen output unit 101, searches the access rights information management table shown in Figure 4 for records (rows) where the "file name" matches the obtained file name "Company Presenter Applicant List", obtains the user identifiers "U002" and "U003", which are attribute values ​​of "User ID" for all detected records, and determines whether there is a user identifier among the obtained user identifiers that matches "U001", which is the user identifier obtained as a result of the face recognition performed above. In this case, since there is no user identifier among the obtained user identifiers that matches "U001", the determination unit 106 determines that the third security condition is met, which is that the image obtained by the image acquisition unit 103 contains a face that does not have access rights to the file output on the screen. Then, the security processing unit 107 performs the same security processing as above to hide the output file.

[0106] Furthermore, suppose the image acquired by the image acquisition unit 103 is an image like the one shown in Figure 8(a), and the judgment unit 106 detects an image 81 of another person behind the face-recognized user A in the image acquired by the image acquisition unit 103. In this case, the judgment unit 106 determines that it has detected a predetermined movement in the image, that another user has entered, and determines that the fourth security condition, that a predetermined action has been detected, has been met. Then, the security processing unit 107 performs the same security processing as described above to hide the output file.

[0107] Furthermore, suppose the image acquired by the image acquisition unit 103 is an image like the one shown in Figure 8(b), and the judgment unit 106 detects an image 82 in which user A's left and right palms cover the screen. Therefore, the judgment unit 106 determines that it has detected a predetermined movement that covers the screen in the image, and determines that the fourth security condition, that a predetermined action has been detected, has been met. Then, the security processing unit 107 performs the same security processing as described above to hide the output file.

[0108] Furthermore, even after the image acquisition unit 103 has performed processing such as determining whether the series of security conditions described above are met for the image it has acquired, it is preferable that the image acquisition unit 103 repeatedly performs the process of acquiring a new image using the camera 102, the determination unit 106 repeatedly performs the process of determining whether the acquired image meets the first to fourth security conditions described above, and the security processing unit 107 repeatedly performs security processing to hide the output file if any of the first to fourth security conditions are met.

[0109] Furthermore, if the screen displayed by the screen output unit 101 contains information including both an image and text, as shown in Figure 9(a), the security processing unit 107 may, as a security measure, hide only the image on the output screen, as shown in Figure 9(b), or hide only the text on the output screen, as shown in Figure 9(c).

[0110] As described above, according to this embodiment, security can be easily ensured for the output screen, which is then hidden when security conditions are met, and security processing is performed to hide the output screen.

[0111] In the above embodiment, instead of having access rights information in the access rights information storage unit 104 that associates a file or data identifier with a user identifier of a user who has the right to access the file or data identified by this identifier, the storage unit 104 may store level management information that associates a user identifier with information indicating the security level set for the user, and access rights information that associates information indicating the security level with a file or data identifier that can be accessed by a user with the security level indicated by this information. The determination unit 106 may then use this level management information and access rights information to obtain a user identifier corresponding to the file or data identifier, thereby enabling it to identify a user who has the right to access the file or data output by the screen output unit 101.

[0112] In each of the above embodiments, each process (each function) may be implemented by centralized processing by a single device (system), or by distributed processing by multiple devices.

[0113] Furthermore, although the above embodiments describe the case where the information processing device is standalone, the information processing device may be a standalone device or a server device in a server-client system. In the latter case, the output unit and reception unit will receive input and output a screen via a communication line.

[0114] Furthermore, in each of the above embodiments, each component may be configured with dedicated hardware, or, if it is a component that can be implemented by software, it may be implemented by executing a program. For example, each component can be implemented by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. During execution, the program execution unit may execute the program while accessing the storage unit (for example, a recording medium such as a hard disk or memory).

[0115] The software that implements the information processing device in each of the above embodiments is as follows. In other words, this program causes the computer to function as a screen output unit that outputs information to the screen, an image acquisition unit that acquires an image taken when the information is being output to the screen, a determination unit that determines whether the image acquired by the image acquisition unit satisfies security conditions, and a security processing unit that performs security processing to change the screen being output by the screen output unit if the determination unit determines that the image satisfies the security conditions.

[0116] Furthermore, the functions implemented by the above program do not include any functions that can only be implemented by hardware. For example, functions that can only be implemented by hardware such as modems or interface cards in the information acquisition unit or the information output unit are not included in the functions implemented by the above program.

[0117] Furthermore, the computer running this program may be a single computer or multiple computers. In other words, it may perform centralized processing or distributed processing.

[0118] Figure 10 is a schematic diagram showing an example of the appearance of a computer that implements the information processing device according to the above embodiment by executing the above program. The above embodiment can be implemented by computer hardware and a computer program executed on it.

[0119] In Figure 10, the computer system 900 includes a computer 901 with a CD-ROM (Compact Disk Read Only Memory) drive 905, a keyboard 902, a mouse 903, and a monitor 904.

[0120] Figure 11 shows the internal configuration of the computer system 900. In Figure 11, the computer 901 includes, in addition to the CD-ROM drive 905, an MPU (Micro Processing Unit) 911, a ROM 912 for storing programs such as boot-up programs, a RAM (Random Access Memory) 913 connected to the MPU 911 for temporarily storing instructions for application programs and providing temporary storage space, a hard disk 914 for storing application programs, system programs, and data, and a bus 915 that interconnects the MPU 911, ROM 912, etc. The computer 901 may also include a network card (not shown) that provides connectivity to a LAN.

[0121] The program that causes the computer system 900 to execute the functions of the information processing device, etc., according to the above embodiment may be stored on a CD-ROM 921, inserted into a CD-ROM drive 905, and transferred to the hard disk 914. Alternatively, the program may be transmitted to a computer 901 via a network (not shown) and stored on the hard disk 914. The program is loaded into RAM 913 when executed. The program may also be loaded directly from the CD-ROM 921 or the network.

[0122] The program does not necessarily include an operating system (OS) or third-party program that causes the computer 901 to execute the functions of the information processing device according to the above embodiment. The program may include only the instruction portion that calls appropriate functions (modules) in a controlled manner to obtain the desired result. How the computer system 900 operates is well known, so a detailed explanation is omitted.

[0123] It goes without saying that the present invention is not limited to the embodiments described above, and various modifications are possible, all of which are also included within the scope of the present invention. [Industrial applicability]

[0124] As described above, the information processing device, etc., according to the present invention is suitable as a device, etc., for outputting information, and is particularly useful as a device, etc., for outputting information for which security must be ensured. [Explanation of symbols]

[0125] 1. Information Processing Device 100 Information Storage Unit 101 Screen output section 102 Cameras 103 Image acquisition unit 104 Access rights information storage unit 105 Face Recognition Information Storage Unit 106 Judgment Department 107 Security Processing Unit

Claims

1. A screen output unit that outputs information to the screen, An image acquisition unit that acquires a video image captured when the aforementioned information is displayed on the screen, The image acquisition unit acquires the video image, and using a learning device that has learned the movement of a user in front of the screen moving away from the screen, the unit detects the movement of a user in front of the screen moving away from the screen, and when such movement is detected, the unit determines that the video image satisfies a security condition, which is a condition used to determine that processing is necessary to maintain security. An information processing apparatus comprising: a security processing unit that performs security processing to change the screen output by the screen output unit when the determination unit determines that the video image satisfies the security conditions.

2. A screen output unit that outputs information to the screen, An image acquisition unit that acquires a video image captured when the aforementioned information is displayed on the screen, In the video image acquired by the image acquisition unit, a learning device that has learned the movement of a user in front of the screen covering the screen is used to detect such movement, and when such movement is detected, a determination unit determines that the video image satisfies a security condition, which is a condition used to determine that processing is necessary to maintain security. An information processing apparatus comprising: a security processing unit that performs security processing to change the screen output by the screen output unit when the determination unit determines that the video image satisfies the security conditions.

3. The information processing apparatus according to claim 1 or claim 2, wherein the security processing is a process of hiding the output information.

4. The information processing apparatus according to claim 3, wherein the security processing is a process of hiding only some of the information among two or more pieces of information that are output, or a process of locking the screen on which the information is output.

5. An information processing method performed using an information processing device comprising a screen output unit, an image acquisition unit, a determination unit, and a security processing unit, The aforementioned screen output unit performs a screen output step in which it outputs information to the screen, The image acquisition step involves the image acquisition unit acquiring a video image captured while the information is being output to the screen, The determination unit, using a learning device that has learned the movement of a user in front of the screen moving away from the screen in the video image acquired in the image acquisition step, detects the movement of a user in front of the screen moving away from the screen, and if such movement is detected, determines that the video image satisfies a security condition, which is a condition used to determine that processing is required to maintain security. An information processing method comprising: a security processing step in which the security processing unit performs security processing to change the screen output in the screen output step when the determination unit determines that the moving image satisfies the security conditions.

6. An information processing method performed using an information processing device comprising a screen output unit, an image acquisition unit, a determination unit, and a security processing unit, The aforementioned screen output unit performs a screen output step in which it outputs information to the screen, The image acquisition step involves the image acquisition unit acquiring a video image captured while the information is being output to the screen, The determination unit, using a learner that has learned the movement of a user in front of the screen to cover the screen in the video image acquired in the image acquisition step, detects the movement of a user in front of the screen to cover the screen, and if such movement is detected, determines that the video image satisfies a security condition, which is a condition used to determine that processing is required to maintain security. An information processing method comprising: a security processing step in which the security processing unit performs security processing to change the screen output in the screen output step when the determination unit determines that the moving image satisfies the security conditions.

7. Computers, A screen output unit that outputs information to the screen, An image acquisition unit that acquires a video image captured when the aforementioned information is displayed on the screen, The image acquisition unit acquires the video image, and using a learning device that has learned the movement of a user in front of the screen moving away from the screen, the unit detects the movement of a user in front of the screen moving away from the screen, and when such movement is detected, the unit determines that the video image satisfies a security condition, which is a condition used to determine that processing is necessary to maintain security. A program that functions as a security processing unit, which performs security processing to change the screen output by the screen output unit when the determination unit determines that the aforementioned video image satisfies the security conditions.

8. Computers, A screen output unit that outputs information to the screen, An image acquisition unit that acquires a video image captured when the aforementioned information is displayed on the screen, In the video image acquired by the image acquisition unit, a learning device that has learned the movement of a user in front of the screen covering the screen is used to detect such movement, and when such movement is detected, a determination unit determines that the video image satisfies a security condition, which is a condition used to determine that processing is necessary to maintain security. A program that functions as a security processing unit, which performs security processing to change the screen output by the screen output unit when the determination unit determines that the aforementioned video image satisfies the security conditions.

Images