Information processing device, information processing method, and information processing program

Abstract

To provide an information processing device, an information processing method, and an information processing program capable of preventing unauthorized photography by a third party, about data displayed on a user's terminal screen.SOLUTION: An information processing device 10 comprises: a camera information acquisition unit 13 which acquires an image in front of a screen of a user terminal, photographed through a camera of the user terminal 20; a face authentication unit 14 which performs face authentication of a person on the basis of face information of a user, when the person is captured in the acquired image; an outer shape extraction unit 15 which determines whether or not the outer shape of an imaging device is extracted near the person whose face is not successfully authenticated, in the image; a posture determination unit 16 which determines a direction of a body of the person in the captured image; a fraud determination unit 17 which detects fraud when the outer shape of the imaging device is extracted near the person whose face is not successfully authenticated, and the direction of the body of the person is the direction of the screen of the user terminal; and a display control unit 18 which controls data on the screen of the user terminal, when the fraud is detected.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] An embodiment of the present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Teleworking, in which company work is done from a remote location away from the company's base, is becoming more common. With teleworking, work can be done from anywhere via a network, so it is possible to work not only at home but also in public spaces.

[0003] While teleworking allows flexibility in choosing where to carry out work, if work is carried out in a public space or other place where third parties are present, there is a risk that third parties may secretly view the PC screen while work is being carried out, especially if the PC screen is illegally photographed using a mobile device such as a smartphone or tablet, resulting in the theft of work information, etc.

[0004] Conventionally, technology has been disclosed that detects an incident when a third party other than the person performing the work is captured on camera during teleworking. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2015-222555 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, in a method that detects an incident when a third party other than the person performing the work is captured on camera, for example in a public space, there is always a third party behind the person performing the work, so even if a third party is simply captured on camera, it may be judged to be an incident of fraud, and there is a risk of detecting an incident even when there is no risk of fraud.

[0007] The present invention has been made in consideration of the above circumstances, and aims to provide an information processing device, an information processing method, and an information processing program that can prevent third parties from illegally capturing data displayed on a user's terminal screen. [Means for solving the problem]

[0008] An information processing device according to an embodiment of the present invention is characterized in that it includes a camera information acquisition unit that acquires an image of the front of the screen of the user terminal captured via the camera of the user terminal, a face authentication unit that performs face authentication of a person based on the user's face information when a person appears in the acquired image, an outline extraction unit that determines whether the outline of an imaging device is extracted in the vicinity of a person whose face authentication is not successful in the image, a fraud determination unit that detects fraud when the outline of an imaging device is extracted in the vicinity of a person whose face authentication is not successful in the image, and a display control unit that controls data on the screen of the user terminal when fraud is detected. Effect of the Invention

[0009] According to an embodiment of the present invention, an information processing device, an information processing method, and an information processing program are provided that can prevent a third party from illegally capturing an image of data displayed on a user's terminal screen. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an information processing device according to an embodiment of the present invention. [Diagram 2] 1 is a schematic diagram showing a method for detecting unauthorized photography by a third party according to the present embodiment; [Diagram 3] (A) An explanatory diagram showing an example of how an image captured by the camera of a user terminal is judged; (B) An explanatory diagram showing another example of how an image captured by the camera of a user terminal is judged; (C) A diagram showing another example of how an image captured by the camera of a user terminal is judged to have been taken fraudulently by a third party. [Figure 4] 4 is a flowchart showing an example of an information processing method according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a diagram showing an example of the configuration of an information processing device 10 according to the present embodiment. The information processing device 10 is a server that performs face authentication of a person in an image captured by the camera 21 of the user terminal 20, determines whether the outline of the imaging device is extracted near a person for whom face authentication is unsuccessful, determines the body orientation of the person, and, if it detects fraud by a third party, controls the data on the screen of the user terminal 20. The data refers to various types of data stored in the information processing device 10, including files, image data, video data, and the like.

[0012] The user terminal 20 refers to a terminal such as a PC, a smartphone, or a tablet terminal used by an employee of a company, organization, or the like for business purposes, and is a terminal capable of communicating with the information processing device 10 via a network. The user terminal 20 may also be a headset terminal that is driven by being worn on the user's head.

[0013] The user terminal 20 includes a camera 21. The camera 21 is capable of constant communication with the user terminal 20, and is an imaging unit that captures an image in front of the display screen (display) of the terminal. camera21 may be built into the display screen of the user terminal 20, or may be arranged on the side of the display screen of the user terminal 20 and connected externally to the main body of the user terminal 20. When the user terminal 20 is a headset terminal, it is equipped with a camera that captures the surroundings of the user wearing the terminal, particularly the back of the user.

[0014] The images captured by the camera 21 include still images, moving images, etc. The acquired image data is transmitted to the information processing device 10.

[0015] In addition, in Figure 1, three user terminals 20 (20a, 20b, 20c) and three cameras 21 (21a, 21b, 21c) are shown equipped on each user terminal 20, but this configuration is not limited to this, and further multiple user terminals 20 and cameras 21 may be connected.

[0016] A specific configuration of the information processing device 10 according to this embodiment will be described. The information processing device 10 includes a data transmission / reception unit 11, a user information registration unit 12, a camera information acquisition unit 13, a face authentication unit 14, a contour extraction unit 15, a posture discrimination unit 16, a fraud determination unit 17, and a display control unit 18.

[0017] The functions of each unit constituting the information processing device 10 may be realized by executing a predetermined program code using a processor, and are not limited to such software processing, but may be realized, for example, by hardware processing using ASIC or the like, or may be realized by a combination of software processing and hardware processing. Also, each unit constituting the information processing device 10 may be realized by a separate server, and for example, data accessed from the user terminal 20 may be stored in a server separate from the main body of the information processing device 10.

[0018] The data transmitter / receiver 11 transmits data such as files and video data in response to an access request from the user terminal 20, and receives image data captured by the camera 21 of the user terminal 20, and the like.

[0019] The user information registration unit 12 registers authentication information for authenticating a user of the user terminal 20. A login ID, a password, and the like are registered in association with a user ID for identifying each user. Furthermore, face information of the user used for face authentication is registered in association with the user ID. When the user terminal 20 connects to the information processing device 10 and accesses data, the user terminal 20 may be authenticated based on the registered authentication information.

[0020] Also, users having access rights may be set in association with each piece of data. Note that a plurality of users may be set as a group, and access rights to data may be set for each group.

[0021] The camera information acquisition unit 13 acquires images captured by the camera 21 of the user terminal 20. When the user terminal 20 starts accessing data, the camera information acquisition unit 13 causes the camera 21 of the user terminal 20 to start capturing images, and causes the camera 21 to continue capturing images until the access of the user terminal 20 ends. The camera information acquisition unit 13 acquires images captured by the camera 21 at any time.

[0022] The face authentication unit 14 extracts a person's face from an image acquired by the camera 21, and performs face authentication by comparing the extracted face with the user's face information for authentication. When multiple people are extracted from an image, face authentication is performed for each person based on the user's face information. A person whose face is successfully authenticated in the image is a user who has access authority to data, while a person whose face is not successfully authenticated is a user who does not have access authority. When a person whose face is not successfully authenticated exists in the image, the face authentication unit 14 transmits an image including that person to the outline extraction unit 15.

[0023] The facial authentication method may be performed using facial information of the user that is registered in advance in association with the user, or may be performed using facial information for authentication that is stored in the user terminal 20.

[0024] In addition, when multiple people are to view the data, the multiple users who will view the data may be registered in advance, and facial authentication may be performed on the image captured by camera 21 based on the facial information of each registered user.

[0025] The contour extraction unit 15 determines whether the contour of an imaging device is present near a person whose face authentication is unsuccessful. An imaging device means an optical device for taking photos or videos, or a terminal having such a device built in, and includes cameras (including digital cameras), smartphones with built-in cameras, tablets, and other mobile terminals. Specifically, for an image containing a person whose face authentication is unsuccessful, the contour extraction unit 15 determines whether the contour of an imaging device is present within a predetermined distance (for example, within 30 cm) from the person.

[0026] Regarding the extraction of the external shape of the imaging device, the external shape (length, width, etc.) of the imaging device may be registered and the registered external shape may be compared with the image to determine whether or not the external shape of the imaging device is present in the image. Alternatively, the external shape of the imaging device may be machine-learned and the presence or absence of the external shape of the imaging device in the image may be determined based on the learned model. If the external shape of the imaging device is present in the vicinity of a person whose face is not successfully recognized in the image, the external shape extraction unit 15 transmits the image to the posture determination unit 16.

[0027] The posture discrimination unit 16 discriminates the body orientation of a person in a captured image. Specifically, when the outline of an imaging device is present in the vicinity of a person in an image whose face authentication is not successful, the posture discrimination unit 16 discriminates the body orientation of the person. The body orientation of a person means the direction of the front of the body (the direction of the normal to the front of the body). The posture discrimination unit 16 discriminates whether the front of the body faces the direction of the display screen of the user terminal 20. If the front of the body faces the direction of the display screen of the user terminal 20 at least once within a certain period of time (for example, within 10 seconds), it may be determined that the front of the body faces the direction of the display screen of the user terminal 20. The posture discrimination unit 16 may also measure the time that the person whose body orientation is to be discriminated stays in the same position.

[0028] The fraud determination unit 17 detects fraud when the outline of an imaging device is extracted near a person for whom face authentication is unsuccessful and the person's body is facing the screen of the user terminal 20. It determines that an imaging device held by a person who does not have access authority to data is facing the screen of the user terminal 20, and judges that there is a risk of fraudulent photography. When the outline of an imaging device is extracted near a person for whom face authentication is unsuccessful and it is determined that the person's body is facing the screen of the user terminal 20, the fraud determination unit 17 may determine that fraud has occurred if the person has been in the same position for a predetermined time or more (for example, one minute or more).

[0029] When the display control unit 18 determines that there is a risk of unauthorized shooting, it controls so that the data displayed on the user terminal 20 cannot be accessed. Specifically, when unauthorized shooting is determined while video data is being transmitted to the user terminal 20, the transmission of the video data is stopped, and the video data cannot be viewed. When unauthorized shooting is determined, the display control unit 18 may control so as to stop copying or expanding the display screen of the user terminal 20.

[0030] Also, when data is being displayed on the display screen of the user terminal 20, control may be performed so as to switch to a warning screen.

[0031] As a result, if a third party takes a picture of the screen of the user terminal 20 while the user of the user terminal 20 is displaying or playing data on the terminal screen, the data can be immediately made inaccessible, thereby preventing the fraudulent act from occurring. Also, a suspicious person behind the user of the user terminal 20 can be detected.

[0032] 2 is a schematic diagram showing a method for detecting unauthorized shooting by a third party according to the present embodiment (see FIG. 1 as appropriate). Here, a case will be considered in which, when a user A of a user terminal 20 is connected to an information processing device 10 and viewing video data, a third party X is attempting to shoot the video being played using an imaging device 50 (smartphone).

[0033] When playback of video data is started, camera information acquisition unit 13 acquires an image of the front of the display screen of user terminal 20 from camera 21. User A and third party X are captured in the camera image. At this time, face authentication is performed on the captured camera image using face information of user A. While face authentication is successful for user A in the image, authentication is not successful for third party X.

[0034] Then, it is determined whether the outline of the imaging device 50 is extracted near the third party X for whom face authentication is unsuccessful. Since the outline of the imaging device 50 is extracted near the third party X, the orientation of the body of the third party X is determined. If the orientation of the body of the third party X is toward the display screen of the user terminal 20, it is determined that unauthorized photography has been performed by the third party X.

[0035] If it is determined that the image has been taken illegally, the video data displayed on the user terminal 20 is controlled so as to be unviewable.

[0036] 3(A) to 3(C) are explanatory diagrams showing an example of judgment of an image captured by camera 21 of user terminal 20. Here, a case will be considered where user A of user terminal 20 is watching video data and a third party X is present behind him.

[0037] In the example of FIG. 3(A), face authentication is successful for user A of user terminal 20, while face authentication is unsuccessful for third party X. However, since the outline of the imaging device is not extracted in the vicinity of third party X, unauthorized photography is not determined.

[0038] In the example of Fig. 3(B), face authentication of user A of user terminal 20 is successful, while face authentication of third party X is not successful. Then, the outline of a smartphone is extracted in the vicinity of third party X. However, the orientation of the body of third party X is not toward the display screen of user terminal 20. In this case, it is not determined that unauthorized photography has been performed.

[0039] In Fig. 3(C), face authentication is successful for user A of user terminal 20, but face authentication is unsuccessful for third party X. The outline of a smartphone is extracted near third party X, and the body of third party X is facing the display screen of user terminal 20. For this reason, it is determined that there is a risk of unauthorized photography by a third party, and the data displayed on user terminal 20 is controlled so as not to be viewable.

[0040] In this way, it is possible to accurately determine the actions of a third party attempting to illegally photograph the screen of the user terminal 20, and to prevent such illegal photographing from occurring.

[0041] Next, the operation of the information processing device 10 according to the present embodiment will be described.

[0042] FIG. 4 is a flowchart showing an example of an information processing method according to the present embodiment (see FIG. 1 as appropriate).

[0043] Upon receiving a request from the user terminal 20, the data transmitter / receiver 11 transmits image data to the user terminal 20 (S10). The user terminal 20 plays back the video data (S11).

[0044] The camera information acquisition unit 13 acquires an image captured by the camera 21 of a user terminal 20 when the terminal accesses data (S12).

[0045] The face authentication unit 14 determines whether face authentication is successful for the person in the acquired image based on the face information of the user of the user terminal 20 (S13). If face authentication of the person in the image is successful, transmission of the video data continues (S13: YES).

[0046] When a person whose face is not successfully recognized is present in the image, the contour extraction unit 15 determines whether the contour of a smartphone is extracted near the person (S13: NO, S14). When the contour of a smartphone is not extracted near the person, the transmission of video data is continued (S14: NO).

[0047] When the outline of a smartphone is extracted near a person, the posture determination unit 16 determines whether the person's body is facing the display screen of the user terminal 20 (S14: YES, S15). If the person's body is not facing the display screen, the transmission of video data continues (S15: NO).

[0048] The fraud determination unit 17 determines that the person is fraudulent if the orientation of the person's body is toward the display screen of the user terminal 20. When fraud is determined, the display control unit 18 stops the playback of the video data being displayed on the user terminal (S15: YES, S16).

[0049] According to the information processing device of the embodiment described above, face authentication is performed on a person in an image captured by the camera of the user terminal, it is determined whether the outline of the imaging device is extracted near a person for whom face authentication is unsuccessful, the orientation of the person's body is determined, and if fraudulent activity by a third party is detected, the data on the screen of the user terminal 20 is controlled. As a result, if a third party commits fraudulent activity such as photographing the screen of the user terminal, the data can be immediately made inaccessible, and fraudulent activity can be prevented in advance.

[0050] The program executed by the information processing device 10 is provided by being pre-installed in a storage circuit such as a ROM. Alternatively, the program may be provided by being stored in a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD, or flexible disk as a file in an installable or executable format. The program executed by the information processing device 10 may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.

[0051] In addition, each of the units, the camera information acquisition unit 13, the face authentication unit 14, the external shape extraction unit 15, the posture determination unit 16, and the fraud determination unit 17, may be implemented in the user terminal 20 to control the data displayed on the user terminal 20.

[0052] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0053] 10...information processing device, 11...data transmission / reception unit, 12...user information registration unit, 13...camera information acquisition unit, 14...face authentication unit, 15...outline extraction unit, 16...posture determination unit, 17...fraud determination unit, 18...display control unit, 20 (20a, 20b, 20c)...user terminal, 21 (21a, 21b, 21c)...camera, 50...imaging device (smartphone).

Claims

1. a camera information acquisition unit that acquires an image of a front portion of a screen of the user terminal captured via a camera of the user terminal; a face authentication unit that performs face authentication of a person based on face information of a user when the person is captured in the acquired image; an outline extraction unit that determines whether an outline of an imaging device is extracted in the image near a person whose face is not successfully recognized; a posture determination unit that determines a body orientation of the person in the captured image; a fraud determination unit that detects fraud when an outline of an imaging device is extracted near a person for whom face authentication is unsuccessful and the person's body is facing the direction of a screen of the user terminal; a display control unit that controls data on a screen of the user terminal when fraud is detected; An information processing device comprising:

2. The display control unit stops the video data being played on the user terminal when fraud is detected.

2. The information processing apparatus according to claim 1,

3. acquiring an image of a front screen of the user terminal captured via a camera of the user terminal; When a person is captured in the captured image, performing face authentication on the person based on face information of the user; determining whether or not an outline of an imaging device is extracted in the image near a person whose face is not successfully recognized; determining a body orientation of the person in the captured image; detecting fraud when the contour of an imaging device is extracted near a person for whom face authentication is unsuccessful and the person's body is facing the screen of the user terminal; controlling data on a screen of the user terminal if fraud is detected; 13. An information processing method comprising:

4. Computer, A function for acquiring an image of the front of the screen of the user terminal captured via a camera of the user terminal; A function to perform facial recognition of a person based on the user's facial information when the person appears in the captured image; a function of determining whether or not an outline of an imaging device is extracted in the vicinity of a person whose face is not successfully recognized within the image; A function of determining a body orientation of the person in the captured image; A function of detecting fraud when the outline of an imaging device is extracted in the vicinity of a person for whom face authentication is unsuccessful and the person's body is facing the direction of the screen of the user terminal; a function for controlling data on the screen of the user terminal when fraud is detected; An information processing program characterized by causing the program to function as follows.

Images