Information processing device, information processing method, and information processing program

Abstract

To provide an information processing device, an information processing method, and an information processing program with which, for data displayed on the terminal screen of a user, it is possible to detect with good accuracy and prevent fraudulent imaging by an operator.SOLUTION: An information processing device 10 comprises: a camera information acquisition unit 13 for acquiring the front image of a screen of a user terminal 20 having been captured via a camera 21 of the user terminal; a face authentication unit 14 for performing face authentication of a user based on face information of the user when a person is shown in the acquired image; a lens extraction unit 15 for determining whether or not the outer shape of lens of an imaging device can be extracted within the image by using lens information of the imaging device which is stored in advance; a fraud determination unit 16 for detecting a fraud when the outer shape of lens is extracted in the vicinity of the person who is successful in face authentication; and a display control unit 17 for controlling data on the screen of the user terminal when a fraud is detected.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] An embodiment of the present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Teleworking, in which company work is done from a remote location away from the company's base, is becoming more common. Teleworking allows employees to work from anywhere via a network, so it has the advantage of allowing them to flexibly choose where to work, such as their own homes.

[0003] However, in telework, where employees work from home, it is difficult to constantly monitor the status of each employee's work compared to the traditional way of working where employees gather at a company base to perform their work, and it is difficult to detect when an employee behaves in a manner that violates company policies regarding the performance of work. In particular, it is difficult to detect fraudulent actions, even if a device screen is illegally photographed while the employee is working.

[0004] Conventionally, technology has been disclosed that detects an incident when a third party other than the worker in charge of the work is captured on camera during teleworking. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2015-222555 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, while the method of detecting an incident when a third party other than the worker performing the work is captured on camera can detect suspicious third party actions in advance, if a teleworking worker illegally photographs the screen of a work terminal using their own mobile device such as a smartphone or tablet, it is difficult to detect the fraud because the worker is the legitimate user of the work terminal, and there is a risk of work information being leaked.

[0007] The present invention has been made in consideration of the above circumstances, and aims to provide an information processing device, an information processing method, and an information processing program that can accurately detect and prevent unauthorized photographing by workers of data displayed on a user's terminal screen. [Means for solving the problem]

[0008] An information processing device according to an embodiment of the present invention is characterized in that it includes a camera information acquisition unit that acquires an image of the front of the screen of the user terminal captured via the camera of the user terminal; a face authentication unit that performs face authentication of a person based on the user's face information when the person appears in the acquired image; a lens extraction unit that stores lens information of an imaging device and determines whether the lens outline of the imaging device is extracted in the image using the lens information; a fraud determination unit that detects fraud when the lens outline is extracted in the vicinity of a person whose face authentication has been successful; and a display control unit that controls data on the screen of the user terminal when fraud is detected. Effect of the Invention

[0009] According to an embodiment of the present invention, an information processing device, an information processing method, and an information processing program are provided that can accurately detect and prevent unauthorized photographing by a worker of data displayed on a user's terminal screen. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an information processing device according to an embodiment of the present invention. [Diagram 2] 1 is a schematic diagram showing a method for detecting unauthorized photography according to the present embodiment; [Diagram 3] (A) An explanatory diagram showing an example of judging an image taken by the camera of a user terminal, and (B) an explanatory diagram showing another example of judging an image taken by the camera of a user terminal, in which an explanatory diagram is shown of a case where it is determined that the image was taken illegally by a worker. [Figure 4] 4 is a flowchart showing an example of an information processing method according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a diagram showing an example of the configuration of an information processing device 10 according to the present embodiment. The information processing device 10 is a server that performs face authentication of a person in an image captured by the camera 21 of the user terminal 20, determines whether the lens outline of the imaging device is extracted near a person whose face has been successfully authenticated, and controls data on the screen of the user terminal 20 if fraud is detected. The data refers to various data stored in the information processing device 10, including files, image data, video data, and the like. The data is executed and displayed by an application in the user terminal 20. The imaging device refers to an optical device for taking photographs and videos, and a terminal that has the device built in, and includes cameras (including digital cameras), smartphones with built-in cameras, tablets, and other mobile terminals.

[0012] The user terminal 20 refers to a terminal such as a PC, a smartphone, or a tablet terminal used by an employee of a company, organization, or the like for business purposes, and is a terminal capable of communicating with the information processing device 10 via a network. The user terminal 20 may also be a headset terminal that is driven by being worn on the user's head.

[0013] The user terminal 20 includes a camera 21. The camera 21 is an imaging unit that can constantly communicate with the user terminal 20 and captures an image in front of the display screen (display) of the terminal. The camera 21 may be built into the display screen of the user terminal 20, or may be arranged on the side of the display screen of the user terminal 20 and connected to the main body of the user terminal 20 from the outside. When the user terminal 20 is a headset terminal, it includes a camera that captures an image of the surroundings of the user wearing the terminal, particularly the back of the user.

[0014] The images captured by the camera 21 include still images, moving images, etc. The acquired image data is transmitted to the information processing device 10.

[0015] In addition, in Figure 1, three user terminals 20 (20a, 20b, 20c) and three cameras 21 (21a, 21b, 21c) are shown equipped on each user terminal 20, but this configuration is not limited to this, and further multiple user terminals 20 and cameras 21 may be connected.

[0016] A specific configuration of the information processing device 10 according to this embodiment will be described. The information processing device 10 includes a data transmission / reception unit 11, a user information registration unit 12, a camera information acquisition unit 13, a face authentication unit 14, a lens extraction unit 15, a fraud determination unit 16, and a display control unit 17.

[0017] The functions of each unit constituting the information processing device 10 may be realized by executing a predetermined program code using a processor, and are not limited to such software processing, but may be realized, for example, by hardware processing using ASIC or the like, or may be realized by a combination of software processing and hardware processing. Also, each unit constituting the information processing device 10 may be realized by a separate server, and for example, data accessed from the user terminal 20 may be stored in a server separate from the main body of the information processing device 10.

[0018] The data transmitter / receiver 11 transmits data such as files and video data in response to an access request from the user terminal 20, and receives image data captured by the camera 21 of the user terminal 20, and the like.

[0019] The user information registration unit 12 registers authentication information for authenticating a user of the user terminal 20. A login ID, a password, and the like are registered in association with a user ID for identifying each user. Furthermore, face information of the user used for face authentication is registered in association with the user ID. When the user terminal 20 connects to the information processing device 10 and accesses data, the user terminal 20 may be authenticated based on the registered authentication information.

[0020] Also, users having access rights may be set in association with each piece of data. Note that a plurality of users may be set as a group, and access rights to data may be set for each group.

[0021] The camera information acquisition unit 13 acquires images captured by the camera 21 of the user terminal 20. When the user terminal 20 starts accessing data, the camera information acquisition unit 13 causes the camera 21 of the user terminal 20 to start capturing images, and causes the camera 21 to continue capturing images until the access of the user terminal 20 ends. The camera information acquisition unit 13 acquires images captured by the camera 21 at any time.

[0022] The face authentication unit 14 extracts a person's face from an image acquired by the camera 21, and performs face authentication by comparing the extracted face with the user's face information for authentication. When multiple people are extracted from an image, face authentication is performed for each person based on the user's face information. A person whose face is successfully authenticated in the image is a user who has access authority to the data, while a person whose face is not successfully authenticated is a user who does not have access authority. When a person whose face is successfully authenticated is present in the image, the face authentication unit 14 transmits an image including the person to the lens extraction unit 15. When a person whose face is successfully authenticated is present, access to the data may be denied.

[0023] The facial authentication method may be performed using facial information of the user that is registered in advance in association with the user, or may be performed using facial information for authentication that is stored in the user terminal 20.

[0024] In addition, when multiple people are to view the data, the multiple users who will view the data may be registered in advance, and facial authentication may be performed on the image captured by camera 21 based on the facial information of each registered user.

[0025] The lens extraction unit 15 stores lens information of an imaging device, and uses the lens information to determine whether or not the lens outline of the imaging device is extracted in an image. The lens information of an imaging device is information for identifying the lens of the imaging device from within an image, and refers to information such as the shape (circular with a diameter of 1 cm, etc.) and color of the lens mounted on the imaging device. The lens information may include information such as the number and arrangement of lenses in a rectangular imaging device (e.g., a smartphone or tablet). In this case, the outline (length and width, etc.) of the imaging device is set, and information such as the arrangement of the lenses and the number of lenses within that outline is set.

[0026] The lens extraction unit 15 determines whether the lens outer shape is extracted in an image captured by the camera 21 based on the lens information. The lens extraction unit 15 may also learn a plurality of images capturing the lens outer shape of the imaging device by machine learning, and determine whether the lens outer shape is extracted in an image captured by the camera 21 using a generated image determination model. The lens extraction unit 15 may also determine that the lens outer shape of the imaging device is present in the image when an object matching the shape of the lens stored in the lens information is present in the image and an image of reflected light is present in the object.

[0027] Furthermore, when the lens outline of the imaging device is present in the image, the lens extraction unit 15 may determine the orientation of the lens outline in the captured image. The lens orientation means the direction of the front surface of the lens (the direction of the normal to the front surface of the lens). The lens extraction unit 15 then determines whether the front surface of the lens is oriented toward the display screen of the user terminal 20.

[0028] In addition, the lens extraction unit 15 may determine that the front of the lens is facing the direction of the display screen of the user terminal 20 if the front of the lens is facing the direction of the display screen of the user terminal 20 at least once within a certain period of time (for example, within 10 seconds).

[0029] The fraud determination unit 16 detects fraud when the lens outline of the imaging device is extracted in the vicinity (for example, within a distance of 30 cm) of a person whose face has been successfully authenticated. It determines that a person with access authority to data is pointing their imaging device at the screen of the user terminal 20, and determines that there is a risk of fraudulent photography.

[0030] In addition, the fraud determination unit 16 may detect fraud when the lens outline of the imaging device is extracted near a person whose face has been successfully authenticated, and the front of the lens is facing the display screen of the user terminal 20.

[0031] In addition, the fraud determination unit 16 may detect fraud when the lens outline of the imaging device is extracted near a person whose face has been successfully authenticated and that person remains in the same position for a predetermined period of time or more (for example, one minute or more).

[0032] When the display control unit 17 determines that there is a risk of unauthorized photographing, it controls so that the data displayed on the user terminal 20 cannot be accessed. Specifically, when unauthorized act is detected, the application that executes the data being operated by the user of the user terminal 20 is hidden. As a method of hiding, there is a method of covering the entire application being operated with a black image so that the application that executes the data cannot be viewed. Also, only the application at the forefront of the screen may be hidden. When unauthorized act is no longer detected, the application is released from the hidden state, and the data can be viewed.

[0033] Furthermore, when data is displayed on the display screen of the user terminal 20, control may be performed so as to switch to a warning screen.

[0034] If the display control unit 17 determines that unauthorized photography has occurred, the display control unit 17 may perform control to stop copying or expanding the display screen of the user terminal 20.

[0035] As a result, when a user of user terminal 20 is working with data displayed on the terminal screen and there is a risk of the user engaging in fraudulent activity such as taking a photograph of the terminal screen, the data can be instantly made unviewable, thereby preventing fraudulent activity from occurring.

[0036] 2 is a schematic diagram showing a method for detecting unauthorized photography according to the present embodiment (see FIG. 1 as appropriate). Here, a case will be considered in which a user A of a user terminal 20 connects to an information processing device 10 and browses data using a specific application.

[0037] When browsing of data is started, the camera information acquisition unit 13 acquires an image of the front of the display screen of the user terminal 20 from the camera 21. The camera image captures user A and the like. At this time, face authentication is performed on the captured camera image using the face information of user A. Face authentication is successful for user A in the image.

[0038] Then, using the lens information, it is determined whether the lens outline of the imaging device 50 is extracted near the user A whose face has been successfully authenticated in the image. Since the lens outline of the imaging device 50 is extracted near the user A, it is determined that there is a risk of unauthorized photography by the user A.

[0039] If it is determined that the photograph has been taken illegally, the application that executes the data displayed on the user terminal 20 is hidden so that the data cannot be viewed.

[0040] 3(A) and (B) are explanatory diagrams showing an example of determination of an image captured by camera 21 of user terminal 20. Here, it is assumed that user A of user terminal 20 is viewing data.

[0041] In the example of FIG. 3A, while face authentication of user A of user terminal 20 is successful, the lens outline of the imaging device is not extracted in the vicinity of user A, and therefore unauthorized photography is not determined.

[0042] In Fig. 3(B), face authentication of user A of user terminal 20 is successful, and the outline of a smartphone is extracted in the vicinity of user A, so it is determined that there is a risk of unauthorized photography by user A. Data displayed on user terminal 20 is controlled so as not to be viewable.

[0043] In this way, it is possible to accurately determine the actions of a worker attempting to illegally photograph the screen of the user terminal 20, and to prevent such illegal photographing from occurring.

[0044] Next, the operation of the information processing device 10 according to the present embodiment will be described.

[0045] FIG. 4 is a flowchart showing an example of an information processing method according to the present embodiment (see FIG. 1 as appropriate).

[0046] Upon receiving a request from the user terminal 20, the data transmitter / receiver 11 transmits the work data to the user terminal 20. The user terminal 20 displays the work data (S10).

[0047] When the user terminal 20 accesses the work data, the camera information acquisition unit 13 acquires an image captured by the camera 21 of the terminal (S11).

[0048] The face authentication unit 14 determines whether face authentication is successful for the person in the acquired image based on the face information of the user of the user terminal 20 (S12). If face authentication of the person in the image is not successful, the display of the work data is made non-displayable (S12: NO, S14).

[0049] When a person whose face has been successfully authenticated is present in the image, the lens extraction unit 15 determines whether the lens outline of the smartphone is extracted near the person (S12: YES, S13). When the lens outline is not extracted near the person, the display of the work data continues (S13: NO).

[0050] When the lens outline is extracted near a person whose face has been successfully authenticated, the fraud determination unit 16 determines that the operation is fraudulent. When the operation is determined to be fraudulent, the display control unit 17 hides the display of the work data (S13: YES, S14).

[0051] According to the information processing device of the embodiment described above, face authentication is performed on a person in an image captured by a camera of a user terminal, and it is determined whether the lens outline of an imaging device is extracted near a person whose face authentication has been successful, and if fraudulent activity by a worker is detected, the display of an application being executed on the user terminal 20 is controlled. As a result, if fraudulent photography is performed by a user performing work on the user terminal 20, the display of the application being used for work can be hidden, thereby preventing fraudulent activity in advance.

[0052] The program executed by the information processing device 10 is provided by being pre-installed in a storage circuit such as a ROM. Alternatively, the program may be provided by being stored in a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD, or flexible disk as a file in an installable or executable format. The program executed by the information processing device 10 may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.

[0053] In addition, each of the units, the camera information acquisition unit 13, the face authentication unit 14, the lens extraction unit 15, and the fraud determination unit 16, may be implemented in the user terminal 20 to control the data displayed on the user terminal 20.

[0054] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0055] 10...information processing device, 11...data transmission / reception unit, 12...user information registration unit, 13...camera information acquisition unit, 14...face authentication unit, 15...lens extraction unit, 16...fraud determination unit, 17...display control unit, 20 (20a, 20b, 20c)...user terminal, 21 (21a, 21b, 21c)...camera, 50...imaging device (smartphone).

Claims

1. a camera information acquisition unit that acquires an image of a front portion of a screen of the user terminal captured via a camera of the user terminal; a face authentication unit that performs face authentication of a person based on face information of a user when the person is captured in the acquired image; a lens extraction unit that stores lens information of an imaging device and determines whether or not an outer shape of a lens of the imaging device is extracted in the image by using the lens information; a fraud determination unit that detects fraud when the lens outline is extracted near a person whose face has been successfully authenticated; a display control unit that controls data on a screen of the user terminal when fraud is detected; An information processing device comprising:

2. The display control unit, when fraud is detected, hides an application of the data being operated on the user terminal.

2. The information processing apparatus according to claim 1,

3. acquiring an image of a front screen of the user terminal captured via a camera of the user terminal; When a person is captured in the captured image, performing face authentication on the person based on face information of the user; storing lens information of an imaging device, and determining whether or not an outer shape of a lens of the imaging device is extracted in the image using the lens information; detecting fraud when the lens outline is extracted in the vicinity of a person whose face has been successfully authenticated; controlling data on a screen of the user terminal if fraud is detected; 13. An information processing method comprising:

4. Computer, A function for acquiring an image of the front of the screen of the user terminal captured via a camera of the user terminal; A function to perform facial recognition of a person based on the user's facial information when the person appears in the captured image; a function of storing lens information of an imaging device and determining whether or not the outer shape of the lens of the imaging device is extracted in the image by using the lens information; A function to detect fraud when the lens outline is extracted near a person whose face has been successfully recognized; a function for controlling data on the screen of the user terminal when fraud is detected; An information processing program characterized by causing the program to function as follows.

Images