Information processing device, program, and method

Abstract

To provide an information processing device, a program and a method that prevent information leakage by snooping.SOLUTION: In an information leakage suppress system 100, the control device 170 of a user terminal 120 comprises: a generated image acquisition unit for acquiring a generated image; a display image generation unit for generating a display image; a surrounding image acquisition unit for acquiring a surrounding image; a monitoring object detection unit for analyzing the surrounding image and detecting a monitoring object which is at least one of a monitoring subject 22 and an apparatus being monitored (imaging apparatus existing behind a user 20 of the user terminal 120); a viewpoint estimation unit which, when the monitoring subject is detected, estimates the viewpoint of the monitoring subject in a display region; and a movement determination unit for determining whether the manner of movement of the viewpoint estimated by the viewpoint estimation unit matches a predetermined condition. The display image generation unit determines that a display image with part of the generated image hidden is outputted when it is determined that the manner of viewpoint movement matches the predetermined condition or when the apparatus being monitored is detected.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a program, and a method. [Background technology]

[0002] Patent Document 1 discloses an information display device that automatically sets itself to prevent others from peeking if it is determined that there is a risk of others peeking. Patent Document 2 discloses an electronic device that performs anti-peeping control when it detects the gaze of someone other than the user. Patent Document 3 discloses a technology that acquires gaze information on a screen image that a user gazes at for each user and displays which display area within the screen image the user gazed at. Patent Documents 4 and 5 disclose non-contact gaze detection technology. (Prior art document) (Patent document) (Patent Document 1) JP 2012-129701 A (Patent Document 2) Patent No. 6474921 (Patent Document 3) Patent No. 6485522 (Patent Document 4) JP 2021-043869 A (Patent Document 5) JP 2021-077265 A Summary of the Invention [Means for solving the problem]

[0003] In a first aspect of the present invention, there is provided an information processing device. The information processing device generates a display image to be displayed in a display area of ​​an image display device, for example, based on a generated image generated by an image generation device. The information processing device includes, for example, a generated image acquisition unit that acquires the generated image. The information processing device includes, for example, a display image generation unit that generates a display image based on the generated image. The information processing device includes, for example, a peripheral image acquisition unit that acquires a peripheral image, which is an image captured by an imaging device that captures an image of the periphery of the display area. The information processing device includes, for example, a monitoring target detection unit that analyzes the peripheral image to detect a monitoring target that is at least one of (i) a monitoring target who is a person who is directing their gaze toward the display area and is different from the user of the image display device, and (ii) a monitoring target device that is an imaging device facing the display area. The information processing device includes, for example, a viewpoint estimation unit that estimates a viewpoint of the monitoring target in the display area when the monitoring target is detected. The information processing device includes, for example, a movement determination unit that determines whether a manner of movement of the viewpoint estimated by the viewpoint estimation unit matches a predetermined condition. In the above-mentioned information processing device, the display image generation unit decides to output a display image in which a portion of the generated image is concealed, for example, when (i) it is determined that the manner of movement of the viewpoint matches predetermined conditions, or (ii) a monitored device is detected.

[0004] The information processing device may include a candidate information acquisition unit that acquires candidate information, which is information created in advance based on the contents of an electronic file used by the image generation device in a process for generating at least a portion of a generated image, and is information for identifying one or more candidate data that may be subject to anonymization in the electronic file. The information processing device may include a candidate area determination unit that determines one or more candidate areas in the display area, where each of the one or more candidate data identified by the candidate information is displayed. The information processing device may include a target area determination unit that determines at least one of the one or more candidate areas as a target area, where anonymization is to be performed in the display area, based on a manner of movement of the viewpoint. In the information processing device, each of the one or more candidate data may be data used by the image generation device to generate a different portion of the generated image. In the information processing device, the display image generation unit may determine to output a display image in which the target area determined by the target area determination unit is anonymized.

[0005] In the information processing device, each of the one or more candidate areas may include a start area located at the start position of the candidate area and an end area located at the end position of the candidate area. In the information processing device, the extent of the end area in the display area may be different from the extent of the start area in the display area. In the information processing device, the candidate information may include representative point information that is information for identifying a representative point of the start area of ​​each of the one or more candidate areas and a representative point of the end area of ​​each of the one or more candidate areas. In the information processing device, the predetermined condition may include at least one of: (i) a first condition that, after the viewpoint is estimated to be located in the start area of ​​a first candidate area among the one or more candidate areas, the viewpoint is estimated to be located in the end area of ​​the first candidate area before a predetermined period of time has elapsed; and (ii) a second condition that the viewpoint is estimated to have moved along a vector from the representative point of the start area of ​​the first candidate area toward the representative point of the end area of ​​the first candidate area. In the above information processing device, the target area determination unit may determine at least the first candidate area as the target area when the manner of movement of the viewpoint satisfies at least one of a first condition and a second condition.

[0006] The information processing device may include a behavior determination unit that, when the monitoring target detection unit detects a monitoring target, determines whether a behavior pattern of the monitoring target obtained by analyzing a peripheral image matches a predetermined pattern. In the information processing device, the display image generation unit may (a) determine to output a display image in which a portion of the generated image is concealed when it is determined that the behavior pattern of the monitoring target matches a predetermined pattern and that the manner of viewpoint movement matches a predetermined condition. In the information processing device, the display image generation unit may (b) determine to output the generated image as the display image when it is not determined that the behavior pattern of the monitoring target matches a predetermined pattern or when it is not determined that the manner of viewpoint movement matches a predetermined condition.

[0007] The information processing device may include a recognition degree derivation unit that, when the monitoring target detection unit detects a monitoring target, derives a recognition degree indicating a degree of likelihood that the monitoring target will recognize the content of information displayed in the display area if concealment is not performed, based on a behavioral pattern of the monitoring target obtained by analyzing a peripheral image. In the information processing device, the display image generation unit may determine to output a display image in which a portion of the generated image is concealed when (a) it is determined that the recognition degree exceeds a predetermined degree and that the manner of movement of the viewpoint matches a predetermined condition. In the information processing device, the display image generation unit may determine to output the generated image as the display image when (b) the recognition degree does not exceed the predetermined degree or that the manner of movement of the viewpoint is not determined to match the predetermined condition.

[0008] A second aspect of the present invention provides a method for generating a display image to be displayed in a display area of ​​an image display device based on a generated image generated by an image generation device. The method includes, for example, a generated image acquisition step in which a computer acquires the generated image. The method includes, for example, a display image generation step in which the computer generates a display image based on the generated image. The method includes, for example, a peripheral image acquisition step in which the computer acquires a peripheral image, which is an image captured by an imaging device that captures the periphery of the display area. The method includes, for example, a monitoring target detection step in which the computer analyzes the peripheral image to detect a monitoring target who is directing his or her gaze toward the display area and is a person other than a user of the image display device. The method includes, for example, a viewpoint estimation step in which, when the monitoring target is detected, the computer estimates the viewpoint of the monitoring target in the display area. The method includes, for example, a movement determination step in which the computer determines whether the manner of movement of the viewpoint estimated in the viewpoint estimation step matches a predetermined condition. In the method, the display image generation step includes, for example, a step in which the computer outputs a display image in which a portion of the generated image is concealed when it is determined that the manner of movement of the viewpoint matches the predetermined condition.

[0009] In a third aspect of the present invention, a program is provided. The program may be a program for causing a computer to function as any of the information processing devices according to the first embodiment. The program may be a program for causing a computer to execute the method according to the second embodiment. A computer-readable medium for storing the program may be provided. The computer-readable medium may be a non-transitory computer-readable medium. The computer-readable medium may be a computer-readable recording medium.

[0010] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0011] [Figure 1] 1 shows an example of a system configuration of an information leakage prevention system 100. [Figure 2] An example of an original image 200 is shown schematically. [Figure 3] An example of an output image 300 is shown schematically. [Figure 4] 10 shows an example of information processing in the control device 170. [Figure 5] 1 shows an example of the internal configuration of the control device 170. [Figure 6] 10 shows an example of the internal structure of a value map generation unit 578. [Figure 7] An example of the data structure of a data table 700 is shown schematically. [Figure 8] 10 shows an example of the internal configuration of the periphery monitoring unit 552. [Figure 9] 10 shows an example of the internal configuration of the concealment processing unit 584. [Figure 10] 10 shows an example of information processing in a viewing determination unit 832. [Figure 11] 10 shows an example of information processing in the leakage prevention support unit 586. [Figure 12]3 shows an example of a system configuration of a computer 3000. DETAILED DESCRIPTION OF THE INVENTION

[0012] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. In the drawings, identical or similar parts may be designated by the same reference numerals, and redundant description may be omitted. Furthermore, those skilled in the art who have read the description of this specification will understand that technical matters described in one embodiment may also be applied to other embodiments to the extent that they are not technically inconsistent.

[0013] (Outline of Information Leakage Prevention System 100) An overview of the information leakage prevention system 100 will be explained using Figures 1, 2, and 3. Figure 1 shows an example of the system configuration of the information leakage prevention system 100. In this embodiment, the information leakage prevention system 100 can conceal part of the information presented to a user (for example, a user 20 described later).

[0014] The information leakage prevention system 100 conceals a part of an image displayed on the user terminal 120 in response to an instruction from, for example, an administrator or a user of the user terminal 120 managed by the information leakage prevention system 100. The instruction may be an instruction to register or update a setting related to concealment. The instruction may be stored in the user terminal 120 as a setting related to concealment.

[0015] The administrator or user of the user terminal 120 configures the user terminal 120 so that, for example, when the user terminal 120 is started or immediately after the user terminal 120 is started, a program for concealing a portion of an image displayed by the user terminal 120 is launched. The administrator or user of the user terminal 120 may configure the user terminal 120 so that a portion of an image displayed by the user terminal 120 is concealed when the length of time since the user terminal 120 last received an instruction from the administrator or user becomes greater than a predetermined value.

[0016] In this embodiment, the information leakage prevention system 100 will be described in detail using an example in which the information leakage prevention system 100 (i) first generates an original image 200 that has not been subjected to an anonymization process, and (ii) then anonymizes at least a portion of the original image 200 to generate an output image 300 to be presented to a user. Fig. 2 schematically shows an example of the original image 200. Fig. 3 schematically shows an example of the output image 300.

[0017] 1 , in this embodiment, the information leakage prevention system 100 includes, for example, a user terminal 120. In this embodiment, the user terminal 120 includes, for example, a housing 130, a display device 140, an environmental sensor 150, an input device 160, and a control device 170. In this embodiment, the display device 140 includes, for example, a display area 142. In this embodiment, the environmental sensor 150 includes, for example, a distance sensor 152, a camera 154, and a radio wave detector 156.

[0018] 1, image 10 shows an example of an image captured by camera 154. As shown in image 10, in this embodiment, details of an embodiment of the information leakage prevention system 100 will be described using as an example a case where a person 22 and a person 24, who are different from user 20, are present behind user 20 of user terminal 120. In this embodiment, for the purpose of facilitating understanding of the details of each part of the information leakage prevention system 100 and / or the details of information processing in the information leakage prevention system 100, details of an embodiment of the information leakage prevention system 100 will be described using as an example a case where user 20 is inputting various instructions to the user terminal 120, person 22 is directing his / her gaze toward the display area 142 of the user terminal 120, and person 24 is not directing his / her gaze toward the display area 142 of the user terminal 120.

[0019] (Overview of user terminal 120) In this embodiment, the user terminal 120 outputs various types of information to the user 20. The user terminal 120 may output information by image or by sound. The user terminal 120 may also accept information input from the user 20. The user terminal 120 is configured to be able to execute various types of information processing.

[0020] In this embodiment, the details of the user terminal 120 will be described using an example in which the user terminal 120 outputs an image. For example, a user 20 of the user terminal 120 inputs instructions to the user terminal 120 via the input device 160 for the BIOS, operating system (sometimes referred to as an OS), or application program (sometimes the term "program" is used as a general term for these) running on the user terminal 120. The user terminal 120 executes information processing in accordance with the instructions, and displays an image showing the results of the information processing in the display area 142.

[0021] In this embodiment, the user terminal 120 generates an output image 300 by concealing, for example, a portion of an original image 200 generated by a program running on the user terminal 120. The user terminal 120 displays the output image 300 in the display area 142 of the display device 140. As a result, the output image 300 is presented to the user 20. According to this embodiment, a portion of the output image 300 is concealed. Therefore, even if the output image 300 displayed in the display area 142 is viewed by a person other than the user 20 who is near the display area 142, information leakage can be suppressed. Examples of programs that generate the original image 200 include a BIOS, an operating system (sometimes referred to as an OS), and various application programs.

[0022] Examples of the user terminal 120 include a personal computer and a mobile terminal. Examples of the mobile terminal include a mobile phone, a smartphone, a PDA (registered trademark), a tablet, a notebook computer or laptop computer, and a wearable computer.

[0023] In this embodiment, the housing 130 supports the display device 140, the environmental sensor 150, the input device 160, and the control device 170. At least a portion of each of the display device 140, the environmental sensor 150, the input device 160, and the control device 170 may be housed inside the housing 130 or may be attached to the outside of the housing 130.

[0024] In this embodiment, the display device 140 outputs various images. The display device 140 may output an image or stop outputting an image according to an instruction from the control device 170. The display device 140 may adjust the brightness or contrast of an image according to an instruction from the control device 170. For example, the display device 140 controls the operation of pixels based on data output by the control device 170, thereby being able to output an image, stop outputting an image, or adjust the brightness or contrast of an image according to an instruction from the control device 170. Examples of the display device 140 include a display and a projector.

[0025] In this embodiment, an image output by the display device 140 (sometimes referred to as an output image) is displayed in the display area 142. In this embodiment, the display area 142 is disposed in the housing 130. In this embodiment, the display area 142 is disposed on approximately the same surface as at least a portion of the environmental sensor 150.

[0026] In this embodiment, the environmental sensor 150 acquires various types of information related to the surrounding environment of the display area 142. The environmental sensor 150 may include one or more sensors for acquiring the above-mentioned various types of information. Examples of the above-mentioned sensors include a sensor for acquiring various images, a sensor for acquiring sound, a sensor for detecting brightness, a sensor for detecting temperature, a sensor for detecting distance or depth, and a receiver for receiving wireless signals. The information acquired or generated by the one or more sensors included in the environmental sensor 150 is output to the control device 170, for example.

[0027] In this embodiment, the distance sensor 152 measures the distance between the distance sensor 152 and each of one or more objects arranged around the display area 142. Examples of the periphery of the display area 142 include positions within the viewing angle of the display area 142 and positions whose distance from the display area 142 is within a predetermined numerical range. The distance sensor 152 may acquire point cloud data of each of the above objects. The point cloud data may be three-dimensional point cloud data.

[0028] Furthermore, the relative positional relationship between the display area 142 and the distance sensor 152 is known. Therefore, the distance sensor 152 may derive the distance between the display area 142 and each of the above-mentioned objects based on the distance between the distance sensor 152 and each of the above-mentioned objects and the relative positional relationship between the display area 142 and the distance sensor 152.

[0029] The distance sensor 152 may detect the proximity of the user 20 of the user terminal 120 to the display device 140 or the display area 142. For example, the distance sensor 152 measures the distance between the user 20 of the user terminal 120 and the display device 140 or the display area 142. When the distance between the user 20 of the user terminal 120 and the display device 140 or the display area 142 becomes smaller than a predetermined value, the distance sensor 152 may detect the proximity of the user 20 to the image display device.

[0030] The principle and / or details of the procedure by which the distance sensor 152 measures the distance are not particularly limited. Examples of the distance sensor 152 include (i) a sensor that captures an image, and (ii) a sensor that measures the distance to an object using sound waves, ultrasonic waves, electromagnetic waves, lasers, etc. Examples of the distance sensor 152 include a stereo camera, a monocular camera depth distance measurement device (sometimes referred to as a monocular depth estimation device), LiDAR, a non-contact proximity sensor, etc. An optical element such as a fisheye lens or an ultra-wide-angle lens may be attached to the camera.

[0031] In this embodiment, the camera 154 captures an image of the periphery of the display area 142. This acquires an image 10 of the periphery of the display area 142. The image 10 may be a still image or a moving image.

[0032] The details of the camera 154 are not particularly limited. The camera 154 may be an image sensor for capturing two-dimensional images, or may be an infrared camera for capturing infrared images (sometimes referred to as thermal images). The camera 154 may be a monocular camera or a stereo camera. The camera 154 may be a 180-degree camera or a 360-degree camera. Furthermore, if the distance sensor 152 includes an imaging element, the user terminal 120 may not include the camera 154.

[0033] In this embodiment, the radio wave detector 156 detects radio waves of a wireless signal. The radio wave detector 156 may measure the strength of the wireless signal. Examples of the wireless signal include a GPS signal, a Wi-Fi (registered trademark) signal, and a Bluetooth (registered trademark) signal. As a result, for example, the control device 170 can analyze the detected radio waves to estimate the installation position of the display area 142.

[0034] In this embodiment, the input device 160 accepts input from the user 20 of the user terminal 120. Examples of the input include input related to the operation and / or settings of the user terminal 120, input related to the operation and / or settings of various programs running on the user terminal 120, and input related to the content of work executed by the programs. The input device 160 outputs information input by the user 20 to the input device 160 to the control device 170. Examples of the input device 160 include a keyboard, a pointing device, a touch panel, a camera, and a microphone.

[0035] In this embodiment, the control device 170 controls the user terminal 120. The control device 170 executes various information processes in the user terminal 120. Details of the control device 170 will be described later.

[0036] The information processing method may be a method for generating a display image to be displayed on an image display device by concealing a part of a generated image generated by an image generation device. Each step of the information processing method may be executed by, for example, a computer.

[0037] The information processing method may be a method for generating a display image to be displayed in a display area of ​​an image display device based on a generated image generated by an image generation device. Each step of the information processing method is executed by, for example, a computer. The method includes, for example, a generated image acquisition step of acquiring a generated image. The method includes, for example, a display image generation step of generating a display image based on the generated image. The method includes, for example, a peripheral image acquisition step of acquiring a peripheral image captured by an imaging device that captures the periphery of the display area. The method includes, for example, a surveillance target detection step of analyzing the peripheral image to detect a surveillance target who is directing his or her gaze toward the display area and is a person other than a user of the image display device. The method includes, for example, a viewpoint estimation step of estimating the viewpoint of the surveillance target in the display area when the surveillance target is detected. The method includes, for example, a movement determination step of determining whether the manner of viewpoint movement estimated in the viewpoint tracking step matches a predetermined condition. In the method, the display image generation step includes, for example, a step of outputting a display image in which a portion of the generated image is concealed when it is determined that the manner of viewpoint movement matches the predetermined condition.

[0038] (Outline of original image 200 and output image 300) 2 and 3 , the original image 200 and the output image 300 will be described in detail, taking as an example a case where a window 210 is displayed full-screen when a program for concealing a part of the original image 200 (sometimes referred to as a concealment program) is executed. Therefore, according to this embodiment, a specific position or range in the original image 200 can be an example of a specific position or range in the display area 142. Similarly, a specific position or range in the output image 300 can be an example of a specific position or range in the display area 142.

[0039] As described above, in this embodiment, the information leakage prevention system 100 (i) first generates an original image 200 that has not been subjected to an anonymization process, and (ii) then anonymizes at least a portion of the original image 200 to generate an output image 300 to be presented to the user 20. The output image 300 includes an area (sometimes referred to as an anonymized area) in which at least a portion of the information (sometimes referred to as presented information) to be presented to the user 20 has been anonymized.

[0040] (Overview of original image 200) 2, in this embodiment, the original image 200 has an image displayed in a full-screen window 210. In this embodiment, the window 210 includes a control area 212 and an image output area 214.

[0041] As described above, the window 210 corresponds to the anonymization program. The original image 200 shows an example of a screen displayed in the display area 142 after the anonymization program has been launched and in a state where the anonymization program is not executing an anonymization process.

[0042] In this embodiment, one or more objects for displaying various information related to control of window 210 and / or control of the masking program are arranged in control area 212. For example, control area 212 has an object 222 for presenting to user 20 operations (sometimes referred to as a menu) available to user 20, and an object 224 for presenting to user 20 ON / OFF of masking processing.

[0043] The control area 212 may be provided with an object (not shown) for presenting the name (sometimes called the title) of the concealed program to the user 20, an object (not shown) for allowing the user 20 to change the size of the window 210, and an object (not shown) for allowing the user 20 to select whether to display or hide the window 210.

[0044] In this embodiment, the image output area 214 displays, for example, a background image (not shown) generated by the OS, presentation information 216 which is information presented to the user 20 by the OS, and images of a window 230 and a window 240 corresponding to a program or file other than the OS and the masking program. In this embodiment, the window 230 includes a control area 232, an image output area 234, and presentation information 236. In this embodiment, the window 240 includes a control area 242, an image output area 244, and presentation information 246.

[0045] In this embodiment, the control area 232 has one or more objects arranged therein for displaying various types of information relating to the control of the window 230 and / or the control of the program or file (sometimes referred to as a document file) corresponding to the window 230. The image output area 234 displays presented information 236, which is information presented to the user 20 by the program or file corresponding to the window 230.

[0046] In this embodiment, the control area 242 has one or more objects arranged therein for displaying various information relating to the control of the window 240 and / or the control of the program or file corresponding to the window 240. The image output area 244 displays presentation information 246, which is information presented to the user 20 by the program or file (sometimes referred to as a document file) corresponding to the window 240.

[0047] (Document file summary) As described above, image output area 234 of window 230 displays presented information 236, which is information presented to user 20 by the document file corresponding to window 230. Similarly, image output area 244 of window 240 displays presented information 246, which is information presented to user 20 by the document file corresponding to window 240.

[0048] The document file may be any electronic file that is used by the control device 170 in processing to generate an image of the window 230, and the details thereof are not particularly limited. The document file may be a text file or various types of binary files. Examples of binary files include image files, audio files, compressed files, and files for various application programs. The image file may be a still image file or a moving image file. The format of the document file is not particularly limited, and any known format may be adopted.

[0049] A document file includes data indicating the contents of a document. A document file may include (i) data indicating the contents of the document (sometimes referred to as document data), and (ii) data indicating the layout of the document (sometimes referred to as layout data), and / or data for identifying data indicating the contents of the document that may be subject to anonymization (sometimes referred to as candidate data) (sometimes referred to as associated data).

[0050] In this embodiment, the content of a document is not limited to a character string, but may be composed of, for example, a character string, a picture, a diagram, a table, a photograph, a video, or a combination thereof.

[0051] A single document file may include a single piece of candidate data or multiple pieces of candidate data, each of which is used by the control device 170 to generate a different portion of the original image 200. The content of the information selected as candidate data is not particularly limited.

[0052] Of the information contained in the document, for example, (i) information that is highly valuable to the administrator or user 20 of the user terminal 120, or the creator or administrator of the document (sometimes referred to as user 20, etc.) in terms of preventing leakage, and (ii) information that is highly valuable to the user 20, etc. in terms of restricting access to the document are treated as candidate data. Information that is highly valuable to the user 20, etc. in terms of restricting access to the document is, for example, information that is restricted from access for some reason. Examples of reasons for restricting access include the age of the viewer and the likelihood that the viewer will feel uncomfortable.

[0053] The associated data may store, for example, the position of each of one or more pieces of candidate data in a document. The associated data may store, for each of one or more pieces of candidate data, the position of each piece of candidate data in a document in association with at least one of the content of each piece of candidate data and the value of each piece of candidate data.

[0054] The associated data may include information indicating the start position of each of one or more candidate data, which allows the control device 170 to identify the representative point of the start region of the candidate region corresponding to each candidate data, for example, by reading the document data and the associated data.

[0055] The associated data may include information indicating the end position of each of one or more candidate data, which allows the control device 170 to identify the representative point of the end region of the candidate region corresponding to each candidate data, for example, by reading the document data and the associated data.

[0056] The associated data of a specific document file is created in advance, for example, before the concealment process for the specific document file is executed. The associated data of a specific document file is created, for example, based on the content of the specific document file. The associated data of a specific document file is created, for example, based on the eye movement or input operation history of the creator or updater of the specific document file during the period in which the specific document file is created or updated. The associated data of a specific document file is created, for example, based on the eye movement or input operation history of a user 20 viewing or updating the specific document file.

[0057] In one embodiment, the associated data for a specific document file is created before the specific document file is executed by the control device 170. In this case, the candidate data included in the specific document file can be masked immediately after the specific document file is executed.

[0058] In another embodiment, the associated data of a specific document file is generated by the control device 170 after the control device 170 executes the specific document file and during the period when the specific document file is being executed. In this case, the candidate data included in the specific document file may be anonymized after the specific document file is executed and the associated data is generated.

[0059] 3, the document data of the document file corresponding to window 230 includes a plurality of candidate data 30. Each of the plurality of candidate data 30 is displayed in an area (sometimes referred to as a candidate area) in original image 200 that corresponds to the respective one of the plurality of candidate data 30.

[0060] Using Figure 3, the display mode of candidate data in original image 200 will be explained in detail using an example in which the document file corresponding to window 230 contains the content "abcdef" and the content "bcde" of "abcdef" is information that is highly valuable to user 20 or the like in preventing leakage.

[0061] According to this embodiment, the candidate data "bcde" is displayed in a candidate area 260 of the original image 200. The candidate area 260 is arranged in a part of the original image 200. The candidate area 260 includes, for example, a start area 262 arranged at the start position of the candidate area 260, and an end area 264 arranged at the end position of the candidate area 260. In this embodiment, the range of the start area 262 in the display area 142 or the original image 200 is different from the range of the end area 264 in the display area 142 or the original image 200.

[0062] According to this embodiment, in the original image 200, the candidate data "bcde" is displayed along a vector 40 that starts at the representative point 272 of the start region 262 and ends at the representative point 274 of the end region 264. Note that the vector 40 is a virtual vector and is not displayed in the original image 200. The vector 40 may also be referred to as a vector corresponding to the candidate region 260.

[0063] Depending on the size of display area 142 of window 230, a line break may be inserted in the middle of the candidate data "bcde." In this case, the direction of vector 40 indicated by representative point 272 and representative point 274 does not match the display direction of the candidate data.

[0064] Therefore, according to one embodiment, when a line break is performed in the middle of the candidate data "bcde", if the document displayed in window 230 is written horizontally, the direction of vector 40 is considered to be parallel to the row direction. On the other hand, when the document displayed in window 230 is written vertically, the direction of vector 40 is considered to be parallel to the column direction. According to another embodiment, when a line break is performed in the middle of the candidate data "bcde", the direction of vector 40 may be indicated by a combination of the row direction and the column direction.

[0065] (An example of another embodiment) In this embodiment, the procedure for determining the display direction of candidate data in the original image 200 (specifically, the direction of vector 40) has been described in detail using an example in which document file data is displayed in window 230 and the information is of high value to user 20 and the like in terms of preventing leakage. As described above, the display direction of candidate data is an example of a display mode of candidate data. However, the procedure for determining the display direction of candidate data is not limited to this embodiment.

[0066] According to another embodiment, the data of the image file is displayed in window 230. When the candidate data that may be subject to anonymization is part of a moving image or a still image, it is difficult to uniquely identify the display direction of the candidate data. In such cases, the direction of the candidate data is indicated, for example, by directional divisions. This allows the direction of the candidate data to be determined within a certain range. The number of divisions into directional divisions is not particularly limited, but examples include 3, 4, 6, 8, and 12 divisions. Examples of four divisions include up, down, left, and right. The direction of the candidate data may be indicated probabilistically. For example, the direction of the candidate data is indicated by the probability of each of the above directional divisions. When the directional divisions are up, down, left, and right, the direction of the candidate data is indicated as up = 80%, right = 15%, left = 5%, and down = 0%.

[0067] (Output image 300 overview) 3, in this embodiment, the output image 300 has a full-screen image of the window 210. In this embodiment, the window 210 includes a control area 212 and an image output area 214.

[0068] As described above, the window 210 corresponds to the anonymization program. The output image 300 shows an example of a screen displayed in the display area 142 after the anonymization program has been launched and in a state in which the anonymization process is being executed by the anonymization program.

[0069] In this embodiment, one or more objects for displaying various information related to the control of the window 210 and / or the control of the masking program are arranged in the control area 212. According to this embodiment, in addition to the object 222 and the object 224, the control area 212 is also arranged with an object 326 for presenting the operation mode of the masking process to the user 20, and an object 328 for presenting the type and / or strength of the masking process to the user 20.

[0070] In this embodiment, one or more objects for concealing at least a part of one or more candidate regions included in the original image 200 are arranged in the image output area 214 at or near the position and / or range (sometimes referred to as the viewpoint) of the line of sight of the person 22 in the display area 142. According to the embodiment shown in Fig. 3, an object 330 for concealing the candidate region 260 corresponding to the candidate data "bcde" included in the document file corresponding to the window 230 is arranged in the image output area 214.

[0071] The image of the object 330 is not particularly limited as long as it can conceal the image of the original image 200. The degree of concealment (sometimes referred to as strength) may be determined by initial settings or by the administrator or user 20. The image of the object 330 will be described in detail later.

[0072] In this embodiment, whether to display or hide the object 330 is determined based on the manner of movement of the viewpoint 50 of the person 22 (sometimes referred to as the movement manner). For example, if the degree of match (sometimes referred to as the degree of match) between a portion of the trajectory of the viewpoint 50 of the person 22 and the vector 40 corresponding to the candidate area 260 exceeds a predetermined level, the object 330 is displayed and the candidate area 260 is concealed. On the other hand, if the degree of match does not meet the predetermined level, or if the degree of match no longer meets the predetermined level, the object 330 is not displayed. In this case, the output image 300 in which the candidate data "bcde" corresponding to the candidate area 260 is displayed is displayed in the display area 142 of the display device 140. The procedure for determining whether to display or hide the object 330 will be described in detail below.

[0073] The information leakage prevention system 100 may be an example of an information processing device. The user terminal 120 may be an example of an image generating device, an image display device, an imaging device, or an information processing device. The display device 140 may be an example of an image display device. The display area 142 may be an example of a display area of ​​an image display device. The environmental sensor 150 may be an example of an imaging device. The camera 154 may be an example of an imaging device. The control device 170 may be an example of an image generating device or an information processing device.

[0074] The original image 200 may be an example of a generated image. The image of the window 230 may be an example of at least a portion of a generated image. The candidate area 260 may be an example of one or more candidate areas. The start area 262 may be an example of a start area located at the start position of each candidate area. The end area 264 may be an example of an end area located at the end position of each candidate area. The output image 300 may be an example of a display image. The area where the object 330 is located may be an example of a target area.

[0075] Image 10 may be an example of a peripheral image. User 20 may be an example of a user 20 of an image display device. Person 22 may be an example of a person to be monitored or a monitored object. Multiple candidate data 30 may be an example of one or more candidate data. A concealed area may be an example of a target area. A document file may be an example of an electronic file. Accompanying data may be an example of candidate information. Information indicating the start position of each candidate data may be an example of representative point information. Information indicating the end position of each candidate data may be an example of representative point information. The candidate area corresponding to each candidate data may be an example of one or more candidate areas.

[0076] (An example of another embodiment) In this embodiment, the information leakage prevention system 100 has been described in detail using as an example a case where the user terminal 120 conceals a portion of the original image 200 and generates the output image 300. However, the information leakage prevention system 100 is not limited to this embodiment. In other embodiments, the user terminal 120 may be configured to be able to communicate with another computer disposed outside the user terminal 120 and may execute the various information processes described above in cooperation with the other computer. For example, at least a portion of the control device 170 may be realized by the other computer.

[0077] In the present embodiment, the details of the information leakage prevention system 100 have been described using as an example a case where the display area 142 and the environmental sensor 150 are arranged on approximately the same plane. However, the information leakage prevention system 100 is not limited to this embodiment. In other embodiments, as long as the relative positional relationship between the display area 142 and the environmental sensor 150 can be identified, at least one of the display area 142 and the environmental sensor 150 may be arranged outside the housing 130.

[0078] In the present embodiment, the details of the information leakage prevention system 100 have been described using as an example a case where the control device 170 executes a document file to generate the original image 200 and the output image 300. However, the information leakage prevention system 100 is not limited to this embodiment. In other embodiments, the control device 170 may execute any type of electronic file to generate the original image 200 and the output image 300. For example, the control device 170 executes an image file to generate the original image 200 and the output image 300. The image file may be a still image file or a moving image file.

[0079] In this embodiment, the details of the information leakage prevention system 100 have been described using as an example a case where the information leakage prevention system 100 prevents leakage of confidential information via a person 22 present behind a user 20 of a user terminal 120. However, the information leakage prevention system 100 is not limited to this embodiment.

[0080] In another embodiment, the information leakage prevention system 100 may prevent imaging of confidential information by an imaging device (sometimes referred to as a monitored device) located behind the user 20 of the user terminal 120. Confidential information may be an example of information that is highly valuable to the user 20, etc., and whose viewing should be restricted. Those skilled in the art who have read this specification will understand that in various embodiments described herein, leakage of confidential information using a monitored device can be prevented by replacing the viewpoint 50 of the person 22 with the imaging position of the monitored device in the display area 142. Specifically, leakage of confidential information can be prevented by displaying concealed confidential information in the display area 142.

[0081] In the present embodiment, the details of the information leakage prevention system 100 have been described using as an example a case where display or non-display of the object 330 is determined based on the manner of movement of the viewpoint 50 of the person 22. However, the information leakage prevention system 100 is not limited to the present embodiment.

[0082] In another embodiment, the information leakage prevention system 100 may control whether to display or hide the object 330 based on (i) the relative positional relationship between the person 22 directing their gaze toward the display area 142 and the display area 142, and (ii) the direction of gaze of the person 22. The information leakage prevention system 100 may control whether to display or hide the object 330 based on (i) the relative positional relationship between the user 20, the person 22 directing their gaze toward the display area 142, and the display area 142, and (ii) the direction of gaze of the person 22.

[0083] For example, the control device 170 identifies the range of an area (sometimes referred to as a silhouette) in the display area 142 where the line of sight of the person 22 is blocked by the body of the user 20, based on the relative positional relationship between the user 20, the person 22 directing his / her gaze toward the display area 142, and the display area 142. Examples of the range of the silhouette include the position, shape, and size of the silhouette. When at least a portion of the object 330 is located inside the silhouette of the user 20, the control device 170 may control the display or non-display of the object 330 so that the portion of the object 330 that overlaps with the silhouette is not displayed. As a result, data located inside the silhouette is displayed in the display area 142 without being concealed by the object 330. The object 330 may be generated to cover the entire screen except for the silhouette of the user 20.

[0084] The above silhouette is generated, for example, according to the following procedure. First, the camera 154 captures an image of the user 20, and an appearance of the user 20 is generated on a two-dimensional plane with the direction of the camera 154 as the normal direction. The above silhouette is generated by projecting a mask bounded by the appearance of the user 20 onto the display area 142, taking into account the line of sight of the person 22 who is directing his or her gaze toward the display area 142. The above mask may be an image for obscuring the image of the area where the mask is placed. The line of sight of the person 22 may be determined (i) based on the direction of the face of the person 22, (ii) based on the appearance of one of the person 22's eyes, or (iii) based on the result of separately calculating the line of sight of both eyes of the person 22 and performing an AND operation on the line of sight of the right eye and the line of sight of the left eye.

[0085] Another example of a procedure for generating a silhouette is to have camera 154 photograph user 20, generate or estimate a three-dimensional model, generate a two-dimensional appearance from the line of sight of person 22 looking toward display area 142, and project that appearance onto display area 142 to generate a silhouette. The three-dimensional model is generated or estimated, for example, using xNN-based technology that estimates a 3D silhouette from a monocular planar image.

[0086] In still another embodiment, the information leakage prevention system 100 may control whether to display or hide the object 330 based on (i) the relative positional relationship between the display area 142 and an imaging device (which may be referred to as a monitored device) whose imaging direction is facing the display area 142, and (ii) the imaging direction of the monitored device. The information leakage prevention system 100 may control whether to display or hide the object 330 based on (i) the relative positional relationship between the user 20, the imaging device whose imaging direction is facing the display area 142, and the display area 142, and (ii) the imaging direction of the monitored device.

[0087] For example, the control device 170 identifies the range of an area (sometimes referred to as a silhouette) in the display area 142 where the body of the user 20 obstructs the imaging device from capturing an image, based on the relative positional relationship between the user 20, the imaging device facing the display area 142, and the display area 142. Examples of the range of the silhouette include the position, shape, and size of the silhouette. When at least a portion of the object 330 is located inside the silhouette of the user 20, the control device 170 may control the display or non-display of the object 330 so that the portion of the object 330 that overlaps with the silhouette is not displayed. As a result, data located inside the silhouette is displayed in the display area 142 without being concealed by the object 330. The object 330 may be generated to cover the entire screen except for the silhouette of the user 20.

[0088] (Overview of the concealment process) Fig. 4 schematically shows an example of information processing in the control device 170. Specifically, an example of the above-mentioned anonymization processing will be described with reference to Fig. 4. According to this embodiment, first, in step 422 (step may be abbreviated as S), the control device 170 starts the above-mentioned anonymization program.

[0089] As a result, the window 210 corresponding to the masking program is displayed in full screen in the foreground. The image displayed in the window 210 immediately after the masking program is started is generated based on, for example, the settings related to the masking program.

[0090] (a) If the settings related to the anonymization program indicate that a predetermined image or an image generated or selected by a predetermined algorithm is to be displayed immediately after startup, the control device 170 displays the image in the display area 142. (b) If the settings related to the anonymization program indicate that anonymization processing is not required immediately after startup, the control device 170 determines, for example, to display the original image 200 in the display area 142. (c) If the settings related to the anonymization program indicate that anonymization processing is required immediately after startup, the control device 170 determines, for example, to anonymize at least a portion of the original image 200 in accordance with the settings related to the anonymization program. The control device 170 determines, for example, to display the output image 300 in the display area 142. In this case, the details of the anonymization processing (e.g., the target and manner of anonymization) may be determined based on the settings described above.

[0091] In one embodiment, the activation process of the anonymization program is automatically executed when the user terminal 120 is started up. In another embodiment, the activation process of the anonymization program is executed in response to an instruction from the user 20.

[0092] In still another embodiment, if it is not confirmed that the user terminal 120 is located within a predetermined area, the activation process of the concealment program may be automatically executed. Examples of the predetermined area include an area where the administrator of the user terminal 120 has permission to use the user terminal 120, a room used by an organization to which the user 20 belongs, the office of the user 20, or the home of the user 20. Whether the user terminal 120 is located within the predetermined area may be determined based on an image of the periphery of the display area 142 captured by the camera 154, or may be determined based on information included in a wireless signal detected by the radio wave detector 156.

[0093] Next, in S424, the control device 170 determines a target for anonymization. For example, the control device 170 determines an operation mode related to the target for anonymization. Examples of the operation mode related to the target for anonymization include (a) a mode in which the anonymization process is generally performed, and the anonymization process is not performed exceptionally in an area estimated to be necessary for the work of the user 20 (this mode may be referred to as a first mode), (b) a mode in which the anonymization process is not performed in principle, and the anonymization process is performed exceptionally in an area estimated to be viewed by a person other than the user 20 who is present near the display area 142 (this mode may be referred to as a second mode), and (c) a mode in which the anonymization process is generally performed, and the anonymization process is not performed exceptionally in an area estimated to be difficult for a person other than the user 20 to view (this mode may be referred to as a third mode).

[0094] In this embodiment, in order to facilitate understanding of the information leakage prevention system 100, the details of the information leakage prevention system 100 will be described using as an example a case where the control device 170 operates in the second mode. The control device 170 may be configured to be able to change the operation mode related to the anonymization target in response to an instruction from, for example, an administrator or a user 20.

[0095] Thereafter, the control device 170 executes processing for generating an image for updating the display screen of the display area 142. For example, first, in S432, the control device 170 acquires a document file for generating at least a portion of the original image 200. The control device 170 acquires, for example, a document file corresponding to the window 230. As described above, in this embodiment, the document file includes document data indicating the contents of the document and associated data indicating each of one or more candidate data included in the document.

[0096] Next, in S434, the control device 170 executes the document file. The control device 170 generates an original image 200 including an image showing the contents of the document, for example, based on document data included in the document file. The control device 170 generates the original image 200 based on output from, for example, an OS and / or an application program capable of executing the document file. The application program capable of executing the document file may be a program different from the anonymization program.

[0097] According to this embodiment, the control device 170 reads the associated data and identifies one or more candidate data (sometimes referred to as concealment candidates). The control device 170 also determines the range of each of one or more candidate regions in the original image 200. The control device 170 may determine the start point of a vector corresponding to each candidate region, as well as the direction and length of each vector. The direction and length of each vector are determined, for example, based on information regarding the start point and end point of each vector.

[0098] Next, in S440, the control device 170 determines whether the movement pattern of the viewpoint 50 of the person 22 matches a predetermined condition (sometimes referred to as a condition related to the movement of the viewpoint). This suppresses unnecessary concealment processing, improving the usage experience of the user 20. Note that the movement pattern of the viewpoint 50 of the person 22 matching the condition related to the movement of the viewpoint includes not only a case where the two match completely, but also a case where the two match approximately.

[0099] The condition regarding the movement of the viewpoint may be that a portion of the trajectory of the viewpoint 50 of the person 22 matches a vector corresponding to any one of the one or more candidate areas. In this embodiment, for the purpose of making it easier to understand the information processing in the information leakage prevention system 100, an example of the above-mentioned concealment processing will be described using as an example a case where the control device 170 determines whether a portion of the trajectory of the viewpoint 50 of the person 22 matches a vector 40 corresponding to the above-mentioned candidate area 260.

[0100] In one embodiment, if the viewpoint 50 of the person 22 is estimated to be located in the start region 262 of the candidate region 260 and then, before a period of time having a predetermined length has elapsed, the viewpoint 50 of the person 22 is estimated to be located in the end region 264 of the candidate region 260, it is determined that a portion of the trajectory of the viewpoint 50 of the person 22 matches the vector 40 corresponding to the candidate region 260. In another embodiment, if the viewpoint 50 of the person 22 is estimated to have moved along the vector 40, it is determined that a portion of the trajectory of the viewpoint 50 of the person 22 matches the vector 40 corresponding to the candidate region 260.

[0101] If the movement pattern of the viewpoint 50 of the person 22 does not match the condition regarding the movement of the viewpoint in S440 (No in S440), the control device 170 generates an output image 300 in S442 in which the candidate region 260 of the original image 200 has not been subjected to the anonymization process. The control device 170 controls the display device 140 to display the generated output image 300 in the display region 142. As a result, the candidate region 260 of the output image 300 displayed in the display region 142 of the display device 140 displays "bcde," which is the candidate data corresponding to the region. Note that even in this case, if it is determined that the movement pattern of the viewpoint 50 of the person 22 matches a vector corresponding to another candidate region, the candidate region 260 may also be anonymized along with the other candidate region.

[0102] On the other hand, if the movement pattern of the viewpoint 50 of the person 22 matches the condition related to the movement of the viewpoint in S440 (Yes in S440), the control device 170 determines in S450 whether the behavior pattern of the person 22 matches a predetermined pattern (sometimes referred to as a reference pattern). This further reduces unnecessary concealment processing, further improving the user experience of the user 20. Note that matching the behavior pattern of the person 22 with the reference pattern includes not only a case where the two match completely, but also a case where the two match approximately.

[0103] In one embodiment, if the behavior pattern of the person 22 does not match a pattern characteristic of a person who does not intend to spy on information or a pattern characteristic of a person who is performing an action that makes it physically difficult to spy on information compared to when the person is stationary (which may be referred to as a first behavior pattern), the control device 170 may determine that the behavior pattern of the person 22 matches the reference pattern. In another embodiment, if the behavior pattern of the person 22 matches a pattern characteristic of a person who intends to spy on information (which may be referred to as a second behavior pattern), the control device 170 may determine that the behavior pattern of the person 22 matches the reference pattern. The first behavior pattern and the second behavior pattern will be described in detail below.

[0104] In S450, if the behavior pattern of the person 22 does not match the reference pattern (No in S450), in S442, the control device 170 generates an output image 300 in which no concealment processing has been applied to the candidate region 260 of the original image 200. Details of the information processing in S442 may be as described above.

[0105] On the other hand, if the behavior pattern of the person 22 matches the reference pattern in S450 (Yes in S450), then in S452 the control device 170 generates an output image 300 in which concealment processing has been applied to the candidate area 260 of the original image 200. The control device 170 controls the display device 140 to display the generated output image 300 in the display area 142. This minimizes leakage of the candidate data "bcde" corresponding to the candidate area 260 of the original image 200, even if the person 22 peeks at the display area 142 of the display device 140.

[0106] The control device 170 repeats, for example, the processes of S440 to S452. The control device 170 may repeat the processes of S434 to S452, or may repeat the processes of S432 to S452. In this way, the concealment process is executed.

[0107] The condition regarding movement of the viewpoint may be an example of a first condition or a second condition. Candidate area 260 may be an example of a first candidate area. Vector 40 may be an example of a vector extending from a representative point of the start area of ​​the first candidate area to a representative point of the end area of ​​the first candidate area.

[0108] (An example of another embodiment) In this embodiment, the details of the anonymization process have been described using as an example a case where it is determined in S450 whether the behavior pattern of person 22 matches a reference pattern. However, the anonymization process is not limited to the above embodiment. In other embodiments, the process of S450 may be omitted.

[0109] In this embodiment, the details of the information leakage prevention system 100 have been described using the case where the control device 170 operates in the second mode as an example, in order to facilitate understanding of the information leakage prevention system 100. However, the information leakage prevention system 100 is not limited to this embodiment.

[0110] In another embodiment, the control device 170 may operate in the first mode or the third mode. When the control device 170 operates in the third mode, the control device 170 determines whether to output the output image 300 in which a portion of the original image 200 is concealed, for example, based on the viewpoint of the person 22 and the silhouette of the user 20. The control device 170 determines whether to output the output image 300 in which a portion of the original image 200 is concealed, for example, based on the gaze position (i.e., the viewpoint) of the person 22 in the display area 142 and the silhouette of the user 20 in the display area 142. The control device 170 determines whether to output the output image 300 in which a portion of the original image 200 is concealed, for example, based on the shooting position of the monitored device in the display area 142 and the silhouette of the user 20 in the display area 142. Specifically, when at least a portion of the object 330 is located inside the silhouette of the user 20, the control device 170 controls the display or non-display of the object 330 so that the portion of the object 330 overlapping with the silhouette is not displayed.

[0111] 5 schematically illustrates an example of the internal configuration of the control device 170. In this embodiment, the control device 170 includes, for example, a setting information storage unit 520, a program control unit 532, a window control unit 534, an input control unit 540, and an output control unit 550. In this embodiment, the output control unit 550 includes a periphery monitoring unit 552, an original image generation unit 560, a line-of-sight information acquisition unit 574, an operation information acquisition unit 576, a value map generation unit 578, and an image processing unit 580. In this embodiment, the image processing unit 580 includes a concealment processing unit 584, a leakage prevention support unit 586, and an output image generation unit 588.

[0112] In this embodiment, the setting information storage unit 520 stores information (sometimes referred to as setting information) relating to various settings of the information leakage prevention system 100 and / or the user terminal 120. The setting information includes, for example, information relating to the settings of the anonymization program. Examples of the settings of the anonymization program include (i) settings relating to whether or not anonymization by the anonymization program is necessary or the target, and / or the mode of the anonymization, and (ii) various settings relating to the processing immediately after the anonymization program is started.

[0113] In this embodiment, the program control unit 532 controls various programs that run on the user terminal 120. The program control unit 532 may control the startup process of the above programs, the operation of the above programs, or the termination process of the above programs.

[0114] In this embodiment, the window control unit 534 executes various controls related to windows corresponding to various programs running on the user terminal 120. The window control unit 534 outputs, for example, various pieces of information related to the above-mentioned windows (sometimes referred to as window information) to the output control unit 550. Examples of the window information include information indicating the range of the window on the screen (for example, position, shape, size, etc.), information regarding various objects arranged in the window, and information indicating the type of flag assigned to the window. Examples of the above-mentioned flags include "copy not allowed" and "screenshot not allowed."

[0115] In this embodiment, the input control unit 540 controls various types of information input to the user terminal 120. In one embodiment, the input control unit 540 acquires data output by the environment sensor 150. The input control unit 540 acquires, for example, image data output by the camera 154, and outputs the image data to the line-of-sight information acquisition unit 574.

[0116] In another embodiment, the input control unit 540 controls the operation of the input device 160. The input control unit 540 receives, for example, various instructions input by the user 20 to the input device 160. Examples of the instructions include instructions related to the operation of the user terminal 120 and instructions for various programs running on the user terminal 120. This allows the input control unit 540 to identify, for example, the position or range of an input by the user 20.

[0117] The input control unit 540 outputs, for example, the above instruction to the operation information acquisition unit 576. The input control unit 540 may output information indicating the position or range of the input by the user 20 to the operation information acquisition unit 576. The input control unit 540 may output information indicating the content input by the user 20 to the input device 160 to the operation information acquisition unit 576.

[0118] In this embodiment, output control unit 550 controls the output of information presented to user 20. In one embodiment, output control unit 550 controls the output of image information. In another embodiment, output control unit 550 controls the output of audio information.

[0119] In this embodiment, the periphery monitoring unit 552 monitors the environment surrounding the display area 142. For example, the periphery monitoring unit 552 monitors a person present in the periphery of the display area 142. The periphery monitoring unit 552 may monitor the presence or absence of a person present in the periphery of the display area 142, may monitor the line of sight of the person, or may monitor the behavior of the person. The periphery monitoring unit 552 may store a history of the monitoring results in an arbitrary storage device. The periphery monitoring unit 552 may store data acquired for monitoring in an arbitrary storage device.

[0120] For example, the periphery monitoring unit 552 monitors the environment surrounding the display area 142 by analyzing the image 10 captured by the camera 154. As described above, if a person is present in the periphery of the display area 142, the person will appear in the image 10. The periphery monitoring unit 552 may store data of the image 10 in an arbitrary storage device. The periphery monitoring unit 552 may store data of the image 10 captured during an arbitrary period in the storage device.

[0121] In one embodiment, the periphery monitoring unit 552 may store the data of the image 10 in any storage device while deleting old data so that the most recent data is always stored. The data stored in the storage device may be data from the most recent tens of seconds, data from the most recent hours, or data from the most recent days.

[0122] In another embodiment, the periphery monitoring unit 552 may store the data of the image 10 in the first storage device while deleting old data so that the most recent data is always stored. The data stored in the first storage device may be data for the most recent tens of seconds, data for the most recent hours, or data for the most recent days.

[0123] For example, the periphery monitoring unit 552 monitors the occurrence of a specific event. When the occurrence of the specific event is detected, the periphery monitoring unit 552 may store data of the image 10 from a first time point at which the occurrence of the event is detected to a second time point that is an earlier time point than the time point at which the occurrence of the event is detected in a second storage device different from the first storage device. When the occurrence of the specific event is detected, the periphery monitoring unit 552 may store the data of the image 10 in the first storage device while deleting old data so that the data of the image 10 from the first time point to the second time point is not deleted from the first storage device. The length of the period between the first time point and the second time point may be predetermined by initial setting or by specification by the user 20, etc.

[0124] According to these embodiments, if there is a possibility that data has been spied on, evidence video can be recorded to verify whether or not the data has been spied on. Details of the perimeter monitoring unit 552 will be described later.

[0125] In this embodiment, the original image generation unit 560 generates the original image 200. In this embodiment, the output control unit 550 acquires window information relating to each of one or more windows included in the original image 200 from the window control unit 534.

[0126] In this embodiment, the line-of-sight information acquisition unit 574 acquires an image of the user 20 captured by the camera 154 from the input control unit 540. The line-of-sight information acquisition unit 574 analyzes the image of the user 20 and estimates the position and / or range of the line of sight of the user 20 in the original image 200 (as described above, this may be referred to as the viewpoint). Any known method may be employed as a method for estimating the viewpoint of the user 20 in the original image 200. As described above, the viewpoint of the user 20 in the original image 200 may be an example of the viewpoint of the user 20 in the display area 142.

[0127] In one embodiment, the gaze information acquisition unit 574 analyzes the image to identify a normal vector of the face of the user 20. The gaze information acquisition unit 574 estimates the viewpoint of the user 20 in the original image 200 using the normal vector of the face of the user 20. In another embodiment, the gaze information acquisition unit 574 analyzes the image to identify a normal vector of the pupil of the user 20. The gaze information acquisition unit 574 estimates the viewpoint of the user 20 in the original image 200 using the normal vector of the pupil of the user 20.

[0128] In this embodiment, the operation information acquisition unit 576 acquires various pieces of information (sometimes referred to as operation information) that the user 20 has input to the input device 160 from the input control unit 540. As described above, examples of the operation information include (i) information indicating various instructions or contents that the user 20 has input to the input device 160, and (ii) information indicating the position or range of the input by the user 20.

[0129] In this embodiment, the value map generating unit 578 generates a database (sometimes referred to as a value map) that stores identification information (sometimes referred to as a candidate data ID) of one or more candidate data included in a document file in association with information indicating the range (for example, position, shape, size, etc.) occupied by the candidate area corresponding to each candidate data in the original image 200 of the document file. The value map generating unit 578 outputs the value map of the document file to, for example, the image processing unit 580.

[0130] Thereby, by referring to the value map, the image processing unit 580 can determine where information that is of high value to the user 20 or the like is located in the display area 142 or the original image 200. As described above, examples of information that is of high value to the user 20 or the like include information that is of high value in preventing leakage, information that is of high value in restricting viewing, etc. Details of the value map generation unit 578 will be described later.

[0131] In this embodiment, the image processing unit 580 conceals a part of the original image 200 generated by the original image generation unit 560, and generates an output image 300 to be displayed in the display area 142 of the display device 140. The image processing unit 580 may control the operation of the display device 140 to display the output image 300 in the display area 142.

[0132] In this embodiment, the concealment processing unit 584 executes the concealment processing described above. The concealment processing unit 584 determines whether or not to conceal a part of the original image 200 output by the original image generation unit 560, based on, for example, the monitoring result of the line of sight and / or behavior of the person 22 output by the periphery monitoring unit 552. When it is determined that a part of the original image 200 should be concealed, the concealment processing unit 584 determines the target and / or manner of concealment.

[0133] In one embodiment, the concealment processing unit 584 outputs information indicating the target and / or mode of concealment to the output image generation unit 588. In another embodiment, the concealment processing unit 584 acquires the original image 200 generated by the original image generation unit 560. The concealment processing unit 584 conceals a portion of the original image 200 based on the above determination. The concealment processing unit 584 outputs the image in which the portion of the original image 200 has been concealed to the output image generation unit 588. Details of the concealment processing unit 584 will be described later.

[0134] The leakage prevention support unit 586 executes various processes (sometimes referred to as support processes) for preventing information leakage. Examples of the support processes include (a) a process for encouraging the user 20 to take a specific action for preventing information leakage, (b) a process for causing the control device 170 to execute a specific information process for preventing information leakage, and (c) a process for outputting a sound indicating the status of the concealment process by the control device 170.

[0135] An example of the specific behavior is changing the physical position and / or attitude of the user terminal 120. For example, the user 20 can change the physical attitude of the user terminal 120 by rotating the user terminal 120 clockwise or counterclockwise. Examples of the specific information processing are (i) processing for changing the target and / or mode of anonymization when the length of a period during which no input from the user 20 is detected exceeds a predetermined value, and (ii) processing for changing the target and / or mode of anonymization between one display area 142 and another display area 142 when the user terminal 120 has multiple display areas 142.

[0136] In one embodiment, the leak prevention support unit 586 analyzes the image captured by the camera 154 to determine whether two or more people are present around the display area 142. The leak prevention support unit 586 may analyze the image captured by the camera 154 to determine whether one or more people 22 are present around the display area 142. For example, even if another person (particularly person 22) is present behind the user 20, if the user 20 rotates the user terminal 120, it may become difficult for that person to view the display area 142.

[0137] Therefore, when it is determined that two or more people exist around the display area 142, or when it is determined that one or more people 22 exist around the display area 142, the leakage prevention support unit 586 executes processing to prompt the user 20 to change the orientation of the user terminal 120 (for example, the orientation of the normal vector of the display area 142). For example, the leakage prevention support unit 586 controls the output image generation unit 588 to cause the output image generation unit 588 to generate and output an image suggesting the rotation direction of the user terminal 120.

[0138] In another embodiment, if the length of the period during which the input control unit 540 does not detect any input from the user 20 exceeds a predetermined value, the leakage prevention support unit 586 determines to change the settings related to the shape and / or size of the area where anonymization can be performed. This changes the target of anonymization. The leakage prevention support unit 586 also controls the output image generation unit 588 to update the screen displayed in the display area 142.

[0139] For example, if the above setting is changed so that the range that can be concealed becomes larger, the number of candidate areas that are concealed when peeping by the person 22 is detected may increase compared to before the setting was changed. For example, if the above setting is changed so that the range that can be concealed becomes the entire screen, all candidate areas arranged in the display area 142 may be automatically concealed when peeping by the person 22 is detected.

[0140] In yet another embodiment, when the user terminal 120 has multiple display areas 142, the leakage prevention support unit 586 adjusts the settings related to the shape and / or size of the area where concealment can be applied so that the entire screen of a display or screen other than the main display or main screen (which may be simply referred to as a sub-display) becomes the area where concealment can be applied. In this way, when it is detected that the person 22 is peeking at the sub-display, all candidate areas arranged on the sub-display can be automatically concealed.

[0141] As a result, for example, when the control device 170 outputs images of the same document file to a display built into the housing 130 of the user terminal 120 and to a projector connected to the user terminal 120, the number of concealed candidate data items in each image may differ when comparing the image output to the display when the person 22 is looking at the display with the image output to the screen when the person 22 is looking at the image output to the screen by the projector. The display built into the housing 130 of the user terminal 120 may be an example of a main display. The screen onto which an image is projected by the projector connected to the user terminal 120 may be an example of a sub-display.

[0142] In still another embodiment, the leak prevention support unit 586 may communicate the details of the anonymized candidate data by voice. The leak prevention support unit 586 may also communicate the reason why the candidate data or candidate region is anonymized by voice. The leak prevention support unit 586 may output the audio information via a directional speaker, earphones, or headphones arranged in the user terminal 120.

[0143] In this embodiment, the output image generation unit 588 acquires, for example, the original image 200 generated by the original image generation unit 560. The output image generation unit 588 generates an output image 300 based on the original image 200, for example.

[0144] The output image generation unit 588 may generate an image in which the concealment processing determined by the concealment processing unit 584 has been applied to at least a part of the original image 200 generated by the original image generation unit 560. The output image generation unit 588 may generate an image in which the support processing determined by the leak prevention support unit 586 has been applied to at least a part of the original image 200 generated by the original image generation unit 560. The output image generation unit 588 may generate an image in which the concealment processing determined by the concealment processing unit 584 and the support processing determined by the leak prevention support unit 586 have been applied to a part of the original image 200 generated by the original image generation unit 560.

[0145] As a result, the output image 300 is generated. The output image generation unit 588 controls the operation of the display device 140 to display the generated output image 300 in the display area 142. When a screen capture process is executed in response to an instruction from the user 20, the output image generation unit 588 may control the operation of the display device 140 to display the original image 200. As a result, the unanonymized original image 200 may be captured. In this case, the output image 300 may remain displayed in the display area 142 of the display device 140.

[0146] The output control unit 550 may be an example of an image generation device or an information processing device. The original image generation unit 560 may be an example of an image generation device. The image processing unit 580 may be an example of a generated image acquisition unit or an information processing device. Each unit of the image processing unit 580 may be an example of a generated image acquisition unit.

[0147] 6 schematically illustrates an example of the internal structure of the value map generation unit 578. In this embodiment, the value map generation unit 578 includes, for example, a document file acquisition unit 612, an associated data acquisition unit 614, a region data assignment unit 616, and an associated data generation unit 620. In this embodiment, the associated data generation unit 620 includes, for example, a candidate data determination unit 622 and a value determination unit 624.

[0148] The document file acquisition unit 612 acquires a document file of a document to be displayed in the display area 142. As described above, the document file includes, for example, document data indicating the contents of the document and associated data for identifying candidate data included in the document. In this case, the document file acquisition unit 612 outputs the associated data to, for example, the associated data acquisition unit 614. The document file acquisition unit 612 may output the document data to the area data assignment unit 616.

[0149] If the document file does not include any associated data, the document file acquisition unit 612 outputs, for example, information indicating that the document file does not include any associated data to the associated data acquisition unit 614. The document file acquisition unit 612 may output the document data to the area data assignment unit 616 and / or the associated data generation unit 620.

[0150] In this embodiment, the associated data acquisition unit 614 acquires, for example, the associated data of the document displayed in the display area 142. The associated data acquisition unit 614 outputs the associated data to the area data attachment unit 616, for example.

[0151] The associated data acquisition unit 614 acquires the associated data from, for example, the document file acquisition unit 612. If the document file acquired by the document file acquisition unit 612 does not contain any associated data, the associated data acquisition unit 614 acquires, for example, information indicating that the document file does not contain any associated data. In this case, the associated data acquisition unit 614 executes a process for searching for or generating the associated data of the document file.

[0152] In one embodiment, the associated data acquisition unit 614 accesses a storage device that stores a database that stores, for each of a plurality of document files, identification information of each document file and associated data of each document file in association with each other, and requests extraction of the associated data associated with the identification information of the document file. This allows the associated data acquisition unit 614 to acquire the associated data of the document file.

[0153] The URI of the storage device is stored, for example, in the setting information storage unit 520. The storage device may be (i) a storage device installed in the user terminal 120, or (ii) a storage device installed in an information processing device arranged outside the user terminal 120 and configured to be able to send and receive information to and from the user terminal 120 via a communication network.

[0154] In another embodiment, the associated data acquisition unit 614, for example, requests the associated data generation unit 620 to generate associated data for the document file. This allows the associated data acquisition unit 614 to acquire the associated data generated by the associated data generation unit 620.

[0155] The area data assigning unit 616 acquires the associated data of the document file from the associated data acquiring unit 614. The area data assigning unit 616 determines one or more areas arranged in a part of the original image 200, in which each of the one or more candidate data specified by the associated data is displayed (as described above, these are candidate areas corresponding to each candidate data).

[0156] Since each candidate area is a position where each candidate data is displayed in the original image 200 obtained by executing the document file, the area data assigning unit 616 can determine the candidate area corresponding to each candidate data according to any known procedure. As described above, in this embodiment, the one or more areas in the original image 200 where each of the one or more candidate data is displayed may be an example of the one or more areas in the display area 142 where each of the one or more candidate data is displayed.

[0157] As described above, the associated data includes, for example, information indicating the start position of each piece of one or more pieces of candidate data and information indicating the end position of each piece of candidate data. This allows the region data assigning unit 616 to identify the start region and end region of the candidate region corresponding to each piece of candidate data. The region data assigning unit 616 can identify the position of the representative point of the start region of the candidate region corresponding to each piece of candidate data, for example, based on settings regarding the procedure for determining the representative point of the start region. The region data assigning unit 616 can identify the position of the representative point of the end region of the candidate region corresponding to each piece of candidate data, for example, based on settings regarding the procedure for determining the representative point of the end region. These settings are stored, for example, in the setting information storage unit 520.

[0158] The area data assigning unit 616 stores, for example, identification information of each of one or more candidate data in association with information indicating the range occupied by the candidate area corresponding to each candidate data in the original image 200. In this way, a value map is generated.

[0159] The area data assigning unit 616 may generate a value map by assigning, to information on each piece of candidate data included in the associated data, information indicating the range occupied by the candidate area corresponding to each piece of candidate data in the original image 200. For example, if the associated data is a database having one or more records corresponding to each piece of candidate data, the area data assigning unit 616 assigns, to the record of each piece of candidate data, information indicating the range occupied by the candidate area corresponding to each piece of candidate data in the original image 200.

[0160] In this embodiment, the associated data generation unit 620 generates, for example, the associated data described above. The associated data generation unit 620 generates associated data related to a specific document file in response to, for example, a request from the associated data acquisition unit 614. The associated data generation unit 620 outputs the generated associated data to, for example, the associated data acquisition unit 614 and / or the area data assignment unit 616.

[0161] In this embodiment, the candidate data determination unit 622 determines one or more candidate data items contained in a specific document file. For example, the candidate data determination unit 622 identifies information that is highly valuable to the user 20, etc., among the information contained in the specific document file, and determines the highly valuable information as candidate data. This allows the candidate data determination unit 622 to identify the start position of each candidate data item. Similarly, the candidate data determination unit 622 can identify the end position of each candidate data item.

[0162] In one embodiment, the candidate data determination unit 622 analyzes the content of a specific document file or the original image 200 obtained when the original image generation unit 560 executes the specific document file, and identifies information that is highly valuable to the user 20, etc. The candidate data determination unit 622 identifies information that is highly valuable to the user 20, etc., by using, for example, any known natural language processing technology, pattern recognition technology, and / or image recognition technology.

[0163] Examples of information that is valuable to the user 20 or the like include at least one selected from the group consisting of a character string, a picture, a diagram, a table, a photograph, a video, and parts thereof. Examples of parts of character strings include proper nouns, verbs, subjects of verbs, objects of verbs, etc. included in the character string. Examples of parts of pictures, photographs, or videos include human faces.

[0164] In another embodiment, the candidate data determination unit 622 identifies information that is highly valuable to the user 20, etc., based on the operation history of the pointing device by the user 20, etc., during the period when the user 20, etc., is viewing, creating, or updating a specific document file. For example, the candidate data determination unit 622 identifies (i) information that was displayed in an area where the number of clicks per unit time is greater than a predetermined value, (ii) information that was displayed in an area selected by a drag operation, etc., as information that is highly valuable to the user 20, etc.

[0165] In still another embodiment, the candidate data determination unit 622 identifies information that is highly valuable to the user 20, etc., based on the movement history of the line of sight or viewpoint of the user 20, etc., during a period when the user 20, etc., is viewing, creating, or updating a specific document file. For example, the candidate data determination unit 622 identifies, as information that is highly valuable to the user 20, etc., (i) information that was displayed in an area where the length of time the line of sight or viewpoint stayed was greater than a predetermined value, (ii) information that was displayed in an area where an extremely fast involuntary movement known as a saccade was detected, etc.

[0166] In yet another embodiment, the candidate data determination unit 622 identifies information that is highly valuable to the user 20, etc., based on the type of application program that can execute a specific document file or the URI of the specific document file. For example, if the application program is an application program for spreadsheets, presentations, or document creation, the candidate data determination unit 622 identifies at least one selected from the group consisting of character strings, pictures, diagrams, tables, photographs, and videos, and parts thereof, contained in the document, as information that is highly valuable to the user 20, etc. The candidate data determination unit 622 may identify information that is highly valuable to the user 20, etc., using any known natural language processing technology, pattern recognition technology, and / or image recognition technology.

[0167] In this embodiment, the candidate data determination unit 622 may generate the associated data described above. For example, the candidate data determination unit 622 generates the associated data by associating, for each of one or more pieces of candidate data, identification information of each piece of candidate data with information indicating the position of each piece of candidate data in a document presented by a specific document file. As described above, the associated data may be a database having one or more records corresponding to each of the one or more pieces of candidate data.

[0168] In this embodiment, the value determination unit 624 determines the value of the information indicated by each of the one or more candidate data determined by the candidate data determination unit 622. For each of the one or more candidate data included in a specific document file, the value determination unit 624 determines the value of the information indicated by each of the candidate data based on at least one of, for example, (a) the type of information indicated by each of the candidate data, (b) the operation history of the pointing device by the user 20 or the like during the period when the user 20 or the like is viewing, creating, or updating the specific document file, (c) the movement history of the line of sight or viewpoint of the user 20 or the like during the period when the user 20 or the like is viewing, creating, or updating the specific document file, (d) the type of application program that can execute the specific document file, and (e) the URI of the specific document file.

[0169] The value determination unit 624 determines the value of the information indicated by each candidate data based on, for example, a score predetermined for each type of information. Examples of the type of information include a character string, a picture, a diagram, a table, a photograph, and a video. The type of information may be a proper noun, a verb, a subject of the verb, an object of the verb, a human face, and the like.

[0170] The value determination unit 624 determines the value of the information indicated by each candidate data based on, for example, the type of operation of the pointing device performed on the information indicated by each candidate data. The value determination unit 624 determines the value of the information indicated by each candidate data based on, for example, a score determined in advance for each operation of the pointing device by the user 20 or the like.

[0171] The value determination unit 624 determines the value of the information indicated by each candidate data based on, for example, the length of time that the line of sight or viewpoint of the user 20 or the like stays on the information indicated by each candidate data. For example, information included in an area where the line of sight or viewpoint of the user 20 or the like stays for a long time is estimated to be information of high value to the user 20 or the like.

[0172] The value determination unit 624 determines the value of the information indicated by each candidate data based on, for example, a score predetermined for each type of application program. The value determination unit 624 determines the value of the information indicated by each candidate data based on, for example, a score predetermined for each URI. For example, information posted on a website publicly available on the Internet is estimated to be less valuable than information on an intranet.

[0173] In this embodiment, the value determining unit 624 may assign information indicating the value of each piece of candidate data to information on each piece of candidate data included in the associated data generated by the candidate data determining unit 622. For example, if the associated data is a database having one or more records corresponding to one or more pieces of candidate data, the value determining unit 624 assigns information indicating the value of each piece of candidate data to the record of each piece of candidate data.

[0174] The associated data acquisition unit 614 may be an example of a candidate information acquisition unit. The area data assignment unit 616 may be an example of a candidate information acquisition unit or a candidate area determination unit. The associated data generation unit 620 may be an example of a candidate information acquisition unit.

[0175] FIG. 7 schematically illustrates an example of the data structure of a data table 700. The data table 700 may be an example of associated data related to a specific document file. The data table 700 may be an example of the value map described above. The data table 700 is acquired by, for example, the associated data acquisition unit 614. The data table 700 is generated by, for example, the value map generation unit 578. The data table 700 may be stored in, for example, the setting information storage unit 520. The data table 700 may be used in a process of anonymizing candidate data, or in a process of de-anonymizing anonymized candidate data.

[0176] In this embodiment, the data table 700 includes one or more records corresponding to one or more candidate data. Each of the one or more records stores, for example, a candidate data ID 722, which is identification information for identifying each candidate data, in association with information indicating a position 724 of each candidate data in a document. Examples of the position 724 of each candidate data in a document include the start position of each candidate data and the end position of each candidate data.

[0177] In this embodiment, each of the one or more records further associates and stores a candidate data ID 722, information indicating content 726 of each candidate data, information indicating value 728 of each candidate data, and information indicating the range 732 that the candidate area corresponding to each candidate data occupies in the original image 200 (sometimes referred to as the range of the candidate area in the original image 200). The range of the candidate area in the original image 200 is represented by, for example, the position of the representative point of the candidate area in the original image 200, and the shape and size of the candidate area.

[0178] 8 schematically shows an example of the internal configuration of the periphery monitoring unit 552. In this embodiment, the periphery monitoring unit 552 includes a peripheral image acquisition unit 822, a monitoring target detection unit 824, a viewpoint monitoring unit 826, and a behavior monitoring unit 828.

[0179] In this embodiment, the surrounding image acquisition unit 822 acquires the image 10 captured by the camera 154. As described above, the camera 154 captures an image of the periphery of the display area 142 of the display device 140. Therefore, if a person is present in the periphery of the display area 142, the image 10 that captures the person is obtained.

[0180] In this embodiment, the monitoring target detection unit 824, for example, analyzes the image 10 acquired by the surrounding image acquisition unit 822 to detect a person (the above-mentioned person 22) who is directing his or her gaze toward the display area 142 and who is different from the user 20 of the user terminal 120 or the display device 140. The monitoring target detection unit 824 can detect the person 22 by using, for example, any known pattern recognition technology and / or image recognition technology.

[0181] In this embodiment, the viewpoint monitoring unit 826 monitors the viewpoint of the person 22 in the display area 142. For example, the viewpoint monitoring unit 826 analyzes the image 10 acquired by the surrounding image acquisition unit 822 to monitor the viewpoint of the person 22 in the display area 142. The viewpoint monitoring unit 826 outputs information indicating the monitoring result of the viewpoint of the person 22 to the image processing unit 580 (for example, the concealment processing unit 584).

[0182] For example, when the person 22 is detected by image analysis by the monitoring target detection unit 824, the viewpoint monitoring unit 826 estimates the viewpoint of the person 22 in the display area 142. The viewpoint monitoring unit 826 estimates the viewpoint of the person 22 in the display area 142 periodically or at any timing. This allows the viewpoint monitoring unit 826 to track the movement of the viewpoint of the person 22 in the display area 142. The viewpoint monitoring unit 826 can estimate and / or track the viewpoint of the person 22 by using, for example, any known viewpoint estimation technology and / or viewpoint tracking technology.

[0183] When the person 22 is detected by the image analysis of the monitoring target detection unit 824, the viewpoint monitoring unit 826 may simplify the estimation of the viewpoint of the person 22 in the display area 142. The viewpoint monitoring unit 826 may estimate the viewpoint by assuming that the pupil is located in the center of the eyeball, or may estimate the viewpoint based on the normal direction of the face. The normal direction of the face is determined by estimating 3D landmarks of the face using, for example, xNN technology.

[0184] In this embodiment, the behavior monitoring unit 828 monitors the behavior of the person 22. For example, the behavior monitoring unit 828 analyzes the image 10 acquired by the surrounding image acquisition unit 822 to monitor the behavior of the person 22. The behavior monitoring unit 828 outputs information indicating the monitoring results of the behavior of the person 22 to the image processing unit 580 (for example, the anonymization processing unit 584).

[0185] The behavior monitoring unit 828 may monitor short-term behavior of the person 22, or may monitor long-term behavior of the person 22. Long-term behavior may be behavior lasting from 1 minute to 1 hour, from 2 minutes to 30 minutes, from 5 minutes to 30 minutes, from 2 minutes to 20 minutes, or from 5 minutes to 20 minutes.

[0186] The behavior monitoring unit 828 may store physical characteristics of the person 22. Examples of physical characteristics include facial characteristics, skeletal characteristics, and walking characteristics. This allows the behavior monitoring unit 828 to monitor the long-term behavior of the person 22. For example, even if the person 22 moves out of the field of view of the camera 154 and then returns to the field of view of the camera 154, the behavior monitoring unit 828 can continue to monitor the behavior of the person 22.

[0187] In this embodiment, when a person 22 is detected by image analysis by the monitoring target detection unit 824, the behavior monitoring unit 828 analyzes the image 10 to extract a behavior pattern of the person 22. The behavior monitoring unit 828 determines whether or not the extracted behavior pattern of the person 22 matches a predetermined pattern (for example, the above-mentioned reference pattern).

[0188] Examples of cases in which the behavior pattern of person 22 matches the reference pattern include (i) a case in which the behavior pattern of person 22 matches a first behavior pattern, which is a pattern characteristic of a person who does not intend to peek at information, or a pattern characteristic of a person who is performing an action that makes it physically difficult to peek at information compared to when the person is standing still, and (ii) a case in which the behavior pattern of person 22 matches a second behavior pattern, which is a pattern characteristic of a person who intends to peek at information. The first behavior pattern may be used as a whitelist. The second behavior pattern may be used as a blacklist.

[0189] Examples of the first behavioral pattern include (i) a pattern of shaking the face, head, or upper body from side to side or up and down while keeping the change in gaze point in the display area 142 below a predetermined threshold; (ii) a pattern of head movement speed greater than a predetermined threshold; (iii) a pattern of rapid eye movement such as saccades; (iv) a pattern in which the angle between the normal vector of the face and the normal vector of the pupil is greater than a predetermined threshold; (iv) a pattern of shaking the face from side to side; (v) a pattern in which the order of gaze movement in the display area 142 is irregular; and (vi) a pattern in which the number of body parts whose movement magnitude exceeds a predetermined standard among body parts that significantly affect gaze change exceeds a predetermined threshold. The face may be a body part other than the eyes that significantly affects gaze change. When the first behavioral pattern is detected, even if the person 22 views information displayed in the display area 142, it is unlikely that the person 22 recognizes the content of the information.

[0190] Examples of the second behavior pattern include (i) a pattern in which the length of time spent around the display area 142 is longer than a predetermined threshold, (ii) a pattern in which the length of time spent looking at the display area 142 is longer than a predetermined threshold, (iii) a pattern in which the number of times the gaze is directed at the display area 142 during a unit time is longer than a predetermined threshold, (iv) a pattern in which the ratio of the length of the period in which the gaze is directed at the display area 142 to the length of the unit time is longer than a predetermined threshold, and (v) a pattern in which, when the user terminal 120 has multiple display areas 142, the gaze is directed at two or more display areas 142. When the second behavior pattern is detected, there is a possibility that the person 22 is trying to peek at the information displayed in the display area 142.

[0191] The behavior monitoring unit 828 can determine whether the behavior pattern of the person 22 matches a predetermined pattern by using, for example, any known pattern determination technology. The behavior monitoring unit 828 may determine whether the behavior pattern of the person 22 matches a pattern registered in a blacklist. The behavior monitoring unit 828 may determine whether the behavior pattern of the person 22 matches a pattern registered in a whitelist.

[0192] The behavior monitoring unit 828 may perform the above determination process based on a rule-based algorithm, or may perform the above determination process using a trained estimation model. The behavior monitoring unit 828 may perform the above determination process by comprehensively determining the output results of multiple trained estimation models based on a rule-based algorithm. The behavior monitoring unit 828 may divide the display area 142 into multiple blocks and perform the above determination process for each block.

[0193] The behavior monitoring unit 828 may acquire the physical characteristics of the authorized user of the user terminal 120 by referring to a database that stores the physical characteristics of authorized users who have authority to use the user terminal 120. The behavior monitoring unit 828 may compare the physical characteristics of the authorized user of the user terminal 120 with the physical characteristics of the person 22 to determine whether or not the person 22 is the authorized user of the user terminal 120. The behavior monitoring unit 828 may change the criteria for determining whether or not the behavior pattern of the person 22 matches a predetermined pattern depending on whether or not the person 22 is determined to be the authorized user of the user terminal 120 and whether or not the person 22 is determined to be the authorized user of the user terminal 120.

[0194] The viewpoint monitoring unit 826 may be an example of a viewpoint estimation unit, and the behavior monitoring unit 828 may be an example of a behavior determination unit.

[0195] 9 schematically illustrates an example of the internal configuration of the anonymization processing unit 584. In this embodiment, the anonymization processing unit 584 includes a mode determination unit 820, a viewing determination unit 832, a recognition determination unit 834, an anonymization area determination unit 842, and an anonymization mode determination unit 844.

[0196] In this embodiment, the mode determination unit 820 determines the operation mode for the target of anonymization described above. As described above, examples of the operation mode for the target of anonymization include (a) a mode in which the anonymization process is performed in principle, and the anonymization process is not performed exceptionally in an area estimated to be necessary for the work of the user 20, and (b) a mode in which the anonymization process is not performed in principle, and the anonymization process is performed exceptionally in an area estimated to be viewed by a person other than the user 20 who is present near the display area 142.

[0197] In one embodiment, the mode determination unit 820 determines the operation mode based on the setting information stored in the setting information storage unit 520. In another embodiment, the mode determination unit 820 determines the operation mode based on an instruction from an administrator of the user terminal 120 or a user 20.

[0198] In this embodiment, the viewing determination unit 832 determines whether or not the manner of movement of the viewpoint of the person 22 estimated by the viewpoint monitoring unit 826 meets the above-described conditions regarding movement of the viewpoint (this may be referred to as a movement determination process). Examples of the conditions regarding movement of the viewpoint include (i) a condition that, after the viewpoint of the person 22 is estimated to be located in a start region of a specific candidate region, the viewpoint of the person 22 is estimated to be located in an end region of the specific candidate region before a period of a predetermined length has elapsed (this may be referred to as a first condition), and / or (ii) a condition that the viewpoint of the person 22 has moved along a vector from a representative point of a start region of the specific candidate region toward a representative point of an end region of the specific candidate region (this may be referred to as a second condition).

[0199] When it is determined that the manner of movement of the viewpoint matches the condition regarding the movement of the viewpoint, the viewing determination unit 832 determines, for example, to output the output image 300 in which the specific candidate area in the original image 200 is concealed. On the other hand, when it is not determined that the manner of movement of the viewpoint matches the condition regarding the movement of the viewpoint, the viewing determination unit 832 determines, for example, to output the output image 300 in which the specific candidate area in the original image 200 is not concealed.

[0200] As a result, if it is determined that the manner of movement of the viewpoint matches the condition regarding the movement of the viewpoint, the output control unit 550 or the image processing unit 580 outputs, for example, the output image 300 in which the specific candidate area in the original image 200 is concealed. On the other hand, if it is not determined that the manner of movement of the viewpoint matches the condition regarding the movement of the viewpoint, the output control unit 550 or the image processing unit 580 outputs, for example, the output image 300 in which the specific candidate area in the original image 200 is not concealed.

[0201] As a result, if there is a relatively high possibility that the information displayed in the specific candidate area will be viewed in a manner that makes the content of the information easily recognizable if the information is not anonymized, the anonymization process is executed. As a result, the information displayed in the specific candidate area is anonymized before the person 22 recognizes the content of the information, thereby preventing leakage of the information. On the other hand, according to this embodiment, even if the person 22 views the specific candidate area, the anonymization process is not executed if the content of the information is not viewed in a manner that makes it easily recognizable. This suppresses unnecessary anonymization processes, improving the user experience for the user 20.

[0202] In this embodiment, the recognition determination 834 determines whether or not the person 22 recognizes the content of the information displayed in the display area 142 if no concealment is applied (this may be referred to as a recognition determination process). The recognition determination 834 may (i) decide whether or not to execute the recognition determination process depending on the determination result of the movement determination process by the viewing determination unit 832, or (ii) execute the recognition determination process in parallel with the movement determination process by the viewing determination unit 832.

[0203] For example, if it is determined that the manner of movement of the viewpoint matches the conditions regarding the movement of the viewpoint, the recognition determination 834 executes the above-described recognition determination process. On the other hand, if it is not determined that the manner of movement of the viewpoint matches the conditions regarding the movement of the viewpoint, the recognition determination 834 does not need to execute the above-described recognition determination process.

[0204] In one embodiment, the recognition determination 834 determines, based on the monitoring results of the behavior monitoring unit 828, whether the person 22 would recognize the content of the information displayed in the display area 142 if no anonymization was applied. For example, if the behavior monitoring unit 828 determines that the behavior pattern of the person 22 matches a reference pattern, the recognition determination 834 determines that the person 22 would recognize the content of the information displayed in the display area 142 if no anonymization was applied.

[0205] In another embodiment, the recognition determination unit 834 derives the degree of likelihood (sometimes referred to as the degree of recognition) that the person 22 would recognize the content of the information displayed in the display area 142 if no anonymization was applied, based on the monitoring results of the behavior monitoring unit 828. For example, when the monitoring target detection unit 824 detects the person 22, the recognition determination unit 834 derives the degree of recognition based on the behavior pattern of the person 22 obtained by the behavior monitoring unit 828 analyzing the surrounding image. The degree of recognition may be expressed by a continuous range of values ​​between a predetermined lower limit and an upper limit, or may be expressed by a gradual range. For example, when the degree of recognition of the person 22 exceeds the predetermined degree, the recognition determination unit 834 determines that the person 22 would recognize the content of the information displayed in the display area 142 if no anonymization was applied.

[0206] If it is determined that the person 22 would recognize the content of the information displayed in the display area 142 if the information were not concealed, the recognition decision 834 determines, for example, to output the output image 300 in which a specific candidate area in the original image 200 has been concealed. On the other hand, if it is not determined that the person 22 would recognize the content of the information displayed in the display area 142 if the information were not concealed, the recognition decision 834 determines, for example, to output the output image 300 in which the specific candidate area in the original image 200 has not been concealed.

[0207] According to one embodiment, the recognition determination 834 may (a) determine to output the output image 300 in which a specific candidate region in the original image 200 is concealed when it is determined that the behavior pattern of the person 22 matches a predetermined pattern and the manner of movement of the viewpoint matches a predetermined condition. On the other hand, (b) when it is not determined that the behavior pattern of the person 22 matches a predetermined pattern or the manner of movement of the viewpoint matches a predetermined condition, the recognition determination 834 may determine to output the output image 300 in which the specific candidate region in the original image 200 is not concealed.

[0208] As a result, (a) if it is determined that the behavior pattern of the person 22 matches a predetermined pattern and that the manner of movement of the viewpoint matches a predetermined condition, the output control unit 550 or the image processing unit 580 outputs, for example, an output image 300 in which a specific candidate area in the original image 200 is concealed. On the other hand, (b) if it is not determined that the behavior pattern of the person 22 matches a predetermined pattern or that the manner of movement of the viewpoint matches a predetermined condition, the output control unit 550 or the image processing unit 580 outputs, for example, an output image 300 in which the above-mentioned specific candidate area in the original image 200 is not concealed.

[0209] According to another embodiment, the recognition determination 834 may determine to output the output image 300 in which the specific candidate region in the original image 200 is anonymized (a) when it is determined that the degree of recognition of the person 22 exceeds a predetermined level and that the manner of movement of the viewpoint matches a predetermined condition. On the other hand, (b) when the degree of recognition of the person 22 does not exceed a predetermined level or when it is not determined that the manner of movement of the viewpoint matches a predetermined condition, the recognition determination 834 may determine to output the output image 300 in which the specific candidate region in the original image 200 is not anonymized.

[0210] As a result, (a) if it is determined that the degree of recognition of the person 22 exceeds a predetermined level and that the manner of movement of the viewpoint matches a predetermined condition, the output control unit 550 or the image processing unit 580 outputs, for example, the output image 300 in which a specific candidate area in the original image 200 is concealed. On the other hand, (b) if the degree of recognition of the person 22 does not exceed a predetermined level or if it is not determined that the manner of movement of the viewpoint matches a predetermined condition, the output control unit 550 or the image processing unit 580 outputs, for example, the output image 300 in which the specific candidate area in the original image 200 is not concealed.

[0211] In this embodiment, the concealment area determination unit 842 determines an area to be concealed in the original image 200 (sometimes referred to as a concealment area). As a result, for example, an output image 300 can be generated in which an object 330 is arranged in the concealment area of ​​the original image 200. As described above, the output image 300 may be generated by the image processing unit 580 or the output image generation unit 588. The output image 300 may also be generated by the concealment processing unit 584 (for example, the concealment area determination unit 842 or the concealment mode determination unit 844).

[0212] In this embodiment, the concealment area determination unit 842 determines at least one candidate area from among one or more candidate areas as a concealment area based on the manner of movement of the viewpoint of the person 22 estimated by the viewpoint monitoring unit 826. For example, if the manner of movement of the viewpoint satisfies at least one of the first condition and the second condition described above, the concealment area determination unit 842 determines at least the specific candidate area described above as a concealment area.

[0213] The concealment area determination unit 842 may determine whether to include other candidate areas located near the above-mentioned specific candidate area in the concealment area based on the value of the information included in the above-mentioned specific candidate area (e.g., candidate area 260). The concealment area determination unit 842 obtains the value of the information included in the above-mentioned specific candidate area, for example, by referring to the value map generated by the value map generation unit 578. The concealment mode determination unit 844 determines other candidate areas to be included in the concealment area such that, for example, the greater the value of the information, the greater the maximum value of the distance between the other candidate areas included in the concealment area and the above-mentioned specific candidate area.

[0214] In this embodiment, the concealment mode determination unit 844 determines the concealment mode to be applied to each of one or more concealment regions. As described above, examples of the concealment mode include the type or method of concealment processing, the strength of concealment processing, etc.

[0215] The anonymization mode determination unit 844 determines the anonymization mode for each anonymization area based on, for example, the value of the information included in each anonymization area. The anonymization mode determination unit 844 may obtain the value of the information included in each anonymization area by referring to the value map generated by the value map generation unit 578.

[0216] The anonymization mode determination unit 844 determines the anonymization mode for each anonymization area so that, for example, the more valuable the information, the lower the visibility of the anonymization method is adopted.The anonymization mode determination unit 844 determines the anonymization mode for each anonymization area so that, for example, the more valuable the information, the stronger the anonymization process is.

[0217] The viewing determination unit 832 may be an example of a movement determination unit. The recognition determination unit 834 may be an example of a behavior determination unit or a recognition degree derivation unit. The concealment area determination unit 842 may be an example of a target area determination unit or a display image generation unit. The specific candidate area may be an example of a first candidate area.

[0218] 10 schematically illustrates an example of information processing in the viewing determination unit 832. In this embodiment, the movement determination process in the viewing determination unit 832 will be described in detail using an example in which the person 22 views the candidate area 260 described in relation to FIG.

[0219] As described above, the viewing determination unit 832 determines whether the manner of movement of the viewpoint 50 of the person 22 estimated by the viewpoint monitoring unit 826 matches the conditions related to the movement of the viewpoint. Also, the vector 40 of the candidate region 260 is a vector directed from the representative point 272 of the start region 262 of the candidate region 260 to the representative point 274 of the end region 264 of the candidate region 260.

[0220] In this embodiment, the viewing determination unit 832 determines that the manner of movement of the viewpoint 50 of the person 22 meets the condition regarding the viewpoint movement when the viewpoint 50 of the person 22 moves above the candidate area 260 in the direction from the start area 262 to the end area 264. However, as shown in Fig. 10 , the vector 40 is represented by a straight line. On the other hand, the movement history of the viewpoint 50 of the person 22 is represented by a curved line.

[0221] Therefore, according to one embodiment, if the viewpoint 50 of the person 22 is estimated to be located in the start region 262 of the candidate region 260, which is an example of a specific candidate region, and then the viewpoint 50 of the person 22 is estimated to be located in the end region 264 of the candidate region 260 before a period of a predetermined length has elapsed, the viewing determination unit 832 determines that the viewpoint 50 of the person 22 has moved along the vector 40. In this case, the viewing determination unit 832 also determines that the viewpoint 50 of the person 22 has moved above the candidate region 260 in the direction from the start region 262 to the end region 264.

[0222] In another embodiment, if the error between the movement history of the viewpoint 50 of the person 22 and the vector 40 is smaller than a predetermined value, the viewing determination unit 832 determines that the viewpoint 50 of the person 22 has moved along the vector 40. In this case, the viewing determination unit 832 also determines that the viewpoint 50 of the person 22 has moved above the candidate region 260 in the direction from the start region 262 to the end region 264.

[0223] In yet another embodiment, if the error between the moving average line of the estimated position of the viewpoint 50 of the person 22 and the vector 40 is smaller than a predetermined value, the viewing determination unit 832 determines that the viewpoint 50 of the person 22 has moved along the vector 40. In this case, the viewing determination unit 832 also determines that the viewpoint 50 of the person 22 has moved above the candidate region 260 in the direction from the start region 262 to the end region 264. 11 shows an example of information processing in the information leakage prevention support unit 586. According to this embodiment, an overview of processing for encouraging the user 22 to take a specific action to prevent information leakage will be described using an example in which a person 24 other than the user 22 is present behind the user 22 of the user terminal 120.

[0224] In this embodiment, the person 24 is located to the left and behind the user 22. Therefore, when the user 22 rotates the user terminal 120 horizontally clockwise, the person 24 can easily view the display area 142. On the other hand, when the user 22 rotates the user terminal 120 horizontally counterclockwise, it becomes difficult for the person 24 to view the display area 142.

[0225] Therefore, according to the present embodiment, the leakage prevention support unit 586 controls the output image generation unit 588 to display a guidance image 1118 in a part of the display area 142, thereby controlling the behavior of the user 22. Specifically, the leakage prevention support unit 586 adjusts the display position of the guidance image 1118 to prompt the user 22 to rotate the user terminal 120 horizontally counterclockwise.

[0226] The leakage prevention support unit 586 first acquires data of the image captured by the camera 154. The leakage prevention support unit 586 analyzes the image captured by the camera 154 and determines whether two or more people are present around the display area 142. If two or more people are present around the display area 142, the leakage prevention support unit 586 identifies the relative positional relationship between the people. Based on the relative positional relationship between the objects, the leakage prevention support unit 586 determines the rotation direction of the user terminal 120 that will further suppress information leakage. As described above, in this embodiment, information leakage is further suppressed by rotating horizontally counterclockwise.

[0227] The leakage prevention support unit 586 controls the output image generation unit 588 to output an image 1102 to the display area 142. In the image 1102, a part of the right side of the image output area 214 is covered by a guidance image 1118. This enables the leakage prevention support unit 586 to prompt the user 22 to rotate the user terminal 120 counterclockwise.

[0228] When the user 22 rotates the user terminal 120 counterclockwise, the leakage prevention support unit 586 controls the output image generation unit 588 to output the image 1104 in the display area 142. The guidance image 1118 is not displayed in the image 1104, and the user 22 can continue working as usual.

[0229] On the other hand, when the user 22 rotates the user terminal 120 clockwise, the leakage prevention support unit 586 controls the output image generation unit 588 to output the image 1106 in the display area 142. A larger portion of the image 1106 is covered by the guidance image 1118 compared to the image 1102. This allows the leakage prevention support unit 586 to prompt the user 22 to rotate the user terminal 120 counterclockwise.

[0230] 12 shows an example of a computer 3000 in which multiple aspects of the present invention may be embodied in whole or in part. At least a part of the information leakage prevention system 100 may be realized by the computer 3000. For example, at least a part of the user terminal 120 is realized by the computer 3000 or a part thereof. For example, at least a part of the user terminal 120, or the control device 170, is realized by the computer 3000 or a part thereof.

[0231] A program installed on the computer 3000 can cause the computer 3000 to function as or perform operations associated with an apparatus according to an embodiment of the present invention or one or more "parts" of the apparatus, and / or to perform a process or steps of the process according to an embodiment of the present invention. Such a program can be executed by the CPU 3012 to cause the computer 3000 to perform specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0232] A computer 3000 according to this embodiment includes a CPU 3012, a RAM 3014, a GPU 3016, and a display device 3018, which are interconnected by a host controller 3010. The computer 3000 also includes input / output units such as a communication interface 3022, a hard disk drive 3024, a DVD-ROM drive 3026, and an IC card drive, which are connected to the host controller 3010 via an input / output controller 3020. The computer also includes legacy input / output units such as a ROM 3030 and a keyboard 3042, which are connected to the input / output controller 3020 via an input / output chip 3040.

[0233] The CPU 3012 operates according to programs stored in the ROM 3030 and RAM 3014, thereby controlling each unit. The GPU 3016 acquires image data generated by the CPU 3012 into a frame buffer or the like provided in the RAM 3014 or into the GPU 3016 itself, and causes the image data to be displayed on the display device 3018.

[0234] The communication interface 3022 communicates with other electronic devices via a network. The hard disk drive 3024 stores programs and data used by the CPU 3012 in the computer 3000. The DVD-ROM drive 3026 reads programs or data from the DVD-ROM 3001 and provides the programs or data to the hard disk drive 3024 via the RAM 3014. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0235] The ROM 3030 stores therein a boot program or the like that is executed by the computer 3000 upon activation, and / or programs that depend on the hardware of the computer 3000. The input / output chip 3040 may also connect various input / output units to the input / output controller 3020 via a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0236] The programs are provided by a computer-readable storage medium such as a DVD-ROM 3001 or an IC card. The programs are read from the computer-readable storage medium, installed in the hard disk drive 3024, RAM 3014, or ROM 3030, which are also examples of computer-readable storage media, and executed by the CPU 3012. The information processing described in these programs is read by the computer 3000, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing operations or processing of information in accordance with the use of the computer 3000.

[0237] For example, when communication is performed between computer 3000 and an external device, CPU 3012 may execute a communication program loaded into RAM 3014 and instruct communication interface 3022 to perform communication processing based on the processing described in the communication program. Under the control of CPU 3012, communication interface 3022 reads transmission data stored in a transmission buffer area provided in RAM 3014, hard disk drive 3024, DVD-ROM 3001, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes received data received from the network to a reception buffer area or the like provided on the recording medium.

[0238] Furthermore, CPU 3012 may cause all or a necessary portion of a file or database stored on an external recording medium such as hard disk drive 3024, DVD-ROM drive 3026 (DVD-ROM 3001), IC card, etc. to be read into RAM 3014, and may perform various types of processing on the data on RAM 3014. CPU 3012 may then write back the processed data to the external recording medium.

[0239] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and subjected to information processing. The CPU 3012 may perform various types of processing on data read from the RAM 3014, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 3014. The CPU 3012 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries, each having an attribute value of a first attribute associated with an attribute value of a second attribute, are stored on the recording medium, the CPU 3012 may search for an entry whose attribute value of the first attribute matches a specified condition from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0240] The above-described programs or software modules may be stored in a computer-readable storage medium on or near the computer 3000. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable storage medium, thereby providing the above-described programs to the computer 3000 via the network.

[0241] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. Furthermore, details described for a particular embodiment can be applied to other embodiments to the extent that they are not technically inconsistent. It is apparent from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0242] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order.

[0243] The present specification describes, for example, the following: (Item A-1) An information processing device that generates a display image to be displayed in a display area of ​​an image display device based on a generated image generated by an image generation device, a generated image acquisition unit that acquires the generated image; a display image generation unit that generates the display image based on the generated image; a peripheral image acquisition unit that acquires a peripheral image that is an image captured by an imaging device that captures an image of the periphery of the display area; a monitoring target detection unit that analyzes the peripheral image and detects a monitoring target, which is a monitoring target device that is an imaging device facing an imaging direction toward the display area; Equipped with the display image generation unit determines, when the monitored device is detected, to output the display image in which a part of the generated image is concealed. Information processing device. (Item A-2) the display image generation unit determines whether to output the display image in which a portion of the generated image is concealed, based on the shooting direction and a silhouette of a user of the image generation device. The information processing device according to item A-1. (Item B-1) An information processing device that generates a display image to be displayed in a display area of ​​an image display device based on a generated image generated by an image generation device, a generated image acquisition unit that acquires the generated image; a display image generation unit that generates the display image based on the generated image; a peripheral image acquisition unit that acquires a peripheral image that is an image captured by an imaging device that captures an image of the periphery of the display area; a monitoring target detection unit that analyzes the peripheral image and detects a monitoring target who is directing his / her gaze toward the display area and is a person different from a user of the image display device; a viewpoint estimation unit that estimates a viewpoint of the person to be monitored in the display area when the person to be monitored is detected; Equipped with the display image generation unit determines whether to output the display image in which a portion of the generated image is concealed, based on the viewpoint and the silhouette of the user. Information processing device. (Item C-1) An information processing device that generates a display image to be displayed in a display area of ​​an image display device based on a generated image generated by an image generation device, a generated image acquisition unit that acquires the generated image; a display image generation unit that generates the display image based on the generated image; a peripheral image acquisition unit that acquires a peripheral image that is an image captured by an imaging device that captures an image of the periphery of the display area; a monitoring target detection unit that analyzes the peripheral image and detects a monitoring target that is at least one of (i) a monitoring target person who is directing his / her gaze toward the display area and who is a person different from the user of the image display device, and (ii) a monitoring target device that is an imaging device that is pointing its imaging direction toward the display area; a viewpoint estimation unit that, when the monitoring target is detected, estimates at least one of (i) a viewpoint of the monitoring target in the display area and (ii) an imaging position of the monitoring target device in the display area; Equipped with the display image generation unit determines whether to output the display image in which a part of the generated image is concealed, based on at least one of the viewpoint and the imaging position and a silhouette of the user. Information processing device. (Item C-2) a movement determination unit that determines whether or not a movement mode of at least one of the viewpoint and the imaging position estimated by the viewpoint estimation unit matches a predetermined condition; The information processing device according to item C-1. (Item C-3) the viewpoint estimation unit estimates a viewpoint of the monitoring target in the display area when the monitoring target is detected; the movement determination unit determines whether or not the manner of movement of the viewpoint estimated by the viewpoint estimation unit matches a predetermined condition; the display image generation unit determines whether to output the display image in which a portion of the generated image is concealed, based on the viewpoint and the silhouette of the user. The information processing device according to item C-1 or C-2. (Item C-4) the display image generation unit determines to output the display image in which a part of the generated image is concealed when (i) it is determined that the manner of movement of the viewpoint matches the predetermined condition, or (ii) when the monitored device is detected; The information processing device according to any one of items C-1 to C-3. [Explanation of symbols]

[0244] 10 images 20 users 22 People 24 People 30 candidate data 40 Vector 50 viewpoints 100 Information Leakage Prevention System 120 User Terminals 130 cabinet 140 Display device 142 Display area 150 Environmental Sensors 152 Distance Sensor 154 Camera 156 Radio wave detector 160 Input Devices 170 Control device 200 original images 210 Window 212 Control Area 214 Image Output Area 216 Presentation information 222 objects 224 objects 230 Window 232 Control Area 234 Image Output Area 236 Presentation information 240 windows 242 Control Area 244 Image Output Area 246 Presentation information 260 candidate area 262 Start area 264 End area 272 representative points 274 representative points 300 output images 326 objects 328 objects 330 objects 520 Setting information storage section 532 Program Control Unit 534 Window control unit 540 Input control section 550 Output control section 552 Perimeter Surveillance Department 560 Original Image Generation Unit 574 Gaze information acquisition unit 576 Operation information acquisition section 578 Value Map Generation Unit 580 Image Processing Unit 584 Concealment Processing Unit 586 Leakage Prevention Support Department 588 Output Image Generation Unit 612 Document File Acquisition Unit 614 Accompanying Data Acquisition Unit 616 Area Data Assignment Unit 620 Accompanying data generation unit 622 Candidate Data Determination Unit 624 Value Determination Department 700 Data Tables 722 Candidate Data ID 724 position 726 Contents 728 Value 732 range 820 Mode determination unit 822 Peripheral image acquisition unit 824 Monitoring target detection unit 826 Viewpoint Monitoring Department 828 Behavioral Monitoring Department 832 Viewing Determination Department 834 Recognition judgment 842 Concealment area determination unit 844 Concealment mode determination unit 1102 images 1104 images 1106 images 1118 Guidance Image 3000 computers 3001 DVD-ROM 3010 host controller 3012 CPU 3014 RAM 3016 GPU 3018 Display Device 3020 Input / Output Controller 3022 Communication Interface 3024 hard disk drive 3026 DVD-ROM drive 3030 ROM 3040 I / O chip 3042 keyboard

Claims

1. An information processing device that generates a display image to be displayed in a display area of ​​an image display device based on a generated image generated by an image generating device, A generated image acquisition unit that acquires the generated image; a display image generating unit that generates the display image based on the generated image; a surrounding image acquisition unit that acquires a surrounding image that is an image captured by an imaging device that captures an image of the periphery of the display area; a monitoring target detection unit that analyzes the peripheral image to detect a monitoring target that is at least one of (i) a monitoring target person who is directing his / her gaze toward the display area and who is a person other than a user of the image display device, and (ii) a monitoring target device that is an imaging device whose imaging direction is directed toward the display area; a viewpoint estimation unit that estimates a viewpoint of the person to be monitored in the display area when the person to be monitored is detected; a movement determination unit that determines whether or not the manner in which the viewpoint is moved estimated by the viewpoint estimation unit matches a predetermined condition; Equipped with the display image generating unit determines to output the display image in which a part of the generated image is concealed when (i) it is determined that the manner of movement of the viewpoint matches the predetermined condition, or (ii) the monitored device is detected; Information processing device.

2. a candidate information acquisition unit that acquires candidate information, the candidate information being information that has been created in advance based on the contents of an electronic file used in a process for the image generation device to generate at least a part of the generated image, and being information for identifying one or more candidate data that are data in the electronic file that may be subject to the anonymization; a candidate area determination unit that determines, in the display area, one or more candidate areas in which each of the one or more candidate data items specified by the candidate information is to be displayed; a target area determination unit that determines, based on a state of movement of the viewpoint, at least one candidate area among the one or more candidate areas as a target area that is an area in the display area to which the concealment is to be performed; Further equipped with each of the one or more candidate data is data used by the image generating device to generate a different portion of the generated image; the display image generation unit determines to output the display image in which the target region determined by the target region determination unit is concealed. The information processing device according to claim 1 .

3. Each of the one or more candidate regions comprises: a start region located at the start position of each candidate region; an end region disposed at the end position of each candidate region; Including, a range of the end region in the display region is different from a range of the start region in the display region; the candidate information includes representative point information that is information for identifying a representative point of the start region of each of the one or more candidate regions and a representative point of the end region of each of the one or more candidate regions, The predetermined condition includes at least one of: (i) a first condition that, after the viewpoint is estimated to be located in the start region of a first candidate region among the one or more candidate regions, the viewpoint is estimated to be located in the end region of the first candidate region before a period of time having a predetermined length has elapsed; and (ii) a second condition that the viewpoint is estimated to have moved along a vector from a representative point of the start region of the first candidate region toward a representative point of the end region of the first candidate region. The information processing device according to claim 2 .

4. the target region determination unit determines at least the first candidate region as the target region when a manner of movement of the viewpoint satisfies at least one of the first condition and the second condition. The information processing device according to claim 3 .

5. and a behavior determination unit that, when the monitoring target detection unit detects the monitoring target, determines whether or not a behavior pattern of the monitoring target obtained by analyzing the surrounding image matches a predetermined pattern, The display image generating unit is (a) when it is determined that the behavior pattern of the surveillance target person matches the predetermined pattern and the manner of movement of the viewpoint matches the predetermined condition, determining to output the display image in which a portion of the generated image is concealed; (b) when it is not determined that the behavior pattern of the monitored person matches the predetermined pattern, or when it is not determined that the manner of movement of the viewpoint matches the predetermined condition, determining to output the generated image as the display image; The information processing device according to claim 1 .

6. a recognition degree derivation unit that, when the surveillance target detection unit detects the surveillance target, derives a recognition degree indicating a degree of possibility that the surveillance target will recognize the content of the information displayed in the display area, unless concealment is performed, based on a behavior pattern of the surveillance target obtained by analyzing the surrounding image, The display image generating unit is (a) when it is determined that the degree of recognition exceeds a predetermined degree and that the manner of movement of the viewpoint matches the predetermined condition, determining to output the display image in which a portion of the generated image is concealed; (b) determining that the generated image is to be output as the display image when the degree of recognition does not exceed a predetermined degree, or when it is not determined that the manner of movement of the viewpoint matches the predetermined condition; The information processing device according to claim 1 .

7. A program for causing a computer to function as the information processing device according to any one of claims 1 to 6.

8. 1. A method for generating a display image to be displayed on a display area of ​​an image display device based on a generated image generated by an image generating device, comprising: A generated image acquisition step in which a computer acquires the generated image; a display image generating step in which the computer generates the display image based on the generated image; a peripheral image acquisition step in which the computer acquires a peripheral image which is an image captured by an imaging device that captures an image of the periphery of the display area; a monitoring target detection step in which the computer analyzes the peripheral image to detect a monitoring target who is directing his / her gaze toward the display area and is a person other than a user of the image display device; a viewpoint estimating step in which, when the monitoring target person is detected, the computer estimates a viewpoint of the monitoring target person in the display area; a movement determination step in which the computer determines whether or not the manner of movement of the viewpoint estimated in the viewpoint estimation step matches a predetermined condition; having The display image generating step includes a step of outputting the display image in which a part of the generated image is concealed when it is determined that the manner of movement of the viewpoint matches the predetermined condition, method.

Images