Information processing device, program and method

Abstract

An information processing device, a program, and a method for preventing information leakage due to peeping are provided. [Solution] In the information leakage prevention system 100, the control device 170 of the user terminal 120 includes a generated image acquisition unit that acquires a generated image, a display image generation unit that generates a display image, a peripheral image acquisition unit that acquires a peripheral image, a monitoring target detection unit that analyzes the peripheral image to detect monitoring targets that are at least one of a monitored person 22 and a monitored device (an imaging device present behind the user 20 of the user terminal 120), a viewpoint estimation unit that estimates the viewpoint of the monitored person in the display area when the monitored person is detected, and a movement determination unit that determines whether or not the manner of movement of the viewpoint estimated by the viewpoint estimation unit matches a predetermined condition. When it is determined that the manner of movement of the viewpoint matches the predetermined condition or when the monitored device is detected, the display image generation unit decides to output a display image in which a part of the generated image is concealed.

Description

[Technical field]

[0001] The present invention relates to an information processing device, a program, and a method. [Background technology]

[0002] Patent Document 1 discloses an information display device that automatically sets itself to prevent others from peeking when it is determined that there is a risk of others peeking. Patent Document 2 discloses an electronic device that performs anti-peeking control when it detects the gaze of someone other than the user. Patent Document 3 discloses a technology that acquires gaze information of a user's gaze on a screen image for each user and displays which display area in the screen image the user has gazed on. Patent Documents 4 and 5 disclose non-contact gaze detection technology. (Prior art document) (Patent Documents) (Patent Document 1) JP 2012-129701 A (Patent Document 2) Patent No. 6474921 (Patent Document 3) Patent No. 6485522 (Patent Document 4) JP 2021-043869 A (Patent Document 5) JP 2021-077265 A Summary of the Invention [Means for solving the problem]

[0003] In a first aspect of the present invention, an information processing device is provided. The information processing device generates a display image to be displayed in a display area of ​​an image display device, for example, based on a generated image generated by an image generating device. The information processing device includes, for example, a generated image acquisition unit that acquires a generated image. The information processing device includes, for example, a display image generation unit that generates a display image based on the generated image. The information processing device includes, for example, a peripheral image acquisition unit that acquires a peripheral image that is an image captured by an imaging device that captures the periphery of the display area. The information processing device includes, for example, a monitoring target detection unit that analyzes the peripheral image to detect a monitoring target that is at least one of (i) a monitoring target person who is a person who is directing his / her gaze toward the display area and is different from a user of the image display device, and (ii) a monitoring target device that is an imaging device that is facing the shooting direction toward the display area. The information processing device includes, for example, a viewpoint estimation unit that estimates a viewpoint of the monitoring target person in the display area when the monitoring target person is detected. The information processing device includes, for example, a movement determination unit that determines whether or not a manner of movement of the viewpoint estimated by the viewpoint estimation unit matches a predetermined condition. In the above-mentioned information processing device, the display image generation unit decides to output a display image in which a portion of the generated image is concealed, for example, when (i) it is determined that the manner of movement of the viewpoint matches a predetermined condition, or (ii) a monitored device is detected.

[0004] The information processing device may include a candidate information acquisition unit that acquires candidate information, which is information created in advance based on the contents of an electronic file used in a process for the image generating device to generate at least a part of a generated image, and is information for identifying one or more candidate data that may be subject to anonymization in the electronic file. The information processing device may include a candidate area determination unit that determines one or more candidate areas in the display area in which each of the one or more candidate data identified by the candidate information is displayed. The information processing device may include a target area determination unit that determines at least one of the one or more candidate areas as a target area in which anonymization is performed in the display area based on a manner of movement of the viewpoint. In the information processing device, each of the one or more candidate data may be data used by the image generating device to generate a different part of the generated image. In the information processing device, the display image generation unit may determine to output a display image in which the target area determined by the target area determination unit is anonymized.

[0005] In the above information processing device, each of the one or more candidate areas may include a start area arranged at the start position of each candidate area and an end area arranged at the end position of each candidate area. In the above information processing device, the range of the end area in the display area may be different from the range of the start area in the display area. In the above information processing device, the candidate information may include a representative point of each of the start areas of the one or more candidate areas and representative point information that is information for identifying a representative point of each of the end areas of the one or more candidate areas. In the above information processing device, the predetermined condition may include at least one of (i) a first condition that the viewpoint is estimated to be located in the end area of ​​the first candidate area before a period having a predetermined length has elapsed after the viewpoint is estimated to be located in the start area of ​​a first candidate area among the one or more candidate areas, and (ii) a second condition that the viewpoint is estimated to have moved along a vector from the representative point of the start area of ​​the first candidate area toward the representative point of the end area of ​​the first candidate area. In the above information processing device, the target region determining section may determine at least the first candidate region as the target region when the manner of movement of the viewpoint matches at least one of a first condition and a second condition.

[0006] The information processing device may include a behavior determination unit that, when the monitoring target detection unit detects a monitoring target, determines whether or not a behavior pattern of the monitoring target obtained by analyzing the surrounding image matches a predetermined pattern. In the information processing device, the display image generation unit may determine to output a display image in which a portion of the generated image is concealed when (a) it is determined that the behavior pattern of the monitoring target matches a predetermined pattern and the manner of movement of the viewpoint matches a predetermined condition. In the information processing device, the display image generation unit may determine to output the generated image as a display image when (b) it is not determined that the behavior pattern of the monitoring target matches a predetermined pattern or the manner of movement of the viewpoint matches a predetermined condition.

[0007] The information processing device may include a recognition degree derivation unit that, when the monitoring target detection unit detects a monitoring target, derives a recognition degree indicating a degree of possibility that the monitoring target would recognize the content of the information displayed in the display area if the information is not concealed, based on a behavior pattern of the monitoring target obtained by analyzing the surrounding image. In the information processing device, the display image generation unit may determine to output a display image in which a part of the generated image is concealed, when (a) it is determined that the recognition degree exceeds a predetermined degree and the manner of movement of the viewpoint matches a predetermined condition. In the information processing device, the display image generation unit may determine to output the generated image as a display image, when (b) the recognition degree does not exceed a predetermined degree or the manner of movement of the viewpoint is not determined to match a predetermined condition.

[0008] In a second aspect of the present invention, a method for generating a display image to be displayed in a display area of ​​an image display device based on a generated image generated by an image generating device is provided. The above method includes, for example, a generated image acquisition step in which a computer acquires a generated image. The above method includes, for example, a display image generation step in which a computer generates a display image based on the generated image. The above method includes, for example, a peripheral image acquisition step in which a computer acquires a peripheral image that is an image captured by an imaging device that captures the periphery of the display area. The above method includes, for example, a monitoring target detection step in which a computer analyzes the peripheral image to detect a monitoring target who is a person who is directing his / her gaze toward the display area and is a person different from a user of the image display device. The above method includes, for example, a viewpoint estimation step in which a computer estimates a viewpoint of the monitoring target in the display area when a monitoring target is detected. The above method includes, for example, a movement determination step in which a computer determines whether or not a mode of movement of the viewpoint estimated in the viewpoint estimation step matches a predetermined condition. In the above method, the display image generation step includes, for example, a step in which a computer outputs a display image in which a part of the generated image is concealed when it is determined that a mode of movement of the viewpoint matches a predetermined condition.

[0009] In a third aspect of the present invention, a program is provided. The program may be a program for causing a computer to function as any of the information processing devices according to the first embodiment. The program may be a program for causing a computer to execute the method according to the second embodiment. A computer-readable medium storing the program may be provided. The computer-readable medium may be a non-transitory computer-readable medium. The computer-readable medium may be a computer-readable recording medium.

[0010] The above summary of the invention does not list all of the necessary features of the present invention. Also, subcombinations of these features may also be inventions. [Brief description of the drawings]

[0011] [Figure 1] 1 shows an example of a system configuration of an information leakage prevention system 100. [Diagram 2] An example of an original image 200 is shown diagrammatically. [Diagram 3] An example of an output image 300 is shown diagrammatically. [Figure 4] 13 illustrates an example of information processing in the control device 170. [Diagram 5] 2 shows an example of the internal configuration of the control device 170. [Figure 6] 13 shows an example of the internal structure of a value map generator 578. [Figure 7] 7 shows a schematic diagram of an example of the data structure of a data table 700. [Figure 8] 13 shows an example of an internal configuration of the periphery monitoring unit 552. [Figure 9] 13 shows an example of the internal configuration of the concealment processing unit 584. [Figure 10] 13 illustrates an example of information processing in a viewing determination unit 832. [Figure 11] 13 illustrates an example of information processing in the leakage prevention support unit 586. [Figure 12]An example of the system configuration of a computer 3000 is shown in schematic form. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The present invention will be described below through the embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. In addition, in the drawings, the same reference numbers may be used for the same or similar parts, and duplicated explanations may be omitted. In addition, a person skilled in the art who has come into contact with the description of this specification can understand that technical matters described in one embodiment may be applied to other embodiments to the extent that there is no technical contradiction.

[0013] (Overview of information leakage prevention system 100) An overview of the information leakage prevention system 100 will be described with reference to Figures 1, 2, and 3. Figure 1 illustrates an example of a system configuration of the information leakage prevention system 100. In this embodiment, the information leakage prevention system 100 can conceal a part of information presented to a user (for example, a user 20 described later).

[0014] The information leakage prevention system 100 conceals a part of an image displayed by the user terminal 120 in response to an instruction from, for example, an administrator or a user of the user terminal 120 managed by the information leakage prevention system 100. The above instruction may be an instruction to register or update a setting related to concealment. The above instruction may be stored in the user terminal 120 as a setting related to concealment.

[0015] The administrator or user of the user terminal 120 configures the user terminal 120 so that, for example, a program for concealing a part of an image displayed by the user terminal 120 is launched when the user terminal 120 is started or immediately after the user terminal 120 is started. The administrator or user of the user terminal 120 may configure the user terminal 120 so that a part of an image displayed by the user terminal 120 is concealed when the length of time since the user terminal 120 last received an instruction from the administrator or user becomes greater than a predetermined value.

[0016] In this embodiment, the information leakage prevention system 100 will be described in detail with an example in which the information leakage prevention system 100 (i) first generates an original image 200 that has not been subjected to an anonymization process, and (ii) then anonymizes at least a part of the original image 200 to generate an output image 300 to be presented to a user. Fig. 2 shows an example of the original image 200. Fig. 3 shows an example of the output image 300.

[0017] 1, in this embodiment, the information leakage prevention system 100 includes, for example, a user terminal 120. In this embodiment, the user terminal 120 includes, for example, a housing 130, a display device 140, an environmental sensor 150, an input device 160, and a control device 170. In this embodiment, the display device 140 includes, for example, a display area 142. In this embodiment, the environmental sensor 150 includes a distance sensor 152, a camera 154, and a radio wave detector 156.

[0018] In FIG. 1, an image 10 shows an example of an image captured by the camera 154. As shown in the image 10, in this embodiment, a case where a person 22 and a person 24 different from the user 20 are present behind the user 20 of the user terminal 120 is taken as an example, and an embodiment of the information leakage prevention system 100 is described in detail. In this embodiment, for the purpose of facilitating understanding of the details of each part of the information leakage prevention system 100 and / or the details of information processing in the information leakage prevention system 100, a case where the user 20 inputs various instructions to the user terminal 120, the person 22 directs his / her gaze to the display area 142 of the user terminal 120, and the person 24 does not direct his / her gaze to the display area 142 of the user terminal 120 is taken as an example, and an embodiment of the information leakage prevention system 100 is described in detail.

[0019] (Overview of user terminal 120) In this embodiment, the user terminal 120 outputs various types of information to the user 20. The user terminal 120 may output information by an image or by sound. The user terminal 120 may accept input of information from the user 20. The user terminal 120 is configured to be able to execute various types of information processing.

[0020] In this embodiment, the details of the user terminal 120 will be described by taking as an example a case where the user terminal 120 outputs an image. For example, a user 20 of the user terminal 120 inputs, via the input device 160, instructions to the user terminal 120 for the BIOS, the operating system (sometimes referred to as an OS), or an application program (sometimes referred to as a program as a general term for these) that runs on the user terminal 120. The user terminal 120 executes information processing according to the instructions, and displays an image showing the result of the information processing in the display area 142.

[0021] In this embodiment, the user terminal 120 generates the output image 300 by concealing, for example, a part of the original image 200 generated by a program running on the user terminal 120. The user terminal 120 displays the output image 300 in the display area 142 of the display device 140. In this way, the output image 300 is presented to the user 20. According to this embodiment, a part of the output image 300 is concealed. Therefore, even if the output image 300 displayed in the display area 142 is viewed by a person other than the user 20 who is in the vicinity of the display area 142, information leakage can be suppressed. Examples of the program that generates the original image 200 include a BIOS, an operating system (sometimes referred to as an OS), and various application programs.

[0022] Examples of the user terminal 120 include a personal computer and a mobile terminal. Examples of the mobile terminal include a mobile phone, a smartphone, a PDA (registered trademark), a tablet, a notebook computer or laptop computer, and a wearable computer.

[0023] In this embodiment, the housing 130 supports the display device 140, the environmental sensor 150, the input device 160, and the control device 170. Each of the display device 140, the environmental sensor 150, the input device 160, and the control device 170 may be at least partially housed inside the housing 130, or may be attached to the outside of the housing 130.

[0024] In this embodiment, the display device 140 outputs various images. The display device 140 may output an image or stop outputting an image according to an instruction from the control device 170. The display device 140 may adjust the brightness or contrast of an image according to an instruction from the control device 170. The display device 140 can output an image, stop outputting an image, or adjust the brightness or contrast of an image according to an instruction from the control device 170, for example, by controlling the operation of pixels based on data output by the control device 170. Examples of the display device 140 include a display and a projector.

[0025] In this embodiment, an image output by the display device 140 (sometimes referred to as an output image) is displayed in the display area 142. In this embodiment, the display area 142 is disposed in the housing 130. In this embodiment, the display area 142 is disposed on approximately the same surface as at least a portion of the environmental sensor 150.

[0026] In this embodiment, the environmental sensor 150 acquires various information related to the surrounding environment of the display area 142. The environmental sensor 150 may include one or more sensors for acquiring the above-mentioned various information. Examples of the above-mentioned sensors include a sensor for acquiring various images, a sensor for acquiring sound, a sensor for detecting brightness, a sensor for detecting temperature, a sensor for detecting distance or depth, a receiver for receiving wireless signals, and the like. The information acquired or generated by the one or more sensors included in the environmental sensor 150 is output to the control device 170, for example.

[0027] In this embodiment, the distance sensor 152 measures the distance between the distance sensor 152 and each of one or more objects arranged around the display area 142. Examples of the periphery of the display area 142 include positions within the range of the viewing angle of the display area 142, and positions whose distance from the display area 142 is within a predetermined numerical range. The distance sensor 152 may acquire point cloud data of each of the above objects. The point cloud data may be three-dimensional point cloud data.

[0028] In addition, the relative positional relationship between the display area 142 and the distance sensor 152 is known. Therefore, the distance sensor 152 may derive the distance between the display area 142 and each of the above-mentioned objects based on the distance between the distance sensor 152 and each of the above-mentioned objects and the relative positional relationship between the display area 142 and the distance sensor 152.

[0029] The distance sensor 152 may detect the approach of the user 20 of the user terminal 120 to the display device 140 or the display area 142. For example, the distance sensor 152 measures the distance between the user 20 of the user terminal 120 and the display device 140 or the display area 142. When the distance between the user 20 of the user terminal 120 and the display device 140 or the display area 142 becomes smaller than a predetermined value, the distance sensor 152 may detect the approach of the user 20 to the image display device.

[0030] The principle and / or details of the procedure by which the distance sensor 152 measures the above-mentioned distance are not particularly limited. Examples of the distance sensor 152 include (i) a sensor that captures an image, and (ii) a sensor that measures the distance to an object by using sound waves, ultrasonic waves, electromagnetic waves, lasers, etc. Examples of the distance sensor 152 include a stereo camera, a monocular camera depth distance measurement device (sometimes referred to as a monocular depth estimation device), LiDAR, a non-contact proximity sensor, etc. An optical member such as a fisheye lens or an ultra-wide-angle lens may be attached to the above-mentioned camera.

[0031] In this embodiment, the camera 154 captures an image of the periphery of the display area 142. In this way, an image 10 of the periphery of the display area 142 is acquired. The image 10 may be a still image or a moving image.

[0032] Details of the camera 154 are not particularly limited. The camera 154 may be an image sensor for capturing a two-dimensional image, or an infrared camera for capturing an infrared image (sometimes referred to as a thermal image). The camera 154 may be a monocular camera or a stereo camera. The camera 154 may be a 180-degree camera or a 360-degree camera. In addition, when the distance sensor 152 includes an imaging element, the user terminal 120 does not need to include the camera 154.

[0033] In this embodiment, the radio wave detector 156 detects radio waves of a wireless signal. The radio wave detector 156 may measure the strength of the wireless signal. Examples of the wireless signal include a GPS signal, a WiFi (registered trademark) signal, and a Bluetooth (registered trademark) signal. As a result, for example, the control device 170 can analyze the detected radio waves to estimate the installation position of the display area 142.

[0034] In this embodiment, the input device 160 accepts input from the user 20 of the user terminal 120. Examples of the above input include input related to the operation and / or settings of the user terminal 120, input related to the operation and / or settings of various programs running on the user terminal 120, and input related to the content of the work executed by the programs. The input device 160 outputs information input by the user 20 to the input device 160 to the control device 170. Examples of the input device 160 include a keyboard, a pointing device, a touch panel, a camera, and a microphone.

[0035] In this embodiment, the control device 170 controls the user terminal 120. The control device 170 executes various types of information processing in the user terminal 120. Details of the control device 170 will be described later.

[0036] The information processing method may be a method for generating a display image to be displayed on an image display device by concealing a part of a generated image generated by an image generating device. Each step of the information processing method may be executed by, for example, a computer.

[0037] The information processing method may be a method for generating a display image to be displayed in a display area of ​​an image display device based on a generated image generated by an image generating device. Each step of the information processing method is executed by, for example, a computer. The method includes, for example, a generated image acquisition step of acquiring a generated image. The method includes, for example, a display image generation step of generating a display image based on the generated image. The method includes, for example, a peripheral image acquisition step of acquiring a peripheral image that is an image captured by an imaging device that captures the periphery of the display area. The method includes, for example, a monitoring target detection step of analyzing the peripheral image to detect a monitoring target who is a person who is directing his / her gaze toward the display area and is a person different from the user of the image display device. The method includes, for example, a viewpoint estimation step of estimating the viewpoint of the monitoring target in the display area when the monitoring target is detected. The method includes, for example, a movement determination step of determining whether or not the aspect of the movement of the viewpoint estimated in the viewpoint tracking step matches a predetermined condition. In the method, the display image generation step includes, for example, a step of outputting a display image in which a part of the generated image is concealed when it is determined that the aspect of the movement of the viewpoint matches the predetermined condition.

[0038] (Outline of original image 200 and output image 300) 2 and 3, the original image 200 and the output image 300 will be described in detail, taking as an example a case where a window 210 is displayed full screen when a program for concealing a part of the original image 200 (sometimes referred to as a concealment program) is executed. Therefore, according to this embodiment, a specific position or range in the original image 200 can be an example of a specific position or range in the display area 142. Similarly, a specific position or range in the output image 300 can be an example of a specific position or range in the display area 142.

[0039] As described above, in this embodiment, the information leakage prevention system 100 (i) first generates an original image 200 that has not been subjected to an anonymization process, and (ii) then anonymizes at least a portion of the original image 200 to generate an output image 300 to be presented to the user 20. The output image 300 includes an area (sometimes referred to as an anonymized area) in which at least a portion of information (sometimes referred to as presented information) to be presented to the user 20 has been anonymized.

[0040] (Overview of original image 200) 2, in this embodiment, an original image 200 has an image of a full-screen displayed window 210. In this embodiment, the window 210 includes a control area 212 and an image output area 214.

[0041] As described above, the window 210 corresponds to the anonymization program. The original image 200 shows an example of a screen displayed in the display area 142 after the anonymization program has been started up and in a state in which the anonymization program is not executing an anonymization process.

[0042] In this embodiment, one or more objects for displaying various information related to control of the window 210 and / or control of the concealment program are arranged in the control area 212. For example, the control area 212 has an object 222 for presenting to the user 20 operations (sometimes referred to as menus) available to the user 20, and an object 224 for presenting to the user 20 ON / OFF of the concealment process.

[0043] The control area 212 may have an object (not shown) for presenting the name of the concealed program (sometimes called the title) to the user 20, an object (not shown) for allowing the user 20 to change the size of the window 210, and an object (not shown) for allowing the user 20 to select whether to display or hide the window 210.

[0044] In this embodiment, the image output area 214 displays, for example, a background image (not shown) generated by the OS, presentation information 216 which is information presented to the user 20 by the OS, and images of a window 230 and a window 240 corresponding to a program or file other than the OS and the concealment program. In this embodiment, the window 230 includes a control area 232, an image output area 234, and presentation information 236. In this embodiment, the window 240 includes a control area 242, an image output area 244, and presentation information 246.

[0045] In this embodiment, the control area 232 has one or more objects arranged therein for displaying various information relating to the control of the window 230 and / or the control of a program or file (sometimes referred to as a document file) corresponding to the window 230. The image output area 234 displays presentation information 236, which is information presented to the user 20 by the program or file corresponding to the window 230.

[0046] In this embodiment, the control area 242 has one or more objects arranged therein for displaying various information relating to the control of the window 240 and / or the control of the program or file corresponding to the window 240. The image output area 244 displays presentation information 246, which is information presented to the user 20 by the program or file (sometimes referred to as a document file) corresponding to the window 240.

[0047] (Summary of document file) As described above, the image output area 234 of the window 230 displays presented information 236, which is information presented to the user 20 by the document file corresponding to the window 230. Similarly, the image output area 244 of the window 240 displays presented information 246, which is information presented to the user 20 by the document file corresponding to the window 240.

[0048] The document file is not particularly limited in detail as long as it is an electronic file used by the control device 170 in processing to generate an image of the window 230. The document file may be a text file or various types of binary files. Examples of binary files include image files, audio files, compressed files, and files for various application programs. The image file may be a still image file or a moving image file. The format of the document file is not particularly limited, and any known format may be adopted.

[0049] A document file includes data indicating the contents of a document. A document file may include (i) data indicating the contents of a document (sometimes referred to as document data), and (ii) data indicating the layout of the document (sometimes referred to as layout data), and / or data for identifying data that may be subject to anonymization (sometimes referred to as candidate data) from among the data indicating the contents of the document (sometimes referred to as associated data).

[0050] In this embodiment, the content of a document is not limited to a character string, and the document may be composed of, for example, a character string, a picture, a diagram, a table, a photograph, a video, or a combination of these.

[0051] A single document file may include a single piece of candidate data or may include a plurality of pieces of candidate data, each of which is used by the control device 170 to generate a different portion of the original image 200, for example. The content of information selected as the candidate data is not particularly limited.

[0052] Of the information contained in the document, for example, (i) information that is highly valuable for preventing leakage to the administrator or user 20 of the user terminal 120, or the creator or administrator of the document (sometimes referred to as user 20, etc.), and (ii) information that is highly valuable for the user 20, etc. to restrict viewing are handled as candidate data. Information that is highly valuable for the user 20, etc. to restrict viewing is exemplified by information that is subject to viewing restrictions for some reason. Examples of reasons for restricting viewing are the age of the viewer, the likelihood that the viewer will feel uncomfortable, etc.

[0053] The associated data may store, for example, the position of each of one or more pieces of candidate data in a document. The associated data may store, for each of one or more pieces of candidate data, the position of each piece of candidate data in a document in association with at least one of the contents of each piece of candidate data and the value of each piece of candidate data.

[0054] The associated data may include information indicating the start position of each of the one or more candidate data. This allows the control device 170 to specify the representative point of the start area of ​​the candidate area corresponding to each candidate data by, for example, reading the document data and the associated data.

[0055] The associated data may include information indicating the end position of each of the one or more candidate data. This allows the control device 170 to specify the representative point of the end area of ​​the candidate area corresponding to each candidate data by, for example, reading the document data and the associated data.

[0056] The associated data of a specific document file is created in advance, for example, before the concealment process is performed on the specific document file. The associated data of a specific document file is created, for example, based on the contents of the specific document file. The associated data of a specific document file is created, for example, based on the eye movement or input operation history of the creator or updater of the specific document file during the period in which the specific document file is created or updated. The associated data of a specific document file is created, for example, based on the eye movement or input operation history of the user 20 viewing or updating the specific document file.

[0057] In one embodiment, the associated data of a specific document file is created before the specific document file is executed by the control device 170. In this case, the candidate data included in the specific document file can be masked immediately after the specific document file is executed.

[0058] In another embodiment, the associated data of a specific document file is generated by the control device 170 after the control device 170 executes the specific document file and during the period when the specific document file is being executed. In this case, the candidate data included in the specific document file may be anonymized after the specific document file is executed and the associated data is generated.

[0059] 3, the document data of the document file corresponding to the window 230 includes a plurality of candidate data 30. Each of the plurality of candidate data 30 is displayed in an area (sometimes referred to as a candidate area) in the original image 200 that corresponds to the respective one of the plurality of candidate data 30.

[0060] Using Figure 3, the display mode of candidate data in original image 200 will be described in detail using an example in which a document file corresponding to window 230 contains the content "abcdef," and the content "bcde" of "abcdef" is information that is of high value to user 20 or the like in terms of preventing leakage.

[0061] According to this embodiment, the candidate data "bcde" is displayed in a candidate area 260 of the original image 200. The candidate area 260 is arranged in a part of the original image 200. The candidate area 260 includes, for example, a start area 262 arranged at the start position of the candidate area 260, and an end area 264 arranged at the end position of the candidate area 260. In this embodiment, the range of the start area 262 in the display area 142 or the original image 200 and the range of the end area 264 in the display area 142 or the original image 200 are different from each other.

[0062] According to this embodiment, in the original image 200, the candidate data "bcde" is displayed along a vector 40 that starts at the representative point 272 of the start region 262 and ends at the representative point 274 of the end region 264. Note that the vector 40 is a virtual vector and is not displayed in the original image 200. The vector 40 may be referred to as a vector corresponding to the candidate region 260.

[0063] Depending on the size of the display area 142 of the window 230, a line break may be executed in the middle of the candidate data "bcde." In this case, the direction of the vector 40 indicated by the representative point 272 and the representative point 274 does not match the display direction of the candidate data.

[0064] Therefore, according to one embodiment, when a line break is performed in the middle of the candidate data "bcde", if the document displayed in the window 230 is written horizontally, the direction of the vector 40 is considered to be parallel to the row direction. On the other hand, when the document displayed in the window 230 is written vertically, the direction of the vector 40 is considered to be parallel to the column direction. According to another embodiment, when a line break is performed in the middle of the candidate data "bcde", the direction of the vector 40 may be indicated by a combination of the row direction and the column direction.

[0065] (An example of another embodiment) In this embodiment, the details of the procedure for determining the display direction of the candidate data in the original image 200 (specifically, the direction of the vector 40) have been described using an example in which the data of a document file is displayed in the window 230 and the information is of high value to the user 20 and the like in terms of preventing leakage. As described above, the display direction of the candidate data is an example of the display mode of the candidate data. However, the procedure for determining the display direction of the candidate data is not limited to this embodiment.

[0066] According to another embodiment, the data of the image file is displayed in the window 230. When the candidate data that may be the subject of concealment is a part of a moving image and a still image, it is difficult to uniquely identify the display direction of the candidate data. In such a case, the direction of the candidate data is indicated, for example, by a division regarding the direction. This allows the direction of the candidate data to be determined within a certain range. The number of divisions regarding the direction is not particularly limited, but examples include divisions into three, four, six, eight, and twelve divisions. Examples of the four divisions include up, down, left, and right. The direction of the candidate data may be indicated probabilistically. For example, the direction of the candidate data is indicated by the probability of each of the above divisions regarding the direction. When the divisions regarding the direction are up, down, left, and right, the direction of the candidate data is indicated as up=80%, right=15%, left 5%, and down=0%.

[0067] (Outline of output image 300) 3, in this embodiment, the output image 300 has an image of the window 210 displayed full screen. In this embodiment, the window 210 includes a control area 212 and an image output area 214.

[0068] As described above, the window 210 corresponds to the anonymization program. The output image 300 shows an example of a screen displayed in the display area 142 in a state in which the anonymization program is executing an anonymization process after the anonymization program has been started up.

[0069] In this embodiment, one or more objects for displaying various information related to the control of the window 210 and / or the control of the concealment program are arranged in the control area 212. According to this embodiment, in addition to the object 222 and the object 224, the control area 212 is arranged with an object 326 for presenting the operation mode of the concealment process to the user 20 and an object 328 for presenting the aspect and / or strength of the concealment process to the user 20.

[0070] In this embodiment, one or more objects for concealing at least a part of one or more candidate regions included in the original image 200 are arranged in the image output area 214 at or near the position and / or range (sometimes referred to as the viewpoint) of the line of sight of the person 22 in the display area 142. According to the embodiment shown in Fig. 3, an object 330 for concealing the candidate region 260 corresponding to the candidate data "bcde" included in the document file corresponding to the window 230 is arranged in the image output area 214.

[0071] The image of the object 330 is not particularly limited as long as it can conceal the image of the original image 200. The degree of concealment (sometimes referred to as strength) may be determined by initial settings, or may be determined by the administrator or user 20 described above. The image of the object 330 will be described in detail later.

[0072] In this embodiment, the display or non-display of the object 330 is determined based on the manner of movement of the viewpoint 50 of the person 22 (sometimes referred to as the movement manner). For example, when the degree of agreement between a part of the trajectory of the viewpoint 50 of the person 22 and the vector 40 corresponding to the candidate area 260 (sometimes referred to as the degree of agreement) exceeds a predetermined degree, the object 330 is displayed and the candidate area 260 is concealed. On the other hand, when the degree of agreement does not meet the predetermined degree, or when the degree of agreement no longer meets the predetermined degree, the object 330 is not displayed. In this case, the output image 300 in which the candidate data "bcde" corresponding to the candidate area 260 is displayed is displayed in the display area 142 of the display device 140. The procedure for determining whether to display or not to display the object 330 will be described in detail later.

[0073] The information leakage prevention system 100 may be an example of an information processing device. The user terminal 120 may be an example of an image generating device, an image display device, an imaging device, or an information processing device. The display device 140 may be an example of an image display device. The display area 142 may be an example of a display area of ​​an image display device. The environmental sensor 150 may be an example of an imaging device. The camera 154 may be an example of an imaging device. The control device 170 may be an example of an image generating device or an information processing device.

[0074] Original image 200 may be an example of a generated image. The image of window 230 may be an example of at least a portion of a generated image. Candidate region 260 may be an example of one or more candidate regions. Start region 262 may be an example of a start region located at the start position of each candidate region. End region 264 may be an example of an end region located at the end position of each candidate region. Output image 300 may be an example of a displayed image. The region in which object 330 is located may be an example of a target region.

[0075] Image 10 may be an example of a peripheral image. User 20 may be an example of a user 20 of an image display device. Person 22 may be an example of a person to be monitored or a monitored object. Multiple candidate data 30 may be an example of one or more candidate data. A concealed area may be an example of a target area. A document file may be an example of an electronic file. Accompanying data may be an example of candidate information. Information indicating the start position of each candidate data may be an example of representative point information. Information indicating the end position of each candidate data may be an example of representative point information. A candidate area corresponding to each candidate data may be an example of one or more candidate areas.

[0076] (An example of another embodiment) In this embodiment, the information leakage prevention system 100 has been described in detail by taking as an example a case in which the user terminal 120 conceals a part of the original image 200 to generate the output image 300. However, the information leakage prevention system 100 is not limited to this embodiment. In other embodiments, the user terminal 120 may be configured to be able to communicate with another computer arranged outside the user terminal 120, and may execute the various information processes described above in cooperation with the other computer. For example, at least a part of the control device 170 may be realized by the other computer.

[0077] In the present embodiment, the details of the information leakage prevention system 100 have been described by taking as an example a case where the display area 142 and the environmental sensor 150 are arranged on approximately the same plane. However, the information leakage prevention system 100 is not limited to the present embodiment. In other embodiments, at least one of the display area 142 and the environmental sensor 150 may be arranged outside the housing 130 as long as the relative positional relationship between the display area 142 and the environmental sensor 150 can be specified.

[0078] In this embodiment, the details of the information leakage prevention system 100 have been described by taking as an example a case where the control device 170 executes a document file to generate the original image 200 and the output image 300. However, the information leakage prevention system 100 is not limited to this embodiment. In other embodiments, the control device 170 may execute any type of electronic file to generate the original image 200 and the output image 300. For example, the control device 170 executes an image file to generate the original image 200 and the output image 300. The image file may be a still image file or a moving image file.

[0079] In the present embodiment, the details of the information leakage prevention system 100 have been described by taking as an example a case in which the information leakage prevention system 100 prevents leakage of confidential information via a person 22 present behind a user 20 of a user terminal 120. However, the information leakage prevention system 100 is not limited to the present embodiment.

[0080] In another embodiment, the information leakage prevention system 100 may prevent the imaging of confidential information by an imaging device (sometimes referred to as a monitored device) located behind the user 20 of the user terminal 120. The confidential information may be an example of information that is valuable to the user 20 and the like and that should be restricted from viewing. A person skilled in the art who has read this specification can understand that in various embodiments described in this specification, the viewpoint 50 of the person 22 can be interpreted as the imaging position of the monitored device in the display area 142, thereby preventing the leakage of confidential information using the monitored device. Specifically, the display of concealed confidential information in the display area 142 can prevent the leakage of confidential information.

[0081] In the present embodiment, the details of the information leakage prevention system 100 have been described by taking as an example a case in which the display or non-display of the object 330 is determined based on the manner of movement of the viewpoint 50 of the person 22. However, the information leakage prevention system 100 is not limited to the present embodiment.

[0082] In another embodiment, the information leakage prevention system 100 may control whether to display or hide the object 330 based on (i) a relative positional relationship between the person 22 directing his / her gaze toward the display area 142 and the display area 142, and (ii) a direction of gaze of the person 22. The information leakage prevention system 100 may control whether to display or hide the object 330 based on (i) a relative positional relationship between the user 20, the person 22 directing his / her gaze toward the display area 142, and the display area 142, and (ii) a direction of gaze of the person 22.

[0083] For example, the control device 170 specifies the range of an area (sometimes called a silhouette) in the display area 142 where the line of sight of the person 22 is blocked by the body of the user 20, based on the relative positional relationship between the user 20, the person 22 who is directing his / her gaze toward the display area 142, and the display area 142. Examples of the range of the silhouette include the position, shape, and size of the silhouette. When at least a part of the object 330 is placed inside the silhouette of the user 20, the control device 170 may control the display or non-display of the object 330 so that the part of the object 330 that overlaps with the silhouette is not displayed. As a result, data placed inside the silhouette is displayed in the display area 142 without being concealed by the object 330. The object 330 may be generated so as to cover the entire screen except for the silhouette of the user 20.

[0084] The silhouette is generated, for example, according to the following procedure. First, the camera 154 photographs the user 20, and the appearance of the user 20 is generated on a two-dimensional plane with the direction of the camera 154 as the normal direction. The silhouette described above is generated by projecting a mask with the appearance of the user 20 as a boundary onto the display area 142, taking into consideration the line of sight from the person 22 who is directing his / her gaze toward the display area 142. The mask may be an image for covering the image of the area where the mask is placed. The line of sight from the person 22 may be determined (i) based on the direction of the face of the person 22, (ii) based on the state of one of the person 22's eyes, or (iii) based on the result of ANDing the line of sight of the right eye and the line of sight of the left eye by separately calculating the lines of sight of both eyes of the person 22.

[0085] Another example of a procedure for generating a silhouette may involve using camera 154 to photograph user 20, generating or estimating a three-dimensional model, generating an appearance on a two-dimensional plane from the line of sight of person 22 directing his / her gaze toward display area 142, and projecting the appearance onto display area 142 to generate a silhouette. The three-dimensional model is generated or estimated, for example, using an xNN-based technology that estimates a 3D silhouette from a monocular planar image.

[0086] In still another embodiment, the information leakage prevention system 100 may control whether to display or hide the object 330 based on (i) the relative positional relationship between the imaging device (which may be referred to as a monitored device) facing the imaging direction toward the display area 142 and the display area 142, and (ii) the imaging direction of the monitored device. The information leakage prevention system 100 may control whether to display or hide the object 330 based on (i) the relative positional relationship between the user 20, the imaging device facing the imaging direction toward the display area 142, and the display area 142, and (ii) the imaging direction of the monitored device.

[0087] For example, the control device 170 specifies the range of an area (sometimes called a silhouette) in the display area 142 where the body of the user 20 prevents the imaging device from capturing an image based on the relative positional relationship between the user 20, the imaging device facing the display area 142, and the display area 142. Examples of the range of the silhouette include the position, shape, and size of the silhouette. When at least a part of the object 330 is placed inside the silhouette of the user 20, the control device 170 may control the display or non-display of the object 330 so that the part of the object 330 that overlaps with the silhouette is not displayed. As a result, data placed inside the silhouette is displayed in the display area 142 without being concealed by the object 330. The object 330 may be generated so as to cover the entire screen except for the silhouette of the user 20.

[0088] (Overview of the masking process) Fig. 4 shows an example of information processing in the control device 170. Specifically, an example of the above-mentioned anonymization processing will be described with reference to Fig. 4. According to this embodiment, first, in step 422 (step may be abbreviated as S), the control device 170 starts the above-mentioned anonymization program.

[0089] As a result, the window 210 corresponding to the masking program is displayed in full screen in the foreground. The image displayed in the window 210 immediately after the masking program is started is generated based on, for example, settings related to the masking program.

[0090] (a) When the setting related to the anonymization program indicates that a predetermined image or an image generated or selected by a predetermined algorithm is to be displayed immediately after the start, the control device 170 displays the image in the display area 142. (b) When the setting related to the anonymization program indicates that anonymization processing is not required immediately after the start, the control device 170, for example, determines to display the original image 200 in the display area 142. (c) When the setting related to the anonymization program indicates that anonymization processing is required immediately after the start, the control device 170, for example, determines to anonymize at least a part of the original image 200 according to the setting related to the anonymization program. The control device 170, for example, determines to display the output image 300 in the display area 142. In this case, the details of the anonymization processing (for example, the target and manner of anonymization) may be determined based on the above-mentioned setting.

[0091] In one embodiment, the start-up process of the anonymization program is automatically executed upon start-up of the user terminal 120. In another embodiment, the start-up process of the anonymization program is executed in response to an instruction from the user 20.

[0092] In still another embodiment, when it is not confirmed that the user terminal 120 is located within a predetermined area, the activation process of the concealment program may be automatically executed. Examples of the predetermined area include an area where the administrator of the user terminal 120 permits the use of the user terminal 120, a room used by an organization to which the user 20 belongs, the office of the user 20, and the home of the user 20. Whether or not the user terminal 120 is located within the predetermined area may be determined based on an image of the periphery of the display area 142 captured by the camera 154, or may be determined based on information included in a wireless signal detected by the radio wave detector 156.

[0093] Next, in S424, the control device 170 determines the target of anonymization. For example, the control device 170 determines an operation mode related to the target of anonymization. Examples of the operation mode related to the target of anonymization include (a) a mode in which anonymization is performed in principle, and anonymization is not performed exceptionally in an area estimated to be necessary for the work of the user 20 (sometimes referred to as a first mode), (b) a mode in which anonymization is not performed in principle, and anonymization is performed exceptionally in an area estimated to be viewed by a person other than the user 20 who exists near the display area 142 (sometimes referred to as a second mode), and (c) a mode in which anonymization is performed in principle, and anonymization is not performed exceptionally in an area estimated to be difficult to view by a person other than the user 20 (sometimes referred to as a third mode).

[0094] In this embodiment, in order to facilitate understanding of the information leakage prevention system 100, the details of the information leakage prevention system 100 will be described by taking as an example a case where the control device 170 operates in the second mode. The control device 170 can be configured to be able to change the operation mode related to the object of anonymization in response to an instruction from, for example, an administrator or a user 20.

[0095] Thereafter, the control device 170 executes a process for generating an image for updating the display screen of the display area 142. For example, first, in S432, the control device 170 acquires a document file for generating at least a part of the original image 200. The control device 170 acquires, for example, a document file corresponding to the window 230. As described above, in this embodiment, the document file includes document data indicating the contents of the document and associated data indicating each of one or more candidate data included in the document.

[0096] Next, in S434, the control device 170 executes the document file. The control device 170 generates an original image 200 including an image showing the contents of the document based on, for example, document data included in the document file. The control device 170 generates the original image 200 based on, for example, an output from an OS and / or an application program capable of executing the document file. The application program capable of executing the document file may be a program different from the concealment program.

[0097] According to this embodiment, the control device 170 reads the associated data and identifies one or more candidate data (sometimes referred to as concealment candidates). The control device 170 also determines the range of each of the one or more candidate regions in the original image 200. The control device 170 may determine the start point of a vector corresponding to each candidate region, and the direction and length of each vector. The direction and length of each vector are determined, for example, based on information regarding the start point and end point of each vector.

[0098] Next, in S440, the control device 170 judges whether or not the movement manner of the viewpoint 50 of the person 22 matches a predetermined condition (sometimes referred to as a condition related to the movement of the viewpoint). This suppresses unnecessary concealment processing, improving the usage experience of the user 20. Note that the movement manner of the viewpoint 50 of the person 22 matching the condition related to the movement of the viewpoint includes not only a case where the two match completely, but also a case where the two match approximately.

[0099] The condition regarding the movement of the viewpoint may be a condition that a part of the trajectory of the viewpoint 50 of the person 22 matches a vector corresponding to any one of the one or more candidate areas. In this embodiment, for the purpose of making it easier to understand the information processing in the information leakage prevention system 100, an example of the above-mentioned concealment processing will be described by taking as an example a case in which the control device 170 determines whether or not a part of the trajectory of the viewpoint 50 of the person 22 matches a vector 40 corresponding to the above-mentioned candidate area 260.

[0100] In one embodiment, if the viewpoint 50 of the person 22 is estimated to be located in the start region 262 of the candidate region 260 and then is estimated to be located in the end region 264 of the candidate region 260 before a period of time having a predetermined length has elapsed, it is determined that a portion of the trajectory of the viewpoint 50 of the person 22 matches the vector 40 corresponding to the candidate region 260. In another embodiment, if the viewpoint 50 of the person 22 is estimated to have moved along the vector 40, it is determined that a portion of the trajectory of the viewpoint 50 of the person 22 matches the vector 40 corresponding to the candidate region 260.

[0101] In S440, if the movement of the viewpoint 50 of the person 22 does not match the condition regarding the movement of the viewpoint (No in S440), in S442, the control device 170 generates an output image 300 in which the candidate area 260 of the original image 200 has not been subjected to the concealment process. The control device 170 controls the display device 140 to display the generated output image 300 in the display area 142. As a result, the candidate area 260 of the output image 300 displayed in the display area 142 of the display device 140 displays "bcde", which is the candidate data corresponding to the area. Note that even in this case, as a result of determining that the movement of the viewpoint 50 of the person 22 matches the vector corresponding to another candidate area, the concealment process may be performed on the candidate area 260 together with the other candidate area.

[0102] On the other hand, in S440, if the movement mode of the viewpoint 50 of the person 22 matches the condition related to the movement of the viewpoint (Yes in S440), the control device 170 determines in S450 whether the behavior pattern of the person 22 matches a predetermined pattern (sometimes referred to as a reference pattern). This further suppresses unnecessary concealment processing, and further improves the usage experience of the user 20. Note that matching the behavior pattern of the person 22 with the reference pattern includes not only a case where the two match completely, but also a case where the two match approximately.

[0103] In one embodiment, if the behavior pattern of the person 22 does not match a pattern characteristic of a person who does not intend to peek at information, or a pattern characteristic of a person who is performing an action that makes it physically difficult to peek at information compared to when the person is stationary (which may be referred to as a first behavior pattern), the control device 170 may determine that the behavior pattern of the person 22 matches the reference pattern. In another embodiment, if the behavior pattern of the person 22 matches a pattern characteristic of a person who intends to peek at information (which may be referred to as a second behavior pattern), the control device 170 may determine that the behavior pattern of the person 22 matches the reference pattern. The first behavior pattern and the second behavior pattern will be described in detail later.

[0104] In S450, if the behavior pattern of the person 22 does not match the reference pattern (No in S450), in S442, the control device 170 generates an output image 300 in which no concealment processing has been applied to the candidate region 260 of the original image 200. Details of the information processing in S442 may be as described above.

[0105] On the other hand, if the behavior pattern of the person 22 matches the reference pattern in S450 (Yes in S450), the control device 170 generates an output image 300 in which a concealment process has been applied to the candidate area 260 of the original image 200 in S452. The control device 170 controls the display device 140 to display the generated output image 300 in the display area 142. This minimizes leakage of the candidate data "bcde" corresponding to the candidate area 260 of the original image 200 even if the person 22 peers at the display area 142 of the display device 140.

[0106] The control device 170 repeats, for example, the processes of S440 to S452. The control device 170 may repeat the processes of S434 to S452, or may repeat the processes of S432 to S452. In this way, the concealment process is executed.

[0107] The condition regarding the movement of the viewpoint may be an example of the first condition or the second condition. Candidate area 260 may be an example of a first candidate area. Vector 40 may be an example of a vector extending from a representative point of the start area of ​​the first candidate area to a representative point of the end area of ​​the first candidate area.

[0108] (An example of another embodiment) In this embodiment, the details of the anonymization process are described by taking as an example a case where it is determined whether or not the behavior pattern of the person 22 matches the reference pattern in S450. However, the anonymization process is not limited to the above embodiment. In other embodiments, the process of S450 may be omitted.

[0109] In the present embodiment, the details of the information leakage prevention system 100 have been described by taking as an example a case where the control device 170 operates in the second mode, in order to facilitate understanding of the information leakage prevention system 100. However, the information leakage prevention system 100 is not limited to the present embodiment.

[0110] In another embodiment, the control device 170 may operate in the first mode or the third mode. When the control device 170 operates in the third mode, the control device 170 determines whether or not to output the output image 300 in which a part of the original image 200 is concealed, based on, for example, the viewpoint of the person 22 and the silhouette of the user 20. The control device 170 determines whether or not to output the output image 300 in which a part of the original image 200 is concealed, based on, for example, the position of the line of sight (i.e., the viewpoint) of the person 22 in the display area 142 and the silhouette of the user 20 in the display area 142. The control device 170 determines whether or not to output the output image 300 in which a part of the original image 200 is concealed, based on the shooting position of the monitored device in the display area 142 and the silhouette of the user 20 in the display area 142. Specifically, when at least a part of the object 330 is disposed inside the silhouette of the user 20, the control device 170 controls the display or non-display of the object 330 so that the part of the object 330 that overlaps with the silhouette is not displayed.

[0111] 5 is a schematic diagram showing an example of the internal configuration of the control device 170. In this embodiment, the control device 170 includes, for example, a setting information storage unit 520, a program control unit 532, a window control unit 534, an input control unit 540, and an output control unit 550. In this embodiment, the output control unit 550 includes a surrounding monitoring unit 552, an original image generating unit 560, a line-of-sight information acquiring unit 574, an operation information acquiring unit 576, a value map generating unit 578, and an image processing unit 580. In this embodiment, the image processing unit 580 includes a concealment processing unit 584, a leakage prevention support unit 586, and an output image generating unit 588.

[0112] In this embodiment, the setting information storage unit 520 stores information (sometimes referred to as setting information) on various settings of the information leakage prevention system 100 and / or the user terminal 120. The setting information includes, for example, information on the settings of the anonymization program. Examples of the settings of the anonymization program include (i) settings on the necessity or target of anonymization by the anonymization program and / or the mode of the anonymization, and (ii) various settings on the processing immediately after the anonymization program is started.

[0113] In this embodiment, the program control unit 532 controls various programs that operate on the user terminal 120. The program control unit 532 may control the start-up process of the above programs, may control the operation of the above programs, or may control the termination process of the above programs.

[0114] In this embodiment, the window control unit 534 executes various controls related to windows corresponding to various programs operating on the user terminal 120. The window control unit 534 outputs, for example, various information related to the above-mentioned windows (sometimes referred to as window information) to the output control unit 550. Examples of the window information include information indicating the range of the window on the screen (e.g., position, shape, size, etc.), information regarding various objects arranged in the window, and information indicating the type of flag assigned to the window. Examples of the above-mentioned flag include no copying, no screenshot, etc.

[0115] In this embodiment, the input control unit 540 controls various types of information input to the user terminal 120. In one embodiment, the input control unit 540 acquires data output by the environment sensor 150. The input control unit 540 acquires, for example, image data output by the camera 154, and outputs the image data to the line of sight information acquisition unit 574.

[0116] In another embodiment, the input control unit 540 controls the operation of the input device 160. The input control unit 540 receives, for example, various instructions input by the user 20 to the input device 160. Examples of the instructions include instructions regarding the operation of the user terminal 120 and instructions for various programs running on the user terminal 120. This allows the input control unit 540 to specify, for example, the position or range of the input by the user 20.

[0117] The input control unit 540 outputs, for example, the above instruction to the operation information acquisition unit 576. The input control unit 540 may output information indicating the position or range of the input by the user 20 to the operation information acquisition unit 576. The input control unit 540 may output information indicating the content input by the user 20 to the input device 160 to the operation information acquisition unit 576.

[0118] In this embodiment, the output control unit 550 controls the output of information presented to the user 20. In one embodiment, the output control unit 550 controls the output of image information. In another embodiment, the output control unit 550 controls the output of audio information.

[0119] In this embodiment, the periphery monitoring unit 552 monitors the environment surrounding the display area 142. For example, the periphery monitoring unit 552 monitors a person present in the periphery of the display area 142. The periphery monitoring unit 552 may monitor the presence or absence of a person present in the periphery of the display area 142, may monitor the line of sight of the person, or may monitor the behavior of the person. The periphery monitoring unit 552 may store a history of the monitoring results in an arbitrary storage device. The periphery monitoring unit 552 may store data acquired for monitoring in an arbitrary storage device.

[0120] For example, the periphery monitoring unit 552 monitors the environment surrounding the display area 142 by analyzing the image 10 captured by the camera 154. As described above, if a person is present in the periphery of the display area 142, the person is captured in the image 10. The periphery monitoring unit 552 may store data of the image 10 in an arbitrary storage device. The periphery monitoring unit 552 may store data of the image 10 captured during an arbitrary period in the storage device.

[0121] In one embodiment, the periphery monitoring unit 552 may store the data of the image 10 in an arbitrary storage device while deleting old data so that the latest data is always stored. The data stored in the storage device may be data for the most recent tens of seconds, data for the most recent hours, or data for the most recent days.

[0122] In another embodiment, the periphery monitoring unit 552 may store the data of the image 10 in the first storage device while deleting old data so that the latest data is always stored. The data stored in the first storage device may be data for the most recent tens of seconds, data for the most recent hours, or data for the most recent days.

[0123] For example, the periphery monitoring unit 552 monitors the occurrence of a specific event. When the occurrence of a specific event is detected, the periphery monitoring unit 552 may store data of the image 10 during a period from a first time point at which the occurrence of the event is detected to a second time point that is a time point earlier than the time point at which the occurrence of the event is detected, in a second storage device different from the first storage device. When the occurrence of a specific event is detected, the periphery monitoring unit 552 may store data of the image 10 in the first storage device while deleting old data so that data of the image 10 during the period from the first time point to the second time point is not deleted from the first storage device. The length of the period between the first time point and the second time point may be predetermined by initial setting or designation by the user 20 or the like.

[0124] According to these embodiments, when there is a possibility that data has been peeked, evidence video for verifying whether or not the data has been peeked can be recorded. Details of the perimeter monitoring unit 552 will be described later.

[0125] In this embodiment, the original image generating unit 560 generates the original image 200. In this embodiment, the output control unit 550 obtains window information related to each of the one or more windows included in the original image 200 from the window control unit 534.

[0126] In this embodiment, the gaze information acquisition unit 574 acquires an image of the user 20 captured by the camera 154 from the input control unit 540. The gaze information acquisition unit 574 analyzes the image of the user 20 to estimate the position and / or range of the gaze of the user 20 in the original image 200 (as described above, this may be referred to as a viewpoint). Any known method may be adopted as a method for estimating the viewpoint of the user 20 in the original image 200. As described above, the viewpoint of the user 20 in the original image 200 may be an example of the viewpoint of the user 20 in the display area 142.

[0127] In one embodiment, the gaze information acquisition unit 574 analyzes the image to identify a normal vector of the face of the user 20. The gaze information acquisition unit 574 estimates the viewpoint of the user 20 in the original image 200 using the normal vector of the face of the user 20. In another embodiment, the gaze information acquisition unit 574 analyzes the image to identify a normal vector of the iris of the user 20. The gaze information acquisition unit 574 estimates the viewpoint of the user 20 in the original image 200 using the normal vector of the iris of the user 20.

[0128] In this embodiment, the operation information acquisition unit 576 acquires various pieces of information (sometimes referred to as operation information) input by the user 20 to the input device 160 from the input control unit 540. As described above, examples of the operation information include (i) information indicating various instructions or contents input by the user 20 to the input device 160, and (ii) information indicating the position or range of the input by the user 20.

[0129] In this embodiment, the value map generating unit 578 generates a database (sometimes called a value map) that stores identification information (sometimes called a candidate data ID) of one or more candidate data included in a document file and information indicating the range (e.g., position, shape, size, etc.) occupied by the candidate area corresponding to each candidate data in the original image 200 of the document file in association with each other. The value map generating unit 578 outputs the value map of the document file to, for example, the image processing unit 580.

[0130] Thereby, the image processing unit 580 can refer to the value map to determine where information of high value to the user 20, etc. is located in the display area 142 or the original image 200. As described above, examples of information of high value to the user 20, etc. include information of high value to prevent leakage, information of high value to restrict viewing, etc. Details of the value map generating unit 578 will be described later.

[0131] In this embodiment, the image processing unit 580 conceals a part of the original image 200 generated by the original image generation unit 560, and generates an output image 300 to be displayed in the display area 142 of the display device 140. The image processing unit 580 may control the operation of the display device 140 to cause the output image 300 to be displayed in the display area 142.

[0132] In this embodiment, the anonymization processing unit 584 executes the anonymization processing described above. The anonymization processing unit 584 determines whether or not to anonymize a part of the original image 200 output by the original image generating unit 560 based on, for example, the monitoring result of the line of sight and / or the behavior of the person 22 output by the periphery monitoring unit 552. When it is determined to anonymize a part of the original image 200, the anonymization processing unit 584 determines the target and / or manner of anonymization.

[0133] In one embodiment, the anonymization processing unit 584 outputs information indicating the target and / or manner of anonymization to the output image generation unit 588. In another embodiment, the anonymization processing unit 584 acquires the original image 200 generated by the original image generation unit 560. The anonymization processing unit 584 anonymizes a part of the original image 200 based on the above determination. The anonymization processing unit 584 outputs the image in which the part of the original image 200 has been anonymized to the output image generation unit 588. Details of the anonymization processing unit 584 will be described later.

[0134] The leakage prevention support unit 586 executes various processes (sometimes referred to as support processes) for preventing information leakage. Examples of the support processes include (a) a process for encouraging the user 20 to take a specific action for preventing information leakage, (b) a process for causing the control device 170 to execute a specific information process for preventing information leakage, and (c) a process for outputting a sound indicating the status of the concealment process by the control device 170.

[0135] An example of the specific behavior is changing the physical position and / or attitude of the user terminal 120. For example, the user 20 can change the physical attitude of the user terminal 120 by rotating the user terminal 120 clockwise or counterclockwise. Examples of the specific information processing are (i) a process for changing the target and / or mode of anonymization when the length of a period during which no input from the user 20 is detected exceeds a predetermined value, and (ii) a process for changing the target and / or mode of anonymization between one display area 142 and another display area 142 when the user terminal 120 has multiple display areas 142.

[0136] In one embodiment, the leakage prevention support unit 586 analyzes the image captured by the camera 154 to determine whether or not two or more people are present around the display area 142. The leakage prevention support unit 586 may analyze the image captured by the camera 154 to determine whether or not one or more people 22 are present around the display area 142. For example, even if another person (particularly person 22) is present behind the user 20, if the user 20 rotates the user terminal 120, it may become difficult for that person to view the display area 142.

[0137] Therefore, when it is determined that two or more people exist around the display area 142, or when it is determined that one or more people 22 exist around the display area 142, the leakage prevention support unit 586 executes a process for prompting the user 20 to change the orientation of the user terminal 120 (for example, the orientation of the normal vector of the display area 142). For example, the leakage prevention support unit 586 controls the output image generation unit 588 to cause the output image generation unit 588 to generate and output an image for suggesting the rotation direction of the user terminal 120.

[0138] In another embodiment, when the length of the period during which the input control unit 540 does not detect an input from the user 20 exceeds a predetermined value, the leakage prevention support unit 586 determines to change the settings related to the shape and / or size of the area where anonymization can be performed. This changes the target of anonymization. In addition, the leakage prevention support unit 586 controls the output image generation unit 588 to update the screen displayed in the display area 142.

[0139] For example, when the above setting is changed so that the range in which concealment can be applied is increased, the number of candidate regions to be concealed when peeping by the person 22 is detected may increase compared to before the setting was changed. For example, when the above setting is changed so that the range in which concealment can be applied is the entire screen, all candidate regions arranged in the display area 142 may be automatically concealed when peeping by the person 22 is detected.

[0140] In still another embodiment, when the user terminal 120 has a plurality of display regions 142, the leakage prevention support unit 586 adjusts the settings related to the shape and / or size of the area where concealment can be applied so that the entire screen of a display or screen (which may be simply referred to as a sub-display) other than the main display or main screen becomes the area where concealment can be applied. In this way, when it is detected that the person 22 is peeking at the sub-display, all the candidate areas arranged on the sub-display can be automatically concealed.

[0141] As a result, for example, when the control device 170 outputs images of the same document file to a display built into the housing 130 of the user terminal 120 and a projector connected to the user terminal 120, the number of concealed candidate data in each image may differ when comparing an image output to the display when the person 22 is looking at the display and an image output to the screen when the person 22 is looking at an image output to the screen by the projector. The display built into the housing 130 of the user terminal 120 may be an example of a main display. A screen onto which an image is projected by a projector connected to the user terminal 120 may be an example of a sub-display.

[0142] In still another embodiment, the leakage prevention support unit 586 may transmit the contents of the anonymized candidate data by voice. The leakage prevention support unit 586 may transmit the reason why the candidate data or the candidate region is anonymized by voice. The leakage prevention support unit 586 may output the audio information via a directional speaker, earphones, or headphones arranged in the user terminal 120.

[0143] In this embodiment, the output image generation unit 588 acquires, for example, the original image 200 generated by the original image generation unit 560. The output image generation unit 588 generates the output image 300 based on the original image 200, for example.

[0144] The output image generation unit 588 may generate an image in which the concealment processing determined by the concealment processing unit 584 has been applied to at least a part of the original image 200 generated by the original image generation unit 560. The output image generation unit 588 may generate an image in which the support processing determined by the leakage prevention support unit 586 has been applied to at least a part of the original image 200 generated by the original image generation unit 560. The output image generation unit 588 may generate an image in which the concealment processing determined by the concealment processing unit 584 and the support processing determined by the leakage prevention support unit 586 have been applied to a part of the original image 200 generated by the original image generation unit 560.

[0145] As a result, the output image 300 is generated. The output image generation unit 588 controls the operation of the display device 140 to display the generated output image 300 in the display area 142. When a screen capture process is executed in response to an instruction from the user 20, the output image generation unit 588 may control the operation of the display device 140 to display the original image 200. As a result, the original image 200 that is not concealed may be captured. In this case, the output image 300 may remain displayed in the display area 142 of the display device 140.

[0146] The output control unit 550 may be an example of an image generating device or an information processing device. The original image generating unit 560 may be an example of an image generating device. The image processing unit 580 may be an example of a generated image acquiring unit or an information processing device. Each unit of the image processing unit 580 may be an example of a generated image acquiring unit.

[0147] 6 is a schematic diagram showing an example of an internal structure of the value map generating unit 578. In this embodiment, the value map generating unit 578 includes, for example, a document file acquiring unit 612, an associated data acquiring unit 614, a region data assigning unit 616, and an associated data generating unit 620. In this embodiment, the associated data generating unit 620 includes, for example, a candidate data determining unit 622 and a value determining unit 624.

[0148] The document file acquisition unit 612 acquires a document file of a document displayed in the display area 142. As described above, the document file includes, for example, document data indicating the contents of the document and associated data for identifying candidate data included in the document. In this case, the document file acquisition unit 612 outputs the associated data to, for example, the associated data acquisition unit 614. The document file acquisition unit 612 may output the document data to the area data assignment unit 616.

[0149] When the document file does not include the associated data, the document file acquisition unit 612 outputs, for example, information indicating that the document file does not include the associated data to the associated data acquisition unit 614. The document file acquisition unit 612 may output the document data to the area data attachment unit 616 and / or the associated data generation unit 620.

[0150] In this embodiment, the associated data acquisition unit 614 acquires, for example, associated data of a document displayed in the display area 142. The associated data acquisition unit 614 outputs the associated data to the area data attachment unit 616, for example.

[0151] The associated data acquisition unit 614 acquires the associated data from, for example, the document file acquisition unit 612. If the document file acquired by the document file acquisition unit 612 does not contain any associated data, the associated data acquisition unit 614 acquires, for example, information indicating that the document file does not contain any associated data. In this case, the associated data acquisition unit 614 executes a process for searching for or generating the associated data of the document file.

[0152] In one embodiment, the associated data acquisition unit 614 accesses a storage device that stores a database that stores identification information of each document file and associated data of each document file in association with each other for each of a plurality of document files, and requests extraction of the associated data associated with the identification information of the document file. This allows the associated data acquisition unit 614 to acquire the associated data of the document file.

[0153] The URI of the storage device is stored, for example, in the setting information storage unit 520. The storage device may be (i) a storage device mounted on the user terminal 120, or (ii) a storage device mounted on an information processing device arranged outside the user terminal 120 and configured to be capable of transmitting and receiving information to and from the user terminal 120 via a communication network.

[0154] In another embodiment, the associated data acquisition unit 614, for example, requests the associated data generation unit 620 to generate associated data for the above document file. This allows the associated data acquisition unit 614 to acquire the associated data generated by the associated data generation unit 620.

[0155] The area data adding unit 616 acquires the associated data of the document file from the associated data acquiring unit 614. The area data adding unit 616 determines one or more areas arranged in a part of the original image 200, in which each of the one or more candidate data specified by the associated data is displayed (as described above, these are the candidate areas corresponding to each candidate data).

[0156] Since each candidate area is a position where each candidate data is displayed in the original image 200 obtained by executing the document file, the area data assigning unit 616 can determine the candidate area corresponding to each candidate data according to any known procedure. As described above, in this embodiment, the one or more areas in which the one or more candidate data are displayed in the original image 200 may be an example of the one or more areas in which the one or more candidate data are displayed in the display area 142.

[0157] As described above, the associated data includes, for example, information indicating the start position of each of the one or more candidate data and information indicating the end position of each of the candidate data. This allows the region data attachment unit 616 to specify the start region and end region of the candidate region corresponding to each of the candidate data. The region data attachment unit 616 can specify the position of the representative point of the start region of the candidate region corresponding to each of the candidate data, for example, based on a setting regarding a procedure for determining the representative point of the start region. The region data attachment unit 616 can specify the position of the representative point of the end region of the candidate region corresponding to each of the candidate data, for example, based on a setting regarding a procedure for determining the representative point of the end region. These settings are stored, for example, in the setting information storage unit 520.

[0158] The region data assigning unit 616 stores, for example, identification information of each of one or more candidate data in association with information indicating the range occupied by the candidate region corresponding to each candidate data in the original image 200. In this way, a value map is generated.

[0159] The area data assigning unit 616 may generate a value map by assigning information indicating the range of the candidate area corresponding to each candidate data in the original image 200 to information of each candidate data included in the associated data. For example, when the associated data is a database having one or more records corresponding to each of one or more candidate data, the area data assigning unit 616 assigns information indicating the range of the candidate area corresponding to each candidate data in the original image 200 to the record of each candidate data.

[0160] In this embodiment, the associated data generating unit 620 generates, for example, the associated data described above. For example, the associated data generating unit 620 generates associated data related to a specific document file in response to a request from the associated data acquiring unit 614. The associated data generating unit 620 outputs the generated associated data to, for example, the associated data acquiring unit 614 and / or the area data adding unit 616.

[0161] In this embodiment, the candidate data determination unit 622 determines one or more candidate data included in a specific document file. For example, the candidate data determination unit 622 identifies information that is valuable to the above-mentioned user 20, etc., from among the information included in the specific document file, and determines the valuable information as candidate data. This allows the candidate data determination unit 622 to identify the start position of each candidate data. Similarly, the candidate data determination unit 622 can identify the end position of each candidate data.

[0162] In one embodiment, the candidate data determination unit 622 analyzes the contents of a specific document file or the original image 200 obtained by the original image generation unit 560 executing the specific document file, and identifies information that is highly valuable to the user 20, etc. The candidate data determination unit 622 identifies information that is highly valuable to the user 20, etc., by using, for example, any known natural language processing technology, pattern recognition technology, and / or image recognition technology.

[0163] Examples of information that is valuable to the user 20 and the like include at least one selected from the group consisting of a character string, a picture, a diagram, a table, a photograph, a video, and parts of these. Examples of parts of a character string include proper nouns, verbs, subjects of the verbs, objects of the verbs, and the like contained in the character string. Examples of parts of a picture, a photograph, or a video include a person's face.

[0164] In another embodiment, the candidate data determination unit 622 identifies information that is highly valuable to the user 20, etc., based on the operation history of a pointing device by the user 20, etc., during a period in which the user 20, etc., is viewing, creating, or updating a specific document file. For example, the candidate data determination unit 622 identifies (i) information that was displayed in an area where the number of clicks per unit time is greater than a predetermined value, (ii) information that was displayed in an area selected by a drag operation, etc., as information that is highly valuable to the user 20, etc.

[0165] In still another embodiment, the candidate data determination unit 622 identifies information that is highly valuable to the user 20, etc., based on a movement history of the line of sight or viewpoint of the user 20, etc., during a period when the user 20, etc., is viewing, creating, or updating a specific document file. For example, the candidate data determination unit 622 identifies, as information that is highly valuable to the user 20, etc., (i) information that was displayed in an area where the length of stay of the line of sight or viewpoint is greater than a predetermined value, (ii) information that was displayed in an area where a very fast involuntary movement known as a saccade is detected, etc.

[0166] In still another embodiment, the candidate data determination unit 622 identifies information that is highly valuable to the user 20, etc., based on the type of application program capable of executing a specific document file, or the URI of the specific document file. For example, when the above application program is an application program for spreadsheets, presentations, or document creation, the candidate data determination unit 622 identifies at least one selected from the group consisting of character strings, pictures, diagrams, tables, photographs, and videos, and parts thereof, contained in the document, as information that is highly valuable to the user 20, etc. The candidate data determination unit 622 may identify information that is highly valuable to the user 20, etc., by using any known natural language processing technology, pattern recognition technology, and / or image recognition technology.

[0167] In this embodiment, the candidate data determination unit 622 may generate the associated data described above. For example, the candidate data determination unit 622 generates the associated data by associating, for each of one or more pieces of candidate data, identification information of each piece of candidate data with information indicating the position of each piece of candidate data in a document presented by a specific document file. As described above, the associated data may be a database having one or more records corresponding to each of the one or more pieces of candidate data.

[0168] In this embodiment, the value determination unit 624 determines the value of information indicated by each of the one or more candidate data determined by the candidate data determination unit 622. For example, for each of the one or more candidate data included in a specific document file, the value determination unit 624 determines the value of information indicated by each of the candidate data based on at least one of (a) the type of information indicated by each of the candidate data, (b) the operation history of the pointing device by the user 20, etc. during the period when the user 20, etc. is viewing, creating or updating the specific document file, (c) the movement history of the line of sight or viewpoint of the user 20, etc. during the period when the user 20, etc. is viewing, creating or updating the specific document file, (d) the type of application program capable of executing the specific document file, and (e) the URI of the specific document file.

[0169] The value determination unit 624 determines the value of the information indicated by each candidate data based on, for example, a score predetermined for each type of information. Examples of the type of information include a character string, a picture, a diagram, a table, a photograph, a video, etc. The type of information may be a proper noun, a verb, a subject of the verb, an object of the verb, a human face, etc.

[0170] The value determination unit 624 determines the value of the information indicated by each candidate data based on, for example, the type of operation of the pointing device performed on the information indicated by each candidate data. The value determination unit 624 determines the value of the information indicated by each candidate data based on, for example, a score determined in advance for each operation of the pointing device by the user 20 or the like.

[0171] The value determination unit 624 determines the value of the information represented by each candidate data based on the length of time that the gaze or viewpoint of the user 20 or the like stayed on the information represented by each candidate data. For example, information included in an area where the gaze or viewpoint of the user 20 or the like stayed for a long time is estimated to be information of high value to the user 20 or the like.

[0172] The value determination unit 624 determines the value of the information indicated by each candidate data based on a score predetermined for each type of application program, for example. The value determination unit 624 determines the value of the information indicated by each candidate data based on a score predetermined for each URI, for example. For example, it is estimated that information posted on a website published on the Internet is less valuable than information on an intranet.

[0173] In this embodiment, the value determining unit 624 may add information indicating the value of each piece of candidate data to information on each piece of candidate data included in the associated data generated by the candidate data determining unit 622. For example, when the associated data is a database having one or more records corresponding to each piece of candidate data, the value determining unit 624 adds information indicating the value of each piece of candidate data to the record of each piece of candidate data.

[0174] The associated data acquisition unit 614 may be an example of a candidate information acquisition unit. The area data attachment unit 616 may be an example of a candidate information acquisition unit or a candidate area determination unit. The associated data generation unit 620 may be an example of a candidate information acquisition unit.

[0175] FIG. 7 shows an example of a data structure of a data table 700. The data table 700 may be an example of associated data related to a specific document file. The data table 700 may be an example of the value map described above. The data table 700 is acquired by, for example, the associated data acquisition unit 614. The data table 700 is generated by, for example, the value map generation unit 578. The data table 700 may be stored in, for example, the setting information storage unit 520. The data table 700 may be used in a process of anonymizing candidate data, or in a process of deanonymizing anonymized candidate data.

[0176] In this embodiment, the data table 700 includes one or more records corresponding to one or more candidate data. Each of the one or more records stores, for example, a candidate data ID 722, which is identification information for identifying each candidate data, and information indicating a position 724 of each candidate data in a document in association with each other. Examples of the position 724 of each candidate data in a document include a start position of each candidate data, an end position of each candidate data, and the like.

[0177] In this embodiment, each of the one or more records further stores, in association with each other, a candidate data ID 722, information indicating content 726 of each candidate data, information indicating a value 728 of each candidate data, and information indicating a range 732 (sometimes referred to as the range of the candidate area in the original image 200) that a candidate area corresponding to each candidate data occupies in the original image 200. The range of a candidate area in the original image 200 is represented, for example, by the position of a representative point of the candidate area in the original image 200, and the shape and size of the candidate area.

[0178] 8 shows an example of an internal configuration of the periphery monitoring unit 552. In this embodiment, the periphery monitoring unit 552 includes a peripheral image acquisition unit 822, a monitoring target detection unit 824, a viewpoint monitoring unit 826, and a behavior monitoring unit 828.

[0179] In this embodiment, the surrounding image acquisition unit 822 acquires the image 10 captured by the camera 154. As described above, the camera 154 captures an image of the periphery of the display area 142 of the display device 140. Therefore, if a person is present in the periphery of the display area 142, the image 10 that includes the person is obtained.

[0180] In this embodiment, the monitoring target detection unit 824, for example, analyzes the image 10 acquired by the surrounding image acquisition unit 822 to detect a person (the above-mentioned person 22) who is directing his / her gaze toward the display area 142 and is different from the user 20 of the user terminal 120 or the display device 140. The monitoring target detection unit 824 can detect the person 22 by using, for example, any known pattern recognition technology and / or image recognition technology.

[0181] In this embodiment, the viewpoint monitoring unit 826 monitors the viewpoint of the person 22 in the display area 142. For example, the viewpoint monitoring unit 826 analyzes the image 10 acquired by the peripheral image acquisition unit 822 to monitor the viewpoint of the person 22 in the display area 142. The viewpoint monitoring unit 826 outputs information indicating the monitoring result of the viewpoint of the person 22 to the image processing unit 580 (for example, the concealment processing unit 584).

[0182] For example, when the person 22 is detected by image analysis by the monitoring target detection unit 824, the viewpoint monitoring unit 826 estimates the viewpoint of the person 22 in the display area 142. The viewpoint monitoring unit 826 estimates the viewpoint of the person 22 in the display area 142 periodically or at any timing. This allows the viewpoint monitoring unit 826 to track the movement of the viewpoint of the person 22 in the display area 142. The viewpoint monitoring unit 826 can estimate and / or track the viewpoint of the person 22 by using, for example, any known viewpoint estimation technology and / or viewpoint tracking technology.

[0183] When the person 22 is detected by the image analysis of the monitoring target detection unit 824, the viewpoint monitoring unit 826 may simplify the estimation of the viewpoint of the person 22 in the display area 142. The viewpoint monitoring unit 826 may estimate the viewpoint by assuming that the pupil is located at the center of the eyeball, or may estimate the viewpoint based on the normal direction of the face. The normal direction of the face is determined by estimating 3D landmarks of the face using, for example, an xNN-based technology.

[0184] In this embodiment, the behavior monitoring unit 828 monitors the behavior of the person 22. For example, the behavior monitoring unit 828 analyzes the image 10 acquired by the surrounding image acquisition unit 822 to monitor the behavior of the person 22. The behavior monitoring unit 828 outputs information indicating the monitoring result of the behavior of the person 22 to the image processing unit 580 (for example, the anonymization processing unit 584).

[0185] The behavior monitoring unit 828 may monitor short-term behavior of the person 22, or may monitor long-term behavior of the person 22. Long-term behavior may be behavior lasting from 1 minute to 1 hour, from 2 minutes to 30 minutes, from 5 minutes to 30 minutes, from 2 minutes to 20 minutes, or from 5 minutes to 20 minutes.

[0186] The behavior monitoring unit 828 may store physical characteristics of the person 22. Examples of physical characteristics include facial characteristics, skeletal characteristics, and walking characteristics. This allows the behavior monitoring unit 828 to monitor the long-term behavior of the person 22. For example, even if the person 22 moves out of the field of view of the camera 154 and then returns to the field of view of the camera 154, the behavior monitoring unit 828 can continue to monitor the behavior of the person 22.

[0187] In this embodiment, when a person 22 is detected by the image analysis of the monitoring target detection unit 824, the behavior monitoring unit 828 analyzes the image 10 to extract a behavior pattern of the person 22. The behavior monitoring unit 828 determines whether or not the extracted behavior pattern of the person 22 matches a predetermined pattern (for example, the above-mentioned reference pattern).

[0188] Examples of cases where the behavior pattern of the person 22 matches the reference pattern include (i) a case where the behavior pattern of the person 22 matches a first behavior pattern, which is a pattern specific to a person who does not intend to peek at information, or a pattern specific to a person who is performing an action that makes it physically difficult to peek at information compared to when the person is stationary, and (ii) a case where the behavior pattern of the person 22 matches a second behavior pattern, which is a pattern specific to a person who intends to peek at information. The first behavior pattern may be used as a whitelist. The second behavior pattern may be used as a blacklist.

[0189] Examples of the first behavior pattern include (i) a pattern in which the face, head, or upper body is shaken left and right or up and down while suppressing the change in the viewpoint in the display area 142 to be smaller than a predetermined threshold, (ii) a pattern in which the moving speed of the head is greater than a predetermined threshold, (iii) a pattern in which the eyeballs move rapidly, such as saccades, (iv) a pattern in which the angle between the normal vector of the face and the normal vector of the iris is greater than a predetermined threshold, (iv) a pattern in which the face is shaken left and right, (v) a pattern in which the order of the flow of the viewpoint in the display area 142 is irregular, and (vi) a pattern in which the number of parts whose magnitude of movement exceeds a predetermined standard among parts that greatly affect the change in the line of sight exceeds a predetermined threshold. The face described above may be a part of the human body other than the eyes and is an example of a part that greatly affects the change in the line of sight. When the first behavior pattern is detected, even if the person 22 views the information displayed in the display area 142, there is a small possibility that the person 22 recognizes the content of the information.

[0190] Examples of the second behavior pattern include (i) a pattern in which the length of time spent around the display area 142 is longer than a predetermined threshold, (ii) a pattern in which the length of time the gaze is directed to the display area 142 is longer than a predetermined threshold, (iii) a pattern in which the number of times the gaze is directed to the display area 142 during a unit time is longer than a predetermined threshold, (iv) a pattern in which the ratio of the length of the period the gaze is directed to the display area 142 to the length of the unit time is longer than a predetermined threshold, and (v) a pattern in which the gaze is directed to two or more display areas 142 when the user terminal 120 has multiple display areas 142. When the second behavior pattern is detected, there is a possibility that the person 22 is trying to peek at the information displayed in the display area 142.

[0191] The behavior monitoring unit 828 can determine whether the behavior pattern of the person 22 matches a predetermined pattern by using, for example, any known pattern determination technique. The behavior monitoring unit 828 may determine whether the behavior pattern of the person 22 matches a pattern registered in a blacklist. The behavior monitoring unit 828 may determine whether the behavior pattern of the person 22 matches a pattern registered in a whitelist.

[0192] The behavior monitoring unit 828 may execute the above-mentioned determination process based on a rule-based algorithm, or may execute the above-mentioned determination process by utilizing a trained estimation model. The behavior monitoring unit 828 may execute the above-mentioned determination process by comprehensively determining the output results of a plurality of trained estimation models based on a rule-based algorithm. The behavior monitoring unit 828 may divide the display area 142 into a plurality of blocks and execute the above-mentioned determination process for each block.

[0193] The behavior monitoring unit 828 may acquire the physical characteristics of the legitimate user of the user terminal 120 by referring to a database that stores the physical characteristics of legitimate users who have the authority to use the user terminal 120. The behavior monitoring unit 828 may compare the physical characteristics of the legitimate user of the user terminal 120 with the physical characteristics of the person 22 to determine whether or not the person 22 is a legitimate user of the user terminal 120. The behavior monitoring unit 828 may change the criteria for determining whether or not the behavior pattern of the person 22 matches a predetermined pattern between a case where the person 22 is determined to be a legitimate user of the user terminal 120 and a case where the person 22 is not determined to be a legitimate user of the user terminal 120.

[0194] The viewpoint monitoring unit 826 may be an example of a viewpoint estimation unit, and the behavior monitoring unit 828 may be an example of a behavior determination unit.

[0195] 9 illustrates an example of an internal configuration of the anonymization processing unit 584. In this embodiment, the anonymization processing unit 584 includes a mode determination unit 820, a viewing determination unit 832, a recognition determination unit 834, a concealment area determination unit 842, and a concealment mode determination unit 844.

[0196] In this embodiment, the mode determination unit 820 determines the operation mode for the target of anonymization described above. As described above, examples of the operation mode for the target of anonymization include (a) a mode in which the anonymization process is executed as a rule, and the anonymization process is not executed exceptionally in an area estimated to be necessary for the work of the user 20, and (b) a mode in which the anonymization process is not executed as a rule, and the anonymization process is executed exceptionally in an area estimated to be viewed by a person other than the user 20 who is present near the display area 142.

[0197] In one embodiment, the mode determination unit 820 determines the operation mode based on the setting information stored in the setting information storage unit 520. In another embodiment, the mode determination unit 820 determines the operation mode based on an instruction from an administrator of the user terminal 120 or a user 20.

[0198] In this embodiment, the viewing determination unit 832 determines whether or not the aspect of the movement of the viewpoint of the person 22 estimated by the viewpoint monitoring unit 826 matches the above-mentioned conditions regarding the movement of the viewpoint (sometimes referred to as a movement determination process). Examples of the conditions regarding the movement of the viewpoint include (i) a condition (sometimes referred to as a first condition) that the viewpoint of the person 22 is estimated to be located in an end area of ​​a specific candidate area before a period having a predetermined length has elapsed after the viewpoint of the person 22 is estimated to be located in a start area of ​​the specific candidate area, and / or (ii) a condition (sometimes referred to as a second condition) that the viewpoint of the person 22 has moved along a vector from a representative point of a start area of ​​a specific candidate area toward a representative point of an end area of ​​the specific candidate area.

[0199] When it is determined that the manner of movement of the viewpoint matches the condition regarding the movement of the viewpoint, the viewing determination unit 832 determines, for example, to output the output image 300 in which the specific candidate area in the original image 200 is concealed. On the other hand, when it is not determined that the manner of movement of the viewpoint matches the condition regarding the movement of the viewpoint, the viewing determination unit 832 determines, for example, to output the output image 300 in which the specific candidate area in the original image 200 is not concealed.

[0200] As a result, when it is determined that the manner of movement of the viewpoint matches the condition regarding the movement of the viewpoint, the output control unit 550 or the image processing unit 580 outputs, for example, the output image 300 in which the specific candidate region in the original image 200 is concealed. On the other hand, when it is not determined that the manner of movement of the viewpoint matches the condition regarding the movement of the viewpoint, the output control unit 550 or the image processing unit 580 outputs, for example, the output image 300 in which the specific candidate region in the original image 200 is not concealed.

[0201] As a result, if there is a relatively high possibility that the information displayed in the specific candidate area will be viewed in a manner in which the content of the information is easily recognized if the information is not concealed, the concealment process is executed. As a result, the information is concealed before the person 22 recognizes the content of the information displayed in the specific candidate area, and leakage of the information is prevented. On the other hand, according to this embodiment, even if the person 22 views the specific candidate area, if the content of the information is not viewed in a manner in which it is easily recognized, the concealment process is not executed. As a result, unnecessary concealment processes are suppressed, and the user 20's usage experience is improved.

[0202] In this embodiment, the recognition determination 834 determines whether or not the person 22 recognizes the content of the information displayed in the display area 142 if no concealment is applied (this may be referred to as a recognition determination process). The recognition determination 834 may (i) determine whether or not to execute the recognition determination process depending on the determination result of the movement determination process by the browsing determination unit 832, or (ii) execute the recognition determination process in parallel with the movement determination process by the browsing determination unit 832.

[0203] For example, if it is determined that the manner of movement of the viewpoint matches the condition regarding the movement of the viewpoint, the recognition determination 834 executes the above-mentioned recognition determination process. On the other hand, if it is not determined that the manner of movement of the viewpoint matches the condition regarding the movement of the viewpoint, the recognition determination 834 does not need to execute the above-mentioned recognition determination process.

[0204] In one embodiment, the recognition determination 834 determines, based on the monitoring result of the behavior monitoring unit 828, whether or not the person 22 would recognize the content of the information displayed in the display area 142 if anonymization was not applied. For example, if the behavior monitoring unit 828 determines that the behavior pattern of the person 22 matches a reference pattern, the recognition determination 834 determines that the person 22 would recognize the content of the information displayed in the display area 142 if anonymization was not applied.

[0205] In another embodiment, the recognition determination 834 derives the degree of possibility (sometimes referred to as the degree of recognition) that the person 22 would recognize the content of the information displayed in the display area 142 if the information were not concealed, based on the monitoring result of the behavior monitoring unit 828. For example, when the monitoring target detection unit 824 detects the person 22, the recognition determination 834 derives the degree of recognition based on the behavior pattern of the person 22 obtained by the behavior monitoring unit 828 analyzing the surrounding image. The degree of recognition may be expressed by a continuous numerical value within a range of a predetermined lower limit value and an upper limit value, or may be expressed by a gradual division. For example, when the degree of recognition of the person 22 exceeds a predetermined degree, the recognition determination 834 determines that the person 22 would recognize the content of the information displayed in the display area 142 if the information were not concealed.

[0206] When it is determined that the person 22 would recognize the content of the information displayed in the display area 142 if the information were not concealed, the recognition decision 834 determines, for example, to output the output image 300 in which a specific candidate area in the original image 200 is concealed. On the other hand, when it is not determined that the person 22 would recognize the content of the information displayed in the display area 142 if the information were not concealed, the recognition decision 834 determines, for example, to output the output image 300 in which the specific candidate area in the original image 200 is not concealed.

[0207] According to one embodiment, the recognition determination 834 may determine to output the output image 300 in which a specific candidate region in the original image 200 is concealed when (a) it is determined that the behavior pattern of the person 22 matches a predetermined pattern and the manner of the movement of the viewpoint matches a predetermined condition. On the other hand, (b) if it is not determined that the behavior pattern of the person 22 matches a predetermined pattern or the manner of the movement of the viewpoint matches a predetermined condition, the recognition determination 834 may determine to output the output image 300 in which the specific candidate region in the original image 200 is not concealed.

[0208] As a result, (a) when it is determined that the behavior pattern of the person 22 matches a predetermined pattern and the manner of movement of the viewpoint matches a predetermined condition, the output control unit 550 or the image processing unit 580 outputs, for example, the output image 300 in which a specific candidate area in the original image 200 is concealed. On the other hand, (b) when it is not determined that the behavior pattern of the person 22 matches a predetermined pattern or the manner of movement of the viewpoint matches a predetermined condition, the output control unit 550 or the image processing unit 580 outputs, for example, the output image 300 in which the specific candidate area in the original image 200 is not concealed.

[0209] According to another embodiment, the recognition determination 834 may determine to output the output image 300 in which a specific candidate region in the original image 200 is concealed (a) when it is determined that the degree of recognition of the person 22 exceeds a predetermined degree and the manner of movement of the viewpoint matches a predetermined condition. On the other hand, (b) when the degree of recognition of the person 22 does not exceed a predetermined degree or when it is not determined that the manner of movement of the viewpoint matches a predetermined condition, the recognition determination 834 may determine to output the output image 300 in which the specific candidate region in the original image 200 is not concealed.

[0210] As a result, (a) when it is determined that the degree of recognition of the person 22 exceeds a predetermined degree and the manner of movement of the viewpoint matches a predetermined condition, the output control unit 550 or the image processing unit 580 outputs, for example, the output image 300 in which a specific candidate area in the original image 200 is concealed. On the other hand, (b) when the degree of recognition of the person 22 does not exceed a predetermined degree or when it is not determined that the manner of movement of the viewpoint matches a predetermined condition, the output control unit 550 or the image processing unit 580 outputs, for example, the output image 300 in which the specific candidate area in the original image 200 is not concealed.

[0211] In this embodiment, the concealment region determination unit 842 determines a region to be concealed in the original image 200 (sometimes referred to as a concealment region). As a result, for example, an output image 300 in which an object 330 is arranged in the concealment region of the original image 200 may be generated. As described above, the output image 300 may be generated by the image processing unit 580 or the output image generation unit 588. The output image 300 may be generated by the concealment processing unit 584 (for example, the concealment region determination unit 842 or the concealment mode determination unit 844).

[0212] In this embodiment, the concealment region determination unit 842 determines at least one of the one or more candidate regions as a concealment region based on the manner of movement of the viewpoint of the person 22 estimated by the viewpoint monitoring unit 826. For example, when the manner of movement of the viewpoint matches at least one of the first condition and the second condition described above, the concealment region determination unit 842 determines at least the specific candidate region described above as a concealment region.

[0213] The concealment area determination unit 842 may determine whether or not to include other candidate areas located near the specific candidate area described above in the concealment area based on the value of the information included in the specific candidate area (e.g., the candidate area 260). The concealment area determination unit 842, for example, refers to the value map generated by the value map generation unit 578 to obtain the value of the information included in the specific candidate area described above. The concealment mode determination unit 844 determines other candidate areas included in the concealment area such that, for example, the greater the value of the information, the greater the maximum value of the distance between the other candidate areas included in the concealment area and the specific candidate area described above.

[0214] In this embodiment, the concealment mode determination unit 844 determines the mode of concealment to be applied to each of one or more concealment regions. As described above, examples of the concealment mode include the type or method of concealment processing, the strength of concealment processing, and the like.

[0215] The anonymization mode determination unit 844 determines the anonymization mode in each anonymization area based on the value of the information included in each anonymization area, for example. The anonymization mode determination unit 844 may obtain the value of the information included in each anonymization area by referring to the value map generated by the value map generation unit 578.

[0216] The anonymization mode determination unit 844 determines the anonymization mode in each anonymization area so that, for example, the higher the value of the information, the lower the anonymization method with respect to viewability is adopted. The anonymization mode determination unit 844 determines the anonymization mode in each anonymization area so that, for example, the higher the value of the information, the stronger the anonymization process is.

[0217] The viewing determination unit 832 may be an example of a movement determination unit. The recognition determination unit 834 may be an example of a behavior determination unit or a recognition degree derivation unit. The concealment area determination unit 842 may be an example of a target area determination unit or a display image generation unit. The specific candidate area may be an example of a first candidate area.

[0218] Fig. 10 illustrates an example of information processing in the viewing determination unit 832. In this embodiment, the details of the movement determination process in the viewing determination unit 832 will be described using an example in which the person 22 views the candidate area 260 described in relation to Fig. 2.

[0219] As described above, the viewing determination unit 832 determines whether or not the manner of movement of the viewpoint 50 of the person 22 estimated by the viewpoint monitoring unit 826 matches the condition regarding the movement of the viewpoint. Also, the vector 40 of the candidate region 260 is a vector directed from the representative point 272 of the start region 262 of the candidate region 260 to the representative point 274 of the end region 264 of the candidate region 260.

[0220] In this embodiment, when the viewpoint 50 of the person 22 moves above the candidate region 260 in the direction from the start region 262 to the end region 264, the viewing determination unit 832 determines that the manner of movement of the viewpoint 50 of the person 22 meets the condition regarding the movement of the viewpoint. However, as shown in Fig. 10, the vector 40 is represented as a straight line. On the other hand, the movement history of the viewpoint 50 of the person 22 is represented as a curved line.

[0221] Therefore, according to one embodiment, after the viewpoint 50 of the person 22 is estimated to be located in the start region 262 of the candidate region 260, which is an example of a specific candidate region, if the viewpoint 50 of the person 22 is estimated to be located in the end region 264 of the candidate region 260 before a period of a predetermined length has elapsed, the viewing determination unit 832 determines that the viewpoint 50 of the person 22 has moved along the vector 40. Also, in this case, the viewing determination unit 832 determines that the viewpoint 50 of the person 22 has moved above the candidate region 260 in the direction from the start region 262 to the end region 264.

[0222] In another embodiment, when the error between the movement history of the viewpoint 50 of the person 22 and the vector 40 is smaller than a predetermined value, the viewing determination unit 832 determines that the viewpoint 50 of the person 22 has moved along the vector 40. In this case, the viewing determination unit 832 determines that the viewpoint 50 of the person 22 has moved above the candidate region 260 in the direction from the start region 262 to the end region 264.

[0223] In still another embodiment, when the error between the moving average line of the estimated position of the viewpoint 50 of the person 22 and the vector 40 is smaller than a predetermined value, the viewing determination unit 832 determines that the viewpoint 50 of the person 22 has moved along the vector 40. In this case, the viewing determination unit 832 also determines that the viewpoint 50 of the person 22 has moved above the candidate region 260 in the direction from the start region 262 to the end region 264. 11 shows an example of information processing in the leakage prevention support unit 586. According to this embodiment, the outline of the processing for encouraging the user 22 to take a specific action to prevent information leakage will be described using an example in which a person 24 other than the user 22 of the user terminal 120 is present behind the user 22.

[0224] In this embodiment, the person 24 is located to the left and behind the user 22. Therefore, when the user 22 rotates the user terminal 120 horizontally in a clockwise direction, the person 24 can easily view the display area 142. On the other hand, when the user 22 rotates the user terminal 120 horizontally in a counterclockwise direction, it becomes difficult for the person 24 to view the display area 142.

[0225] Therefore, according to the present embodiment, the leakage prevention support unit 586 controls the output image generation unit 588 to display the guidance image 1118 in a part of the display area 142, thereby controlling the behavior of the user 22. Specifically, the leakage prevention support unit 586 adjusts the display position of the guidance image 1118 to prompt the user 22 to rotate the user terminal 120 horizontally counterclockwise.

[0226] The leakage prevention support unit 586 first acquires data of an image captured by the camera 154. The leakage prevention support unit 586 analyzes the image captured by the camera 154 to determine whether or not two or more people are present around the display area 142. If two or more people are present around the display area 142, the leakage prevention support unit 586 identifies the relative positional relationship between the people. The leakage prevention support unit 586 determines the rotation direction of the user terminal 120 in which information leakage is further suppressed based on the relative positional relationship between the objects. As described above, in this embodiment, information leakage is further suppressed by rotating horizontally counterclockwise.

[0227] The leakage prevention support unit 586 controls the output image generation unit 588 to output an image 1102 to the display area 142. In the image 1102, a part of the right side of the image output area 214 is covered by a guidance image 1118. This enables the leakage prevention support unit 586 to prompt the user 22 to rotate the user terminal 120 counterclockwise.

[0228] When the user 22 rotates the user terminal 120 counterclockwise, the leakage prevention support unit 586 controls the output image generation unit 588 to output the image 1104 in the display area 142. The guidance image 1118 is not displayed in the image 1104, and the user 22 can continue working as usual.

[0229] On the other hand, when the user 22 rotates the user terminal 120 clockwise, the leakage prevention support unit 586 controls the output image generation unit 588 to output the image 1106 in the display area 142. A larger portion of the image 1106 is covered by the guidance image 1118 compared to the image 1102. This allows the leakage prevention support unit 586 to prompt the user 22 to rotate the user terminal 120 counterclockwise.

[0230] 12 shows an example of a computer 3000 in which aspects of the present invention may be embodied in whole or in part. At least a part of the information leakage prevention system 100 may be realized by the computer 3000. For example, at least a part of the user terminal 120 is realized by the computer 3000 or a part thereof. For example, at least a part of the user terminal 120, the control device 170 is realized by the computer 3000 or a part thereof.

[0231] A program installed on the computer 3000 may cause the computer 3000 to function as or perform operations associated with an apparatus according to an embodiment of the present invention or one or more "parts" of the apparatus, and / or to perform a process or steps of the process according to an embodiment of the present invention. Such a program may be executed by the CPU 3012 to cause the computer 3000 to perform certain operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0232] The computer 3000 according to this embodiment includes a CPU 3012, a RAM 3014, a GPU 3016, and a display device 3018, which are connected to each other by a host controller 3010. The computer 3000 also includes input / output units such as a communication interface 3022, a hard disk drive 3024, a DVD-ROM drive 3026, and an IC card drive, which are connected to the host controller 3010 via an input / output controller 3020. The computer also includes legacy input / output units such as a ROM 3030 and a keyboard 3042, which are connected to the input / output controller 3020 via an input / output chip 3040.

[0233] The CPU 3012 operates according to a program stored in the ROM 3030 and the RAM 3014, thereby controlling each unit. The GPU 3016 acquires image data generated by the CPU 3012 into a frame buffer or the like provided in the RAM 3014 or into the GPU 3016 itself, and causes the image data to be displayed on the display device 3018.

[0234] The communications interface 3022 communicates with other electronic devices via a network. The hard disk drive 3024 stores programs and data used by the CPU 3012 in the computer 3000. The DVD-ROM drive 3026 reads programs or data from the DVD-ROM 3001 and provides the programs or data to the hard disk drive 3024 via the RAM 3014. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0235] The ROM 3030 stores therein a boot program or the like executed by the computer 3000 upon activation, and / or a program that depends on the hardware of the computer 3000. The input / output chip 3040 may also connect various input / output units to the input / output controller 3020 via a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0236] The programs are provided by a computer-readable storage medium such as a DVD-ROM 3001 or an IC card. The programs are read from the computer-readable storage medium, installed in the hard disk drive 3024, the RAM 3014, or the ROM 3030, which are also examples of computer-readable storage media, and executed by the CPU 3012. Information processing described in these programs is read by the computer 3000, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by realizing an operation or processing of information according to the use of the computer 3000.

[0237] For example, when communication is performed between computer 3000 and an external device, CPU 3012 may execute a communication program loaded in RAM 3014 and instruct communication interface 3022 to perform communication processing based on the processing described in the communication program. Under the control of CPU 3012, communication interface 3022 reads transmission data stored in a transmission buffer area provided in RAM 3014, hard disk drive 3024, DVD-ROM 3001, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0238] Furthermore, CPU 3012 may cause all or a necessary portion of a file or database stored in an external recording medium such as hard disk drive 3024, DVD-ROM drive 3026 (DVD-ROM 3001), an IC card, etc. to be read into RAM 3014, and perform various types of processing on the data on RAM 3014. CPU 3012 may then write back the processed data to the external recording medium.

[0239] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and undergo information processing. The CPU 3012 may perform various types of processing on the data read from the RAM 3014, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequence of the program, and writes back the results to the RAM 3014. The CPU 3012 may also search for information in a file, database, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 3012 may search for an entry whose attribute value of the first attribute matches a specified condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0240] The above-described programs or software modules may be stored in a computer-readable storage medium on the computer 3000 or in the vicinity of the computer 3000. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the above-described programs to the computer 3000 via the network.

[0241] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications or improvements can be made to the above embodiments. In addition, the matters described for a specific embodiment can be applied to other embodiments to the extent that they are not technically inconsistent. It is clear from the claims that such modified or improved forms can also be included in the technical scope of the present invention.

[0242] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and may be realized in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is explained using "first," "next," etc. for convenience, it does not mean that it is essential to perform the process in this order.

[0243] For example, the present specification describes the following: (Item A-1) An information processing device that generates a display image to be displayed in a display area of ​​an image display device based on a generated image generated by an image generating device, a generated image acquisition unit for acquiring the generated image; a display image generating unit that generates the display image based on the generated image; a surrounding image acquisition unit that acquires a surrounding image that is an image captured by an imaging device that captures an image of the periphery of the display area; a monitoring target detection unit that analyzes the peripheral image and detects a monitoring target, which is a monitoring target device that is an imaging device facing an imaging direction toward the display area; Equipped with the display image generation unit determines, when the monitored device is detected, to output the display image in which a part of the generated image is concealed; Information processing device. (Item A-2) the display image generating unit determines whether or not to output the display image in which a portion of the generated image is concealed, based on the shooting direction and a silhouette of a user of the image generating device. The information processing device according to item A-1. (Item B-1) An information processing device that generates a display image to be displayed in a display area of ​​an image display device based on a generated image generated by an image generating device, a generated image acquisition unit for acquiring the generated image; a display image generating unit that generates the display image based on the generated image; a surrounding image acquisition unit that acquires a surrounding image that is an image captured by an imaging device that captures an image of the periphery of the display area; a surveillance target detection unit that analyzes the peripheral image to detect a surveillance target who is directing his / her gaze toward the display area and is a person other than a user of the image display device; a viewpoint estimation unit that estimates a viewpoint of the person to be monitored in the display area when the person to be monitored is detected; Equipped with The display image generation unit determines whether or not to output the display image in which a part of the generated image is concealed, based on the viewpoint and the silhouette of the user. Information processing device. (Item C-1) An information processing device that generates a display image to be displayed in a display area of ​​an image display device based on a generated image generated by an image generating device, a generated image acquisition unit for acquiring the generated image; a display image generating unit that generates the display image based on the generated image; a surrounding image acquisition unit that acquires a surrounding image that is an image captured by an imaging device that captures an image of the periphery of the display area; a monitoring target detection unit that analyzes the peripheral image to detect a monitoring target that is at least one of (i) a monitoring target person who is directing his / her gaze toward the display area and is a person other than a user of the image display device, and (ii) a monitoring target device that is an imaging device that is directing an imaging direction toward the display area; a viewpoint estimation unit that estimates, when the monitoring target is detected, at least one of (i) a viewpoint of the monitoring target person in the display area and (ii) an imaging position of the monitoring target device in the display area; Equipped with The display image generation unit determines whether to output the display image in which a part of the generated image is concealed, based on at least one of the viewpoint and the imaging position, and a silhouette of the user. Information processing device. (Item C-2) a movement determination unit that determines whether or not a movement mode of at least one of the viewpoint and the imaging position estimated by the viewpoint estimation unit matches a predetermined condition, The information processing device according to item C-1. (Item C-3) the viewpoint estimation unit estimates a viewpoint of the monitoring target person in the display area when the monitoring target person is detected; the movement determination unit determines whether or not a manner of movement of the viewpoint estimated by the viewpoint estimation unit matches a predetermined condition; The display image generation unit determines whether or not to output the display image in which a part of the generated image is concealed, based on the viewpoint and the silhouette of the user. The information processing device according to item C-1 or C-2. (Item C-4) the display image generating unit determines to output the display image in which a part of the generated image is concealed when (i) it is determined that the manner of movement of the viewpoint matches the predetermined condition, or (ii) the monitored device is detected; The information processing device according to any one of items C-1 to C-3. [Explanation of symbols]

[0244] 10 Images 20 users 22 People 24 People 30 Candidate Data 40 Vector 50 Viewpoints 100 Information Leakage Prevention System 120 User terminals 130 Case 140 Display device 142 Display area 150 Environmental Sensors 152 Distance Sensor 154 Cameras 156 Radio Detector 160 Input Devices 170 Control device 200 original images 210 Window 212 Control Area 214 Image Output Area 216 Presentation information 222 Objects 224 Objects 230 Window 232 Control Area 234 Image Output Area 236 Presentation information 240 Window 242 Control Area 244 Image Output Area 246 Presentation information 260 candidate area 262 Start area 264 End area 272 representative points 274 representative points 300 output images 326 Objects 328 Objects 330 Objects 520 Setting information storage section 532 Program Control Unit 534 Window control section 540 Input control section 550 Output control section 552 Periphery Surveillance Department 560 Original Image Generation Unit 574 Gaze information acquisition unit 576 Operation information acquisition section 578 Value Map Generation Department 580 Image Processing Unit 584 Concealment Processing Unit 586 Leak Prevention Support Department 588 Output Image Generation Unit 612 Document File Acquisition Unit 614 Accompanying Data Acquisition Unit 616 Area Data Assignment Unit 620 Accompanying Data Generation Unit 622 Candidate Data Decision Unit 624 Value Determination Department 700 Data Table 722 Candidate Data ID 724 position 726 Contents 728 Value 732 Range 820 Mode determination unit 822 Peripheral image acquisition unit 824 Monitoring target detection unit 826 Viewpoint Monitoring Department 828 Behavioral Surveillance Division 832 Viewing Judgment Department 834 Recognition judgment 842 Concealment area determination unit 844 Concealment mode determination unit 1102 images 1104 images 1106 images 1118 Guidance Image 3000 Computers 3001 DVD-ROM 3010 Host Controller 3012 CPU 3014 RAM 3016 GPU 3018 Display Device 3020 I / O Controller 3022 Communication Interface 3024 Hard Disk Drive 3026 DVD-ROM drive 3030 ROM 3040 I / O chip 3042 Keyboard

Claims

1. An information processing device that generates a display image to be displayed in a display area of ​​an image display device based on a generated image generated by an image generating device, A generated image acquisition unit that acquires the generated image; a display image generating unit that generates the display image based on the generated image; a surrounding image acquisition unit that acquires a surrounding image that is an image captured by an imaging device that captures an image of the periphery of the display area; a surveillance target detection unit that analyzes the peripheral image to detect a surveillance target who is directing his / her gaze toward the display area and is a person other than a user of the image display device; a viewpoint estimation unit that estimates a viewpoint of the person to be monitored in the display area when the person to be monitored is detected; a movement determination unit that determines whether or not the manner in which the viewpoint is moved estimated by the viewpoint estimation unit matches a predetermined condition; a candidate area determination unit that determines, in the display area, one or more candidate areas in which one or more pieces of data that may be subject to anonymity are displayed; a target area determination unit that determines, based on a state of movement of the viewpoint, at least one candidate area among the one or more candidate areas as a target area that is an area in the display area to which the concealment is to be performed; Equipped with when it is determined that the manner of movement of the viewpoint matches the predetermined condition, the display image generation unit determines to output the display image in which the target region in the generated image is concealed; Each of the one or more candidate regions comprises: a start region located at the start position of each candidate region; an end region disposed at the end position of each candidate region; Including, a range of the end region in the display region is different from a range of the start region in the display region; The predetermined condition includes at least one of: (i) a first condition that, after the viewpoint is estimated to be located in the start region of a first candidate region among the one or more candidate regions, the viewpoint is estimated to be located in the end region of the first candidate region before a period of time having a predetermined length has elapsed; and (ii) a second condition that the viewpoint is estimated to have moved along a vector from a representative point of the start region of the first candidate region toward a representative point of the end region of the first candidate region. Information processing device.

2. An information processing device that generates a display image to be displayed in a display area of ​​an image display device based on a generated image generated by an image generating device, A generated image acquisition unit that acquires the generated image; a display image generating unit that generates the display image based on the generated image; a surrounding image acquisition unit that acquires a surrounding image that is an image captured by an imaging device that captures an image of the periphery of the display area; a surveillance target detection unit that analyzes the peripheral image to detect a surveillance target who is directing his / her gaze toward the display area and is a person other than a user of the image display device; a viewpoint estimation unit that estimates a viewpoint of the person to be monitored in the display area when the person to be monitored is detected; a movement determination unit that determines whether or not the manner in which the viewpoint is moved estimated by the viewpoint estimation unit matches a predetermined condition; a candidate information acquisition unit that acquires candidate information, the candidate information being information that has been created in advance based on the contents of an electronic file used in a process for the image generation device to generate at least a part of the generated image, and being information for identifying one or more candidate data that are data that may be subject to anonymity in the electronic file; a candidate area determination unit that determines, in the display area, one or more candidate areas in which each of the one or more candidate data items specified by the candidate information is to be displayed; a target area determination unit that determines, based on a state of movement of the viewpoint, at least one candidate area among the one or more candidate areas as a target area that is an area to be concealed in the display area; Equipped with each of the one or more candidate data is data used by the image generating device to generate a different portion of the generated image; when it is determined that the manner of movement of the viewpoint matches the predetermined condition, the display image generation unit determines to output the display image in which the target region in the generated image is concealed; Each of the one or more candidate regions comprises: a start region located at the start position of each candidate region; an end region disposed at the end position of each candidate region; Including, a range of the end region in the display region is different from a range of the start region in the display region; the candidate information includes representative point information that is information for identifying a representative point of the start region of each of the one or more candidate regions and a representative point of the end region of each of the one or more candidate regions, The predetermined condition includes at least one of: (i) a first condition that, after the viewpoint is estimated to be located in the start region of a first candidate region among the one or more candidate regions, the viewpoint is estimated to be located in the end region of the first candidate region before a period of time having a predetermined length has elapsed; and (ii) a second condition that the viewpoint is estimated to have moved along a vector from a representative point of the start region of the first candidate region toward a representative point of the end region of the first candidate region. Information processing device.

3. the target region determination unit determines at least the first candidate region as the target region when a manner of movement of the viewpoint satisfies at least one of the first condition and the second condition. The information processing device according to claim 1 .

4. and a behavior determination unit that, when the monitoring target detection unit detects the monitoring target, determines whether or not a behavior pattern of the monitoring target obtained by analyzing the surrounding image matches a predetermined pattern, The display image generating unit is (a) when it is determined that the behavior pattern of the surveillance target person matches the predetermined pattern and the manner of movement of the viewpoint matches the predetermined condition, determining to output the display image in which the target area in the generated image is concealed; (b) when it is not determined that the behavior pattern of the monitored person matches the predetermined pattern, or when it is not determined that the manner of movement of the viewpoint matches the predetermined condition, determining to output the generated image as the display image; The information processing device according to claim 1 .

5. a recognition degree derivation unit that, when the surveillance target detection unit detects the surveillance target, derives a recognition degree indicating a degree of possibility that the surveillance target will recognize the content of the information displayed in the display area, unless concealment is performed, based on a behavior pattern of the surveillance target obtained by analyzing the surrounding image, The display image generating unit is (a) when it is determined that the degree of recognition exceeds a predetermined degree and that the manner of movement of the viewpoint matches the predetermined condition, determining to output the display image in which the target region in the generated image is concealed; (b) determining that the generated image is to be output as the display image when the degree of recognition does not exceed a predetermined degree, or when it is not determined that the manner of movement of the viewpoint matches the predetermined condition; The information processing device according to claim 1 .

6. A program for causing a computer to function as the information processing device according to any one of claims 1 to 5.

7. 1. A method for generating a display image to be displayed on a display area of ​​an image display device based on a generated image generated by an image generating device, comprising: A generated image acquisition step in which a computer acquires the generated image; a display image generating step in which the computer generates the display image based on the generated image; a peripheral image acquisition step in which the computer acquires a peripheral image which is an image captured by an imaging device that captures an image of the periphery of the display area; a monitoring target detection step in which the computer analyzes the peripheral image to detect a monitoring target who is directing his / her gaze toward the display area and is a person other than a user of the image display device; a viewpoint estimating step in which, when the monitoring target person is detected, the computer estimates a viewpoint of the monitoring target person in the display area; a movement determination step in which the computer determines whether or not the manner of movement of the viewpoint estimated in the viewpoint estimation step matches a predetermined condition; a candidate area determination step in which the computer determines, in the display area, one or more candidate areas in which one or more pieces of data that may be subject to anonymity are displayed; a target area determination step in which the computer determines, based on a manner of movement of the viewpoint, at least one candidate area among the one or more candidate areas as a target area that is an area in the display area to which the concealment is to be performed; having the display image generating step includes a step of, when it is determined that the manner of movement of the viewpoint matches the predetermined condition, outputting the display image in which the target region in the generated image is concealed by the computer; Each of the one or more candidate regions comprises: a start region located at the start position of each candidate region; an end region disposed at the end position of each candidate region; Including, a range of the end region in the display region is different from a range of the start region in the display region; The predetermined condition includes at least one of: (i) a first condition that, after the viewpoint is estimated to be located in the start region of a first candidate region among the one or more candidate regions, the viewpoint is estimated to be located in the end region of the first candidate region before a period of time having a predetermined length has elapsed; and (ii) a second condition that the viewpoint is estimated to have moved along a vector from a representative point of the start region of the first candidate region toward a representative point of the end region of the first candidate region. method.

8. 1. A method for generating a display image to be displayed on a display area of ​​an image display device based on a generated image generated by an image generating device, comprising: A generated image acquisition step in which a computer acquires the generated image; a display image generating step in which the computer generates the display image based on the generated image; a peripheral image acquisition step in which the computer acquires a peripheral image which is an image captured by an imaging device that captures an image of the periphery of the display area; a monitoring target detection step in which the computer analyzes the peripheral image to detect a monitoring target who is directing his / her gaze toward the display area and is a person other than a user of the image display device; a viewpoint estimating step in which, when the monitoring target person is detected, the computer estimates a viewpoint of the monitoring target person in the display area; a movement determination step in which the computer determines whether or not the manner of movement of the viewpoint estimated in the viewpoint estimation step matches a predetermined condition; a candidate information acquisition step of acquiring candidate information, the candidate information being information created in advance based on the contents of an electronic file used in a process for generating at least a part of the generated image by the image generating device, and being information for identifying one or more candidate data that may be subject to anonymization in the electronic file; a candidate area determination step of determining, in the display area, one or more candidate areas in which each of the one or more candidate data items specified by the candidate information is to be displayed; a target area determination step of determining, based on a manner of movement of the viewpoint, at least one candidate area among the one or more candidate areas as a target area that is an area to be concealed in the display area; having each of the one or more candidate data is data used by the image generating device to generate a different portion of the generated image; the display image generating step includes a step of, when it is determined that the manner of movement of the viewpoint matches the predetermined condition, outputting the display image in which the target region in the generated image is concealed by the computer; Each of the one or more candidate regions comprises: a start region located at the start position of each candidate region; an end region disposed at the end position of each candidate region; Including, a range of the end region in the display region is different from a range of the start region in the display region; the candidate information includes representative point information that is information for identifying a representative point of the start region of each of the one or more candidate regions and a representative point of the end region of each of the one or more candidate regions, The predetermined condition includes at least one of: (i) a first condition that, after the viewpoint is estimated to be located in the start region of a first candidate region among the one or more candidate regions, the viewpoint is estimated to be located in the end region of the first candidate region before a period of time having a predetermined length has elapsed; and (ii) a second condition that the viewpoint is estimated to have moved along a vector from a representative point of the start region of the first candidate region toward a representative point of the end region of the first candidate region. method.

Images