Information presentation device and program

Abstract

According to one embodiment, the present invention first determines whether the type of attention aroused in a user is bottom-up attention or top-down attention. Next, the present invention calculates degrees for the visual appeal of objects to the user with respect to at least one of bottom-up attention and top-down attention on the basis of the determination results for the type of attention and sets a diminishment area that includes objects that will attract the gaze of the user in a work environment on the basis of the calculated degrees of visual appeal. Then the present invention generates diminishment processing data for performing diminishment processing on the set diminishment area and performs diminishment processing on visual information that represents the work environment on the basis of the generated diminishment processing data.

Description

Information presentation device and program

[0003]

[0001] One aspect of this invention relates to, for example, an information presentation device having a concealment processing function for video data obtained by imaging a work environment and a program used in this information presentation device.

[0002] The human attention function has the function of restricting unnecessary information from among a plurality of pieces of information perceived by the five senses and recognizing only the necessary information. Therefore, for example, when performing desk work, it is possible to concentrate on the work by selectively recognizing only the information necessary for the work. <​​​​​​​​​​However, the method described in Non-Patent Document 1 requires the user to pre-set the objects or areas to be concealed, i.e., the objects or areas related to bottom-up attention or top-down attention. As a result, this setup is time-consuming and requires effort, and if the objects to be concealed are not set appropriately, the expected concealment effect cannot be obtained.

[0007] This invention was made in view of the above circumstances and aims to provide a technology that allows users to accurately set the object to be hidden without relying on user settings.

[0008] To solve the above problems, one aspect of the image information processing device according to the present invention is an information presentation device that presents to a user visual information representing a work environment after performing a masking process on objects among a plurality of objects included in the work environment that have little relation to the work, and comprises: a first processing unit that determines, based on the degree of change in the user's gaze direction, whether the type of attention generated by the user is bottom-up attention, in which the user passively directs their gaze towards the object, or top-down attention, in which the user actively directs their gaze towards the object; a second processing unit that calculates the degree to which the user is attracted to the object for at least one of the bottom-up attention and the top-down attention based on the determination result of the attention type; a third processing unit that sets a masking target range including the object that attracts the user's gaze to the work environment based on the degree to which the user is attracted to the object; and a fourth processing unit that generates masking processing data for masking the masking target range and performs masking processing on the visual information representing the work environment based on the generated masking processing data.

[0009] According to one aspect of this invention, the degree to which the user's gaze is drawn is calculated for at least one of bottom-up attention and top-down attention, and based on the calculated degree of drawing attention, the area to be obscured, including the object that the user's gaze is drawn to, is automatically and accurately set. As a result, the user does not need to set the area to be obscured, thereby reducing the user's workload.

[0010] In other words, according to one aspect of this invention, it is possible to provide a technology that enables the precise setting of the object to be hidden without relying on user settings.

[0011] Figure 1 is a diagram showing the overall configuration of an information presentation system according to one embodiment of the present invention. Figure 2 is a block diagram showing an example of the hardware configuration of an information presentation device according to one embodiment of the present invention. Figure 3 is a block diagram showing an example of the software configuration of an information presentation device according to one embodiment of the present invention. Figure 4 is a flowchart showing an example of the processing procedure and processing content of information presentation control using concealment processing, which is executed by the control unit of the information presentation device shown in Figure 3. Figure 5 is a flowchart showing an example of the processing procedure and processing content of the attraction degree calculation process in the information presentation control shown in Figure 4. Figure 6 is a diagram showing an example of a user's work environment. Figure 7 is a diagram showing an example of the correspondence between object names and the set values ​​of the top-down attraction degree. Figure 8 is a diagram showing an example of the calculation result of the top-down attraction degree for objects present in the work environment shown in Figure 6.

[0012] Embodiments of this invention will be described below with reference to the drawings.

[0013] [One Embodiment] (Configuration Example) (1) System diagram 1 is a diagram showing the overall configuration of an information presentation system according to one embodiment of the present invention.

[0014] In one embodiment of the information presentation system, a user (US) wears an information presentation device (HD) on their head. The camera (CM) on the information presentation device (HD) captures images of the work environment, including a work terminal (TM) such as a personal computer, and displays the images on the display device (DP) of the information presentation device (HD). The user (US) then operates the work terminal (TM) while viewing the displayed images, enabling them to perform predetermined tasks such as document creation and image editing.

[0015] (2) Information display device HD Figures 2 and 3 are block diagrams showing examples of the hardware configuration and software configuration of an information display device HD according to one embodiment of the present invention.

[0016] The information display device HD is composed of, for example, a head-mounted display (HMD). The information display device HD includes a control unit 1 that uses a hardware processor such as a central processing unit (CPU). The control unit 1 is connected to a storage medium having a program storage unit 2 and a data storage unit 3, and an input / output interface (hereinafter referred to as I / F) unit 4 via a bus 5.

[0017] The input / output interface 4 is connected to a camera CM and a display device DP, respectively. The camera CM captures images of the work environment, including the work terminal TM, and outputs the video data. The display device DP displays the image obtained by applying a blackout process to the video data captured by the camera CM.

[0018] The program storage unit 2 is configured, for example, by combining a non-volatile memory that can be written to and read at any time, such as an SSD (Solid State Drive), and a non-volatile memory such as ROM (Read Only Memory), and stores application programs necessary to execute various controls according to one embodiment, in addition to middleware such as an OS (Operating System). Hereafter, the OS and each application program will be collectively referred to as a program.

[0019] The data storage unit 3 is, for example, a combination of a non-volatile memory such as an SSD that can be written to and read at any time, and a volatile memory such as RAM (Random Access Memory), as a storage medium. In its storage area, a video data storage unit 31 and an effect data storage unit 32 are provided as the main storage units according to one embodiment of this invention.

[0020] The video data storage unit 31 is used as a buffer memory for temporarily storing video data captured by the camera CM.

[0021] The effect data storage unit 32 stores several types of pre-prepared censorship effect data. The types of censorship effect data include, for example, those that perform "transparency" or "outline" processing to change the appearance of an object by overlaying it with a background, those that perform "blur" processing by changing the resolution, those that perform "desaturation" or "de-saturation" processing by changing the saturation, and those that perform "de-contrast" processing by changing the brightness.

[0022] The control unit 1 includes, as processing functions necessary to realize one embodiment of this invention, a video data acquisition processing unit 11, an image change calculation processing unit 12, a bottom-up attraction degree calculation processing unit 13, a top-down attraction degree calculation processing unit 14, an attraction target determination processing unit 15, a concealment target range setting processing unit 16, an effect selection processing unit 17, a concealment processing data generation processing unit 18, and a video synthesis processing unit 19.

[0023] Each of the above-mentioned processing units 11 to 19 is implemented by causing the hardware processor of the control unit 1 to execute an application program stored in the program storage unit 2. Note that some or all of the above-mentioned processing units 11 to 19 may be implemented using hardware such as LSI (Large Scale Integration) or ASIC (Application Specific Integrated Circuit).

[0024] The video data acquisition processing unit 11 receives video data output from the camera CM via the input / output I / F unit 4 while the user US is working, and stores the received video data in the video data storage unit 31.

[0025] The image change calculation processing unit 12 reads the most recently acquired video data and the previously acquired video data from the video data storage unit 31, and calculates the amount of image change between the frames of the two acquired video data. This amount of image change is used as information representing the degree of change in the user's (US) line of sight.

[0026] The bottom-up attraction calculation processing unit 13 calculates a bottom-up attraction score, which represents the degree of the user's (US's) bottom-up attention, based on the amount of image change calculated by the image change calculation processing unit 12. Bottom-up attention is the attention function in which the user (US) is attracted to a certain object or area in the image and passively directs their gaze towards it.

[0027] The top-down attention calculation processing unit 14 calculates the top-down attention level, which represents the degree of top-down attention of the user (US), based on the amount of image change calculated by the image change calculation processing unit 12. Top-down attention is the attention function in which the user (US) actively directs their gaze towards a particular object or area in the image.

[0028] The attraction target determination processing unit 15 determines, based on the bottom-up attraction degree calculation processing unit 13 and the top-down attraction degree calculation processing unit 14, whether or not there are objects, such as areas or objects, in the video data that are presumed to have attracted the attention of the user US.

[0029] The concealment target range setting processing unit 16 sets the range of the video data that includes an object, such as a region or object, that is presumed to have attracted the attention of the user US, based on the determination result of the attraction target determination processing unit 15, as the concealment target range.

[0030] The effect selection processing unit 17, when it is determined by the attraction target determination processing unit 15 that there is an object or region that is the target of attraction, selects an applicable concealment effect data from the effect data storage unit 32 for the object or region that is the target of attraction.

[0031] The concealment processing data generation processing unit 18 generates concealment processing data for objects or regions included in the concealment target range set by the concealment target range setting processing unit 16, based on the concealment effect data selected by the effect selection processing unit 17.

[0032] The video synthesis processing unit 19 synthesizes the censorship processing data generated by the censorship processing data generation processing unit 18 with the video data stored in the video data storage unit 31 to generate video data in which the censorship target range in the video data has been censored. The video synthesis processing unit 19 then outputs the generated censorship-processed video data from the input / output I / F unit 4 to the display device DP for display.

[0033] (Example of operation) Next, an example of the operation of the information display device HD configured as described above will be explained.

[0034] Figure 4 is a flowchart showing an example of the processing procedure and processing content of information presentation control with concealment processing executed by the control unit 1 of the information presentation device HD.

[0035] (1) When performing the video data acquisition work, the user US wears an information display device HD consisting of a head-mounted display on their head. In this state, the user inputs a request to start the work into the information display device HD.

[0036] The control unit 1 of the information display device HD monitors the input of the request to start the above work in step S1, and when the request to start the work is input, under the control of the video data acquisition processing unit 11, the video data acquisition process is executed in step S2 as follows.

[0037] In other words, upon commencement of the above operation, the camera CM captures an image of the area corresponding to the user US's field of view and outputs the video data. The video data acquisition processing unit 11 receives the video data output from the camera CM via the input / output interface unit 4 and stores the received video data in the video data storage unit 31 on a frame-by-frame basis. Thus, video data capturing the work environment, including the work terminal TM, is acquired.

[0038] (2) Calculation of Attractiveness When the video data acquisition process described above is started, the control unit 1 of the information display device HD performs the process of calculating the attractiveness in step S3. Attractiveness indicates how easily a person's gaze is directed, and a person's gaze indicates the direction in which that person is visually paying attention. Therefore, attractiveness indicates how easily a person's attention is attracted.

[0039] Figure 5 is a flowchart showing an example of the processing procedure and processing content of the saliency calculation process.

[0040] (2-1) The control unit 1 of the calculation information presentation device HD for the change in the line-of-sight direction of the user US first, in step S30, reads the video data recently acquired from the video data storage unit 31 and the video data acquired before that by the image change calculation processing unit 12, and calculates the amount of image change of each pixel between the frames of each read video data. The amount of image change represents the degree of change in the line-of-sight direction of the user US.

[0041] As a method for calculating the degree of change in the line-of-sight direction, any existing method can be used. For example, each frame in the video is divided into a plurality of grid-like blocks. The size of one block is arbitrarily set according to the image size of the frame, the amount of calculation, etc. Then, between the most recent frame and the frame one second before it, the average value of each RGB value of each block is calculated and compared, and when the ratio of the blocks with different RGB values is a predetermined threshold value, for example, 80% or more of the whole, it is determined that the image has changed.

[0042] In the above calculation method, the entire frame is evenly used as the comparison target, but as another method, for example, only a specific area within the frame, such as the central area within the frame, may be selected and set as the comparison target. By doing so, the amount of calculation of the image change amount by the control unit 1 can be reduced.

[0043] (2-2) The control unit 1 of the bottom-up saliency calculation information presentation device HD calculates the saliency of the user US for the objects, regions, etc. included in the video based on the degree of change in the line-of-sight direction of the user US calculated as described above.

[0044] Generally, it is known that a person's line-of-sight direction is determined based on two types of attention functions: bottom-up attention and top-down attention. Bottom-up attention is an attention function that is passively attracted based on physical characteristics such as color, brightness, and movement among the visual stimuli obtained from the visual field. On the other hand, top-down attention is an attention function that is actively directed based on knowledge, experience, purpose, etc.

[0045] There is a time difference in the occurrence of the above two types of attention functions. When seeing a certain scene for the first time or when the visual field changes significantly, first, the selection of the line of sight based on bottom-up attention is preferentially performed. Then, as the object recognition and situation understanding within the visual field progress, top-down attention becomes dominant and the line of sight is selected. That is, a person's attention function gradually switches from bottom-up attention to top-down attention over time.

[0046] Focusing on the characteristics of a person's attention function as described above, the control unit 1 of the information presentation device HD first determines in step S31 whether or not a change in the line-of-sight direction of the user US has occurred by a predetermined amount or more. Then, based on the determination result, when a change in the line-of-sight direction has occurred by a predetermined amount or more, it is determined that the bottom-up attention function has worked, in which the user US is attracted to a certain object or area in the video and passively turns the line of sight. In this case, the control unit 1 of the information presentation device HD calculates a bottom-up attraction degree representing the degree of bottom-up attention under the control of the bottom-up attraction degree calculation processing unit 13 in step S32.

[0047] The calculation of the bottom-up attraction degree can be realized by applying a method using a saliency map corresponding to bottom-up attention, which is an existing technique.

[0048] For example, assume that the bottom-up attraction degree is calculated using a saliency map for video data capturing a work environment as shown in FIG. 6. In this case, when calculating the bottom-up attraction degree for areas other than the work area including the display, keyboard, and mouse of the work terminal TM, the bottom-up attraction degree is obtained as a normalized value (a value between 0 and 1). Therefore, when expressing the bottom-up attraction degree on the image data, the higher the bottom-up attraction degree, the whiter the pixel is displayed, and conversely, the lower the bottom-up attraction degree, the blacker the pixel is displayed. As a result, for objects that are not directly related to the work but are likely to attract the user US's attention, such as those colored in a conspicuous color or those with movement, the bottom-up attraction degree is displayed as a whiter area than other objects. [[ID=X]] [[ID=X]]

[0049] The method for calculating the degree of visual attention using the saliency map described above is described in detail in the following reference, for example: Reference; L. Itti, C. Koch and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254-1259, Nov. 1998, doi: 10.1109 / 34.730558. (2-3) Calculation of Top-Down Visual Attention On the other hand, if the change in gaze is less than a predetermined amount, the control unit 1 of the information presentation device HD determines that the user US has activated a top-down attention function, actively directing their gaze towards a certain object or area in the image. Then, in step S33, the control unit 1 of the information presentation device HD calculates the top-down visual attention, which represents the degree of top-down attention, under the control of the top-down visual attention calculation processing unit 14.

[0050] Here, changes in gaze due to top-down attention are determined based on various factors. For example, in a work environment like the one illustrated in Figure 6, the relationship with the work content is one factor. Other factors include a person's experience and interests. Thus, since the selection of gaze due to top-down attention is multifaceted, various methods are applied when calculating the top-down attention index, depending on the purpose of the calculation and the required accuracy.

[0051] In this embodiment, among various implementation methods, a method is used in which, for example, an object is recognized from video data, a predetermined attractiveness score is determined for each recognized object based on its relevance to the business, and a top-down attractiveness score is calculated for all pixels constituting the object. In this case, the predetermined attractiveness scores are set in advance in a data table, as shown in Figure 7, and stored in the data storage unit 3.

[0052] On the other hand, for pixels in areas that were not recognized as objects, and for each pixel that constitutes an object that was recognized but not registered in the above data table, the top-down attractiveness score is set to "0".

[0053] Figure 8 shows an example of top-down attractiveness obtained by applying the data table exemplified in Figure 7 to objects recognized from video data of the work environment shown in Figure 6.

[0054] This data table may be used in a fixed manner, or it may be modified as needed according to the progress of the work. Furthermore, it may be shared with all users, or it may be created individually for each user.

[0055] (3) Determination of Attractive Objects When the process of calculating the attractiveness level described above is completed, the control unit 1 of the information display device HD, under the control of the attractive object determination processing unit 15, determines in step S4 whether or not there is an area or object in the video data that is presumed to have attracted the attention of the user US, based on the calculation results of the bottom-up attractiveness level or the top-down attractiveness level.

[0056] Any determination method can be applied. For example, a method can be used in which the calculated bottom-up or top-down attraction score is normalized, and then it is determined whether or not there is a region or object that exceeds a predetermined threshold (e.g., 0.8) for the normalized attraction score.

[0057] If, as a result of the above determination, it is determined that there is no area or object that has attracted the attention of the user US, the control unit 1 of the information presentation device HD reads the corresponding video data from the video data storage unit 31 in step S5 and outputs it directly to the display device DP from the input / output I / F unit 4. In other words, the video data is output to the display device DP and displayed directly without any obfuscation processing.

[0058] (4) When the control unit 1 of the concealment target range setting information display device HD determines in step S4 that there is an area or object that has attracted the attention of the user US, it sets the concealment target range in the video data in step S7 under the control of the concealment target range setting processing unit 16.

[0059] In this case, if there are multiple areas or objects that attract the attention of the user US, the concealment target range setting processing unit 16 may set all of them as the concealment target range, or it may set the one with the highest degree of attention as the concealment target range.

[0060] For example, if the attractiveness of each object or region in the video is calculated as shown in Figure 8, the concealment target range setting processing unit 16 sets the region corresponding to the "schedule" SJ, for which the attractiveness is calculated to be "0.9", as the concealment target range. In this case, a rectangular region including the target object "schedule" SJ and its surrounding region may be set as the concealment target range, or the contour of the target object may be detected and the region corresponding to its shape may be set as the concealment target range. The contour can be detected by detecting the coordinates of each vertex of the target object and connecting the markers of these vertices.

[0061] Furthermore, if the area to be concealed moves within the video screen, for example, if the object to be concealed is a person walking, existing motion tracking technology may be applied to move the area to be concealed in accordance with the movement of the object to be concealed.

[0062] (5) When the attracting target determination processing unit 15 determines that there is an area or object to be attracted, the control unit 1 of the concealing effect selection processing unit 17, under the control of the effect selection processing unit 17, selects from the effect data storage unit 32 the concealing effect data to be applied to the area or object to be attracted in step S7. As a selection method, for example, when multiple applicable concealing effect data are available, a method is used in which candidate concealing effect data is presented to the user US and the user US makes a selection.

[0063] Any censorship effect data can be used as long as it is applicable as Diminished Reality (DR) technology. For example, you can use effects such as "transparency" and "outline enhancement" to change the appearance of an object by overlaying it with a background, "blur" to change the resolution, "desaturation" and "de-saturation" to change the saturation, and "low contrast" to change the brightness.

[0064] Furthermore, as an alternative method for selecting censorship effect data, it is also possible to apply a method that selects, for example, highly transparent or low-resolution censorship effect data for areas or objects with high attractiveness, based on the attractiveness calculated for the area or object to be censored, and conversely, selects low-transparency or high-resolution censorship effect data for areas or objects with low attractiveness.

[0065] (6) The control unit 1 of the concealment processing data generation information display device HD, under the control of the concealment processing data generation processing unit 18, generates concealment processing data for each object or region included in the concealment target range set in step S6, based on the concealment effect data selected in step S7, in step S8.

[0066] (7) The control unit 1 of the information display device HD, which generates and outputs the obscured video data, under the control of the video synthesis processing unit 19, in step S9, combines the obscured data generated in step S8 with the obscuration target range of the video data read from the video data storage unit 31 to generate the obscured video data. The generated video data is then output from the input / output I / F unit 4 to the display device DP for display.

[0067] Furthermore, before displaying the video data that has been obscured as described above, attention-grabbing information may be generated and displayed on the display device DP.

[0068] (8) The control unit 1 of the work completion monitoring information display device HD monitors the completion of the work in step S10. During the period when the work is ongoing, it repeatedly performs the series of video data concealment operations in steps S2 to S9. When, for example, user US inputs a request to complete the work, the video data concealment operation is terminated and the device returns to the standby state.

[0069] (Effects) As described above, in one embodiment, the amount of change in the user's (US) line of sight direction is determined by calculating image changes from video data obtained by imaging the work environment. Based on the amount of change in line of sight direction, it is determined whether the type of attention generated by the user (US) is bottom-up attention or top-down attention, and the degree of attraction to an object or area that attracts the user's (US) attention is calculated for each determined type of attention. Based on the calculated degree of attraction, it is determined whether or not there is an object in the video that attracts the user's (US) attention, and if an object exists, a censorship target range including this object is set. Furthermore, censorship effect data applicable to the object or area that attracted the user's (US) attention is selected, censorship processing data is generated for the censorship target range based on the selected censorship effect data, the generated censorship processing data is combined with the video data to generate censorship-processed video data, and this video data is displayed on the display device DP.

[0070] Therefore, by calculating the bottom-up and top-down attention intensities, the scope of objects and areas that require bottom-up and top-down attention can be automatically and accurately extracted. This eliminates the need for users to manually set the scope of objects to be hidden, thus reducing user effort.

[0071] [Other Embodiments] (1) In one embodiment, the process for calculating the attractiveness level was described as calculating either the bottom-up attractiveness level or the top-down attractiveness level, and then performing a concealment process based on the calculated attractiveness level. However, the present invention is not limited thereto.

[0072] For example, as a first method, one could consider a method that focuses on the fact that there is a time lag between the occurrence of bottom-up attention and top-down attention, calculates the bottom-up attention level in a first period from the time a change in gaze direction is detected, calculates the top-down attention level in a second period after the first period has elapsed, and performs concealment processing based on the calculated attention levels.

[0073] In this way, in the first period immediately following a change in gaze direction, the concealment process is performed based on bottom-up attention indices, and in the subsequent second period, the concealment process is performed based on top-down attention indices. Therefore, it becomes possible to perform accurate concealment processing in accordance with changes in the user's attention function as time progresses over the course of a time series.

[0074] (2) As a second method, one can also consider a method in which the calculated bottom-up and top-down attractiveness scores are normalized, these attractiveness scores are weighted and added together using different weighting coefficients, and then an obscuration process is performed based on the added attractiveness scores.

[0075] For example, in the process of calculating the attractiveness level, when it is determined that the user's (US's) line of sight has changed, the weighting coefficient for the bottom-up attractiveness level is set to "0.9" and the weighting coefficient for the top-down attractiveness level is set to "0.1". Then, over a predetermined period thereafter (for example, 3 seconds), the weighting coefficient for the bottom-up attractiveness level is gradually decreased from "0.9" to "0.1", while the weighting coefficient for the top-down attractiveness level is gradually increased from "0.1" to "0.9".

[0076] Then, the normalized values ​​of the calculated bottom-up attraction score and top-down attraction score are multiplied by the weight coefficients mentioned above, and then added together. This weighted and added value is used as the attraction score for each pixel.

[0077] As described above, by focusing on the nature of bottom-up and top-down attention to be exerted in stages with a time lag over time, and by calculating the attractiveness of each attention function while changing the weight coefficient of each attention function over time, it is possible to accurately calculate an attractiveness that takes both bottom-up and top-down attractiveness into account, and based on the calculated attractiveness, it becomes possible to accurately extract targets for attraction.

[0078] (3) In one embodiment, the processing units 11 to 19 related to the obfuscation process in the information presentation device HD were described as being provided on a head-mounted display. However, each of the above processing units may be provided on a terminal such as a personal computer or smartphone operated by the user, or on a server computer on the Web or in the cloud, and the video data after obfuscation processing generated by these terminals may be presented to the user using, for example, smart glasses or a projector.

[0079] (4) The types of work may include not only so-called desk work using the work terminal TM, but also operation and monitoring of manufacturing equipment such as production lines. Furthermore, this invention is applicable to any work in which the user US performs the work using the information display device HD.

[0080] Although embodiments of this invention have been described in detail above, the above description is merely illustrative in all respects. It goes without saying that various improvements and modifications can be made without departing from the scope of this invention. In other words, when implementing this invention, specific configurations may be adopted as appropriate depending on the embodiment.

[0081] In short, this invention is not limited to the embodiments described above, and in the implementation stage, the components can be modified and materialized without departing from the gist of the invention. Furthermore, various inventions can be formed by appropriately combining the multiple components disclosed in the embodiments. For example, some components may be deleted from all the components shown in the embodiments. Moreover, components from different embodiments may be appropriately combined.

[0082] HD... Information display device CM... Camera DP... Display device 1... Control unit 2... Program storage unit 3... Data storage unit 4... Input / output I / F unit 5... Bus 11... Video data acquisition processing unit 12... Image change calculation processing unit 13... Bottom-up attraction level calculation processing unit 14... Top-down attraction level calculation processing unit 15... Attraction target determination processing unit 16... Concealment target range setting processing unit 17... Effect selection processing unit 18... Concealment processing data generation processing unit 19... Video synthesis processing unit 31... Video data storage unit 32... Effect data storage unit

Claims

1. An information presentation device that presents to a user visual information representing a work environment after performing a masking process on objects among a plurality of objects included in the work environment that have little relation to the work, comprising: a first processing unit that determines whether the type of attention generated by the user is bottom-up attention, in which the user passively directs their gaze to the object, or top-down attention, in which the user actively directs their gaze to the object; a second processing unit that calculates the degree to which the user is attracted to the object for at least one of the bottom-up attention and the top-down attention based on the determination result of the type of attention; a third processing unit that sets a masking target range including the object that attracts the user's gaze to the work environment based on the degree to which the user is attracted; and a fourth processing unit that generates masking processing data for masking the masking target range, and performs masking processing on the visual information representing the work environment based on the generated masking processing data.

2. The information presentation device according to claim 1, further comprising a fifth processing unit that detects the degree of change in the user's gaze direction relative to the work environment based on changes in visual information representing the work environment, wherein the second processing unit calculates a first attractiveness level for bottom-up attention during a first period from the time the change in gaze direction is detected, calculates a second attractiveness level for top-down attention during a second period after the first period has elapsed, and sets the first attractiveness level and the second attractiveness level as the attractiveness level.

3. The information presentation device according to claim 1, wherein the second processing unit sets a first weighting coefficient and a second weighting coefficient that change inversely over time for the bottom-up attention and the top-down attention, respectively, and calculates the attractiveness by adding together the first attractiveness calculated for the bottom-up attention and the second attractiveness calculated for the top-down attention, respectively, after weighting them with the first weighting coefficient and the second weighting coefficient.

4. A program that causes a processor in an information presentation device to execute at least one of the processes performed by the first to fifth processing units in an information presentation device according to any one of claims 1 to 3.

Images