Pop-up display control device, work support device, work support system, pop-up display method, pop-up display program, and recording medium

Abstract

To solve the problem in which: when an additional content is displayed on a display in addition to a primary content, the additional content may be displayed at a position unintended by a user or felt as disturbing.SOLUTION: A popup display controller 10 comprises: a detection unit 11 for detecting a position on a display screen that is a blind spot to a user from gaze position information that indicates the user's gaze position on the display screen on which an image based on primary information is displayed and blind spot setting information that indicates a relationship between the user's gaze position and the position of the blind spot; and an additional information display control unit 12 for causing an image based on additional information to an image based on the primary information to be displayed as a popup at the user's blind spot on the display screen detected by the detection unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a pop-up display control device, a work assistance device, a work assistance system, a pop-up display method, a pop-up display program, and a recording medium that display additional information for main information as a pop-up on a display screen of a display device on which the main information is displayed. [Background technology]

[0002] US Pat. No. 6,399,433 discloses a method and system for displaying additional content on a head-up display that displays a virtual reality environment. The system shown in Patent Document 1 displays on the display primary content, which is movies, television programs, video games, virtual reality worlds, and media guide screens, as well as additional content, which is stock price information, sports scores, news information, weather information, a clock, and event schedules.

[0003] In the system shown in Patent Document 1, when the center of the user's gaze is moved, the generated portion of the additional content is changed and displayed so as not to interfere with the user's view of the primary content that corresponds to the foreground area of ​​the user's field of view determined based on the movement of the user's center of gaze. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2019-515361 Summary of the Invention [Problem to be solved by the invention]

[0005] In the system shown in Patent Document 1, the additional content is generated within a part of the virtual reality environment that corresponds to the peripheral area of ​​the user's field of view through the movement of the user's eyes, so there is a possibility that it will be displayed in a position that the user does not intend, and depending on the size of the additional content to be displayed, it may be perceived as a nuisance.

[0006] The present disclosure has been made in consideration of the above-mentioned points, and aims to provide a pop-up display control device that controls the display of an image based on additional information for an image based on main information on a display screen on which an image based on main information is displayed, without being an eyesore to the user or obstructing the view. [Means for solving the problem]

[0007] The present disclosure relates to a pop-up display control device, and a display screen on which an image based on main information is displayed. Dominant Eye The user's gaze position information indicating the gaze position of the user and the blind spot setting information indicating the relationship between the user's gaze position and the position of the blind spot are used to display the user's gaze position on the display screen. Dominant Eye A detection unit detects the position of the blind spot of the user on the display screen detected by the detection unit. Dominant Eye Additional information based on the main information is added to the blind spot of the image. Brief information of Display an image based on the The display device includes a simplified information display control unit and a detailed information display control unit which displays an image based on detailed information in the additional information as a pop-up instead of an image based on simplified information, and when the gaze position on the display screen based on gaze position information of the user's non-dominant eye from the detection unit moves to the position of the blind spot of the user's dominant eye on the display screen based on position information of the blind spot of the user's dominant eye from the detection unit, the pop-up which is an image based on simplified information by the simplified information display control unit is switched to and displayed as a pop-up which is an image based on detailed information by the detailed information display control unit. and an additional information display control unit. Effect of the Invention

[0008] According to the present disclosure, an image based on additional information for an image based on main information is controlled to be displayed as a pop-up at a location on the display screen that is the user's blind spot, so that the pop-up image based on additional information does not become an eyesore to the user. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a work support system according to a first embodiment. [Diagram 2]1 is a perspective view showing an image of a head-mounted display in which a line-of-sight detection device is incorporated in a display device in the work support system according to embodiment 1. FIG. [Diagram 3] 1 is a front view showing an image of an instruction device in the work support system according to the first embodiment. [Figure 4] 1 is a configuration diagram showing a hardware configuration of a work support system according to a first embodiment. [Diagram 5] FIG. 2 is a diagram showing an example of an input screen for setting a dominant eye for a blind spot setting unit in the work support system according to the first embodiment. [Figure 6] FIG. 2 is a diagram showing an example of an input screen for setting the position of a blind spot for a blind spot setting unit in the work support system according to the first embodiment. [Figure 7] 4 is a flowchart showing an operation example of the work support system according to the first embodiment. [Figure 8] FIG. 13 is a diagram showing an example in which an image based on main information is displayed on a display screen of a display device in the work support system according to embodiment 1, and an image based on simplified information among additional information is displayed as a pop-up on the display screen. [Figure 9] FIG. 13 is a diagram showing an example in which an image based on main information is displayed on a display screen of a display device in the work support system according to embodiment 1, and an image based on detailed information in additional information is displayed as a pop-up on the display screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Embodiment 1 The work support system according to the first embodiment will be described with reference to FIGS. The work support system according to embodiment 1 is an MR (Mixed Reality) system that displays work content such as assembly or processing on a display device that is an MR display, and performs work in a virtual reality environment based on the displayed work content.

[0011] The work support system according to the first embodiment includes a work support device 1, a display device 2, a line-of-sight detection device 3, and an instruction device 4. The work assistance device 1 is a device that displays an image based on main information (hereinafter referred to as main information image) as a pop-up on the display screen of a display device 2, on which an image based on main information (hereinafter referred to as main information image) is displayed, and that assists in tasks such as assembly or processing in a virtual reality environment based on the main information image.

[0012] The main information image is work content for a manufacturing device or the like on which a user, who is an operator, performs work such as assembly or processing. The additional information image is work procedure content that shows the user the work content or work procedure (hereinafter collectively referred to as work procedure) of the work when the user performs work in a virtual reality environment based on the main information image displayed on the display screen of the display device 2, and is displayed as a pop-up on the main information image.

[0013] The work support device 1 includes a pop-up display control device 10, a blind spot setting unit 20, and a content storage unit 30. The pop-up display control device 10 includes a gaze position and blind spot position detection unit 11 (hereinafter, abbreviated as detection unit) and an additional information display control unit 12. The additional information display control unit 12 has a display information selection / switching unit 121 (hereinafter, abbreviated as the selection / switching unit), a simplified information display control unit 122, and a detailed information display control unit 123. The work support device 1 will be described in detail later.

[0014] The display device 2 has a display screen on which a main information image and an additional information image are displayed. The display device 2 is a head-mounted display that is an MR display, or a display device having a screen display function such as a transmissive liquid crystal screen or an organic EL screen.

[0015] In the first embodiment, a head mounted display as shown in FIG. The display device 2 consisting of a head-mounted display shown in Figure 2 has a main body 2a, a display screen 2b which is a display provided on the front of the main body 2a, a strap 2c which is attached to the main body 2a and is used for wearing the main body 2a on the user's head, and an image display device (not shown) which is provided inside the main body 2a and which displays images on the display screen 2b.

[0016] The line-of-sight detection device 3 is provided inside the main body 2a of the display device 2. The gaze detection device 3 detects the gaze position, which is the viewpoint from which the user is looking on the display screen 2b of the display device 2, and outputs gaze position information indicating the detected gaze position of the user on the display screen 2b to the detection unit 11 of the pop-up display control device 10 in the work support device 1. Hereinafter, except in special cases, the gaze position will be described as being synonymous with the viewpoint.

[0017] The line-of-sight detection device 3 detects the position of the user's pupils, thereby detecting the position at which the user is looking at the image displayed on the display screen 2b of the display device 2, that is, the line-of-sight position. The gaze position is detected by determining the gaze position from the positional relationship between the user's pupil and corneal reflex, or the positional relationship between the inner corner of the user's eye and the iris. The gaze detection device 3 constantly detects the user's gaze position while the user is using the display device 2, and outputs gaze position information based on the gaze position to the detection unit 11 of the pop-up display control device 10 in the work support device 1.

[0018] The instruction device 4 outputs blind spot setting information for setting the blind spot of the user to the blind spot setting unit 20 in the work support device 1. The instruction device 4 is a VR (Virtual Reality) device controller having built-in operation buttons, a keyboard, a mouse, or a pointer device.

[0019] In the first embodiment, a controller of a VR device as shown in FIG. The instruction device 4 consisting of a controller for the VR device shown in FIG.

[0020] The instruction device 4 allows a user to set his / her dominant eye, and outputs the set dominant eye information to the blind spot setting unit 20 in the work assistance device 1. The pointing device 4 allows the user to move a pointer displayed on the display screen 2b on the display screen 2b by operating the operation button 41 while looking at the center of the field of view on the display screen 2b of the display device 2 with the dominant eye, and sets the position where the pointer disappears from the field of view of the user's dominant eye as the position of the blind spot of the user's dominant eye.

[0021] The pointing device 4 outputs the center position of the visual field as gaze position information and the position where the pointer has disappeared from the visual field as blind spot position information to the blind spot setting unit 20. Similarly, for the non-dominant eye, the center position of the visual field may be obtained as gaze position information, and the position where the pointer disappears from the visual field may be obtained as blind spot position information.

[0022] The instruction device 4 outputs to the selection / switching unit 121 of the additional information display control unit 12 in the pop-up display control device 10 a selection signal for selecting the simplified information display control unit 122 of the additional information display control unit 12 in the initial setting, and a switching signal for switching from the simplified information display control unit 122 to the detailed information display control unit 123 of the additional information display control unit 12, and for switching from the detailed information display control unit 123 to the simplified information display control unit 122.

[0023] The blind spot setting unit 20 in the work assistance device 1 is a part that sets the position of the blind spot relative to the gaze position of the dominant eye of the user and stores blind spot information including blind spot setting information for the dominant eye, and has a setting unit 21 and a storage unit 22. The setting unit 21 receives dominant eye setting information, gaze position information for the dominant eye, and blind spot position information from the instruction device 4, links the user with the dominant eye setting information and the blind spot setting information for the dominant eye to generate blind spot information, and stores the generated blind spot information in the memory unit 22.

[0024] The blind spot setting information indicates the relationship between the position of the blind spot and the position of the line of sight at that time by using the line of sight position information and the blind spot position information. That is, the blind spot setting information is information indicating a calculated value of the length between the center position of the visual field that is set as the viewpoint and the position where the pointer disappears from the visual field. The blind spot setting information indicates the relationship between the gaze position of the user's dominant eye and the blind spot. If the gaze position of the user's dominant eye is known, the position of the user's blind spot relative to the gaze position can be uniquely determined from the blind spot setting information.

[0025] When there are multiple users, the setting unit 21 associates the dominant eye setting information and the dominant eye blind spot setting information for each user and stores the blind spot information in the storage unit 22. Similarly, for the non-dominant eye, the setting unit 21 may store blind spot information in the memory unit 22 together with information for the dominant eye by linking the non-dominant eye setting information and the blind spot setting information for each user.

[0026] The content storage section 30 in the work support device 1 stores two types of additional information, namely, simplified information 31 and detailed information 32, for displaying work procedure content (additional information image) as a pop-up on the work content (main information image) displayed on the display screen 2b of the display device 2.

[0027] The simplified information 31 is image information that simply indicates a work procedure, for example, a single word such as an item name or a symbolic representation of one work procedure using an illustration. The simplified information 31 may be, for example, image information showing one work procedure for each step of the work content. The detailed information 32 corresponds one-to-one to the simple information 31, and is image information showing the work procedure in detail.

[0028] The detection unit 11 of the pop-up display control device 10 in the work support device 1 receives gaze position information based on the gaze position of the user's dominant eye and gaze position information based on the gaze position of the non-dominant eye on the display screen 2b of the display device 2 from the gaze detection device 3, and outputs the gaze position information of the dominant eye and the gaze position information of the non-dominant eye.

[0029] The detection unit 11 reads out the position information of the blind spot of the dominant eye of the user that has been set from the memory unit 22 of the blind spot setting unit 20, detects the position of the current blind spot of the dominant eye of the user on the display screen 2b from the gaze position information of the dominant eye from the gaze detection device 3 and the blind spot position information read from the memory unit 22, and outputs the position information of the current blind spot. The detection unit 11 follows the gaze position information from the gaze detection device 3 and outputs the position information of the current blind spot of the user.

[0030] The additional information display control unit 12 in the pop-up display control device 10 displays an additional information image for the main information image displayed on the display screen 2b of the display device 2 as a pop-up at the position of the user's blind spot on the display screen 2b of the display device 2 detected by the detection unit 11.

[0031] The additional information display control unit 12 has a simplified information display control unit 122 that displays an image based on the simplified information 31 as a pop-up, a detailed information display control unit 123 that displays an image based on the detailed information 32 as a pop-up instead of an image based on the simplified information 31, and a selection / switching unit 121 that selects and switches between the simplified information display control unit 122 and the detailed information display control unit 123.

[0032] First, the selection / switching unit 121 selects the simplified information display control unit 122 in response to a selection signal from the instruction device 4 in the initial setting. Secondly, the selection / switching unit 121 instructs the simplified information display control unit 122 and the detailed information display control unit 123 to switch from the simplified information display control unit 122 to the detailed information display control unit 123 or to switch from the detailed information display control unit 123 to the simplified information display control unit 122 based on a switching signal from the instruction device 4.

[0033] Thirdly, when the gaze position on the display screen 2b of the display device 2 based on the gaze position information of the non-dominant eye from the detection unit 11 moves to the position of the blind spot of the dominant eye on the display screen 2b of the display device 2 based on the position information of the blind spot of the dominant eye from the detection unit 11, the selection / switching unit 121 instructs the simplified information display control unit 122 and the detailed information display control unit 123 to switch from the simplified information display control unit 122 to the detailed information display control unit 123.

[0034] The selection / switching unit 121 determines whether the gaze position of the non-dominant eye is located at the position of the blind spot of the dominant eye on the display screen 2b by comparing the gaze position information of the non-dominant eye with delayed position information of the blind spot of the dominant eye, since the detection unit 11 outputs position information of the blind spot of the dominant eye by tracking the gaze position of the dominant eye.

[0035] Fourthly, when the gaze position on the display screen 2b of the display device 2 based on the gaze position information of the non-dominant eye from the detection unit 11 deviates from the range of the image displayed on the display screen 2b of the display device 2 based on the detailed information 32 from the detailed information display control unit 123, the selection / switching unit 121 instructs the simplified information display control unit 122 and the detailed information display control unit 123 to switch from the detailed information display control unit 123 to the simplified information display control unit 122.

[0036] When selected by the selection / switching unit 121, the simplified information display control unit 122 reads out from the content storage unit 30 simplified information 31 corresponding to the work procedure based on the work procedure signal 5, and outputs simplified information to the display device 2 for displaying an image based on the read out simplified information 31 as a pop-up at the position of the dominant eye's blind spot based on the position information of the dominant eye's blind spot from the detection unit 11 in the main information image displayed on the display screen 2b of the display device 2. The size of the pop-up as an image based on the simplified information 31 output from the simplified information display control unit 122 to the display device 2 is within the range of the size of the blind spot of an average user.

[0037] The simplified information display control unit 122 outputs to the display device 2 simplified information, based on the current blind spot position information in the user's dominant eye from the detection unit 11 which tracks the gaze position information from the gaze detection device 3, such that the position of a pop-up on the main information image displayed on the display screen 2b of the display device 2 also tracks the blind spot position information from the detection unit 11.

[0038] The simplified information display control unit 122 reads out simplified information 31 corresponding to the work procedure from the content storage unit 30 each time the content of the work procedure signal 5 changes, and outputs simplified information to the display device 2 for displaying an image based on the read out simplified information 31 as a pop-up at the position of the blind spot of the dominant eye based on the blind spot position information from the detection unit 11.

[0039] When selected by the selection / switching unit 121, the detailed information display control unit 123 reads out from the content storage unit 30 detailed information 32 corresponding to the simplified information 31 from the simplified information display control unit 122 for the image displayed on the display screen 2b of the display device 2, and outputs the detailed information to the display device 2 to display an image based on the read out detailed information 32 as a pop-up at the position on the main information image of the display screen 2b of the display device 2 where the image based on the simplified information 31 is displayed.

[0040] As shown in FIG. 4, the work support device 1 includes an input interface unit, an output interface unit, a CPU (also called a Central Processing Unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a processor, or a DSP (Digital Signal Processor)), a RAM (Random Access Memory) which is a semiconductor memory, and a ROM (Read Only Memory) which is a non-volatile semiconductor memory which is a storage medium.

[0041] When a head-mounted display is used as the display device 2, the work support device 1 is composed of a printed circuit board on which an input interface unit, an output interface unit, a CPU, a RAM, and a ROM are mounted, and is stored inside the main body 2a of the display device 2. When a display device having a screen display function is used as the display device 2, a DVD (Digital Versatile Disc), a CD (Compact Disc), an HDD (Hard Disk Drive), or a USB memory can be used as a storage medium instead of a ROM.

[0042] The input interface unit receives gaze position information from the gaze detection device 3, blind spot setting information from the instruction device 4, gaze position information indicating the center position of the visual field, information dominant eye setting information, a selection signal, and a switching signal. From the output interface unit, the simplified information from the simplified information display control unit 122 and the detailed information from the detailed information display control unit 123 are output to the display device 2. The memory unit 22 of the blind spot setting unit 20 is configured with a RAM or a ROM, and the content storage unit 30 is configured with a ROM.

[0043] The CPU controls and manages the input interface section, RAM, ROM, and output interface section. The work support device 1 loads a program stored in a ROM into a RAM, and a CPU executes various processes based on the program loaded into the RAM. The work support device 1 is driven by a general-purpose OS (Operating System).

[0044] The program stored in a ROM as a storage medium and executed by a CPU includes the steps of: detecting the position of the blind spot of the user's dominant eye on the display screen from the gaze position of the user's dominant eye on the display screen where the main information image is displayed and the relationship between the gaze position of the user's dominant eye and the position of the blind spot; displaying an image based on simplified information from the additional information for the main information image as a pop-up at the position of the blind spot of the user's dominant eye on the display screen; instructing to switch the image based on the additional information when it is detected that the gaze position of the user's non-dominant eye on the display screen has been moved to the position of the blind spot of the user's dominant eye on the display screen; and, when the instruction to switch the image based on the additional information is received, displaying an image based on detailed information from the additional information for the image based on the main information as a pop-up at the position of the blind spot of the user's dominant eye on the display screen instead of the pop-up image based on simplified information.

[0045] Next, the operation of the work support system according to the first embodiment will be described. First, as a preliminary preparation, blind spot information is stored in the memory unit 22 of the blind spot setting unit 20, in which the user is associated with dominant eye setting information and the blind spot setting information of the dominant eye for each user. The dominant eye setting information is set as the dominant eye setting information linked to the user by the setting unit 21 of the blind spot setting unit 20 when a dominant eye setting screen as shown in FIG. 5 is displayed on the display screen 2b of the display device 2 by operating the operation button 41 of the indicating device 4 and the dominant eye is selected by operating the operation button 41.

[0046] The blind spot setting information for the dominant eye is obtained by operating the operation button 41 of the pointing device 4 to display a blind spot setting screen on the display screen 2b of the display device 2, which displays the visual field range FoV, center M of the visual field, and pointer P, as shown in the upper part of Figure 6. While the user is looking at center M of the visual field with his or her dominant eye, the operation button 41 is operated to move the pointer P on the display screen 2b. When the pointer P has disappeared, i.e., is no longer visible, with the dominant eye, the position information of the pointer P is obtained by the setting unit 21 of the blind spot setting unit 20 through the operation of the operation button 41.

[0047] The setting unit 21 of the blind spot setting unit 20 calculates the distance based on the position coordinates of the center M of the visual field and the position information of the pointer P where the pointer P has disappeared, and sets the blind spot setting information of the dominant eye indicating the calculated value by linking it with the user. In Fig. 6, the blind spot is indicated by BS. The blind spot setting information is information that represents on the coordinate axes the length from the position of the center M of the field of view to the position of pointer P where pointer P has disappeared on the X axis, and the length from the position of the center M of the field of view to the position of pointer P where pointer P has disappeared on the Y axis.

[0048] In this manner, the setting unit 21 of the blind spot setting unit 20 links the set dominant eye setting information and the set dominant eye blind spot setting information with the user to generate dominant eye blind spot information, and stores the generated blind spot information in the memory unit 22 of the blind spot setting unit 20. In addition, for the non-dominant eye, the setting unit 21 of the blind spot setting unit 20 may generate blind spot information for the non-dominant eye in the same manner as for the dominant eye, and store the generated blind spot information in the memory unit 22 of the blind spot setting unit 20.

[0049] Next, a pop-up display method by the work support device 1 in the work support system will be described with reference to the flowchart shown in FIG. The head mounted display serving as the display device 2 is worn, the power is turned on, and the main information image is displayed on the display screen 2b of the display device 2.

[0050] Then, in step ST1, the detection unit 11 acquires gaze position information indicating the gaze position of the user's dominant eye from the gaze detection device 3, reads out the blind spot information stored in the memory unit 22 of the blind spot setting unit 20, detects the position of the blind spot of the user's dominant eye on the display screen 2b of the display device 2 from the gaze position information and the blind spot setting information of the blind spot information, and proceeds to step ST2.

[0051] In step ST2, the selection / switching unit 121 of the additional information display control unit 12 selects the simplified information display control unit 122, and the simplified information display control unit 122 reads out simplified information 31 from the additional information for the main information image from the content storage unit 30 in accordance with the work procedure signal 5, and displays an image based on the read out simplified information 31 on the display screen 2b as a pop-up P1 at the position of the blind spot of the user's dominant eye on the display screen 2b detected by the detection unit 11.

[0052] This state is shown in FIG. 8 as a display image of a pop-up P1. The main information image MV shown in FIG. 8 is shown for reference as an example of work content, and the display contents of the main information image MV are not distinctive.

[0053] Figure 8 shows a display device 2 having a right-eye screen 2R and a left-eye screen 2L, and shows the right eye's field of view RFoV and the left eye's field of view LFoV, a main information image MV and a pop-up P1 displayed by the right-eye screen 2R and the left-eye screen 2L, and the user's right eye gaze position (viewpoint) RPoV and blind spot position (range) RBS, and left eye gaze position (viewpoint) LPoV and blind spot position (range) LBS located on the main information image MV.

[0054] For ease of explanation, the gaze position (viewpoint) RPoV of the user's right eye and the position (range) of the blind spot RBS and the gaze position (viewpoint) LPoV of the left eye and the position (range) of the blind spot LBS are shown in Figure 8, but are not displayed on the main information image MV displayed on the display screen 2b.

[0055] The user is right-eye dominant, and in Fig. 8, the pop-up P1 is displayed at the position RBS of the right eye's blind spot. The pop-up P1 is displayed within the range (position) RBS of the right eye's blind spot. Therefore, since the pop-up P1 is displayed within the range (position) RBS of the blind spot of the dominant eye, the user cannot see or recognize the pop-up P1 with the dominant eye. Although the pop-up P1 is visible with the non-dominant left eye, since the left eye is not the dominant eye, the pop-up P1 does not interfere with the user's field of vision and is not an eyesore.

[0056] In step ST2, the selection / switching unit 121 first automatically selects the simplified information display control unit 122, which then operates to display the pop-up P1. However, the selection / switching unit 121 may operate the operation button 41 of the instruction device 4 to output a selection signal for selecting the simplified information display control unit 122 to the selection / switching unit 121 of the additional information display control unit 12, thereby displaying the pop-up P1 in step ST2.

[0057] In step ST3, the detection unit 11 acquires gaze position information output from the gaze detection device 3 in association with tracking the gaze position RPoV of the user's dominant eye, and the detection unit 11 and the simplified information display control unit 122 operate in the same manner as in steps ST1 and ST2, and display a pop-up P1 at the position RBS of the blind spot of the user's dominant eye in accordance with the movement of the user's gaze. That is, the pop-up P1 is always displayed following the position RBS of the blind spot of the user's dominant eye, and the pop-up P1 never interferes with the user's field of vision and is not an eyesore.

[0058] In step ST4, the detection unit 11 acquires gaze position information indicating the gaze position LPoV of the user's non-dominant eye from the gaze detection device 3, and determines whether the gaze position LPoV based on the acquired gaze position information is at the position RBS of the blind spot of the user's dominant eye on the display screen 2b obtained in step ST3.

[0059] If the gaze position LPoV of the non-dominant eye has not been moved to the blind spot position RBS of the user's dominant eye, the process returns to step ST3. When it is detected that the gaze position LPoV of the non-dominant eye has been moved to the blind spot position RBS of the user's dominant eye, the detection unit 11 instructs the selection / switching unit 121 of the additional information display control unit 12 to switch the image based on the additional information, and the process proceeds to step ST5.

[0060] In step ST5, the selection / switching unit 121 instructs the simplified information display control unit 122 and the detailed information display control unit 123 to switch from the simplified information display control unit 122 to the detailed information display control unit 123 of the additional information display control unit 12. The detailed information display control unit 123, which has received the instruction to switch, displays an image based on detailed information from the additional information for the main information image as a pop-up P2 at the position RBS of the blind spot of the user's dominant eye on the display screen 2b detected by the detection unit 11, instead of the pop-up P1, which is an image based on simplified information. Pop-up P2 is a detailed view corresponding to pop-up P1.

[0061] This state is shown in FIG. 9 as a display image of a pop-up P2. In FIG. 9, the same reference numerals as in FIG. 8 indicate the same or corresponding parts. As is clear from FIG. 9, the pop-up P2 is displayed at the position where the pop-up P1 was displayed, and the pop-up P2 is displayed in place of the pop-up P1 at the position to which the user's gaze has moved. Therefore, when a user wants to know detailed work procedures, the user can quickly and timely view the detailed work procedures without moving his or her eyes significantly, with minimal movement of the eyes.

[0062] In addition to switching the display from pop-up P1 to pop-up P2 on the display screen by detecting the gaze position LPoV of the non-dominant eye, the display may also be switched by operating the operation button 41 of the pointing device 4, which outputs a switching signal from the simplified information display control unit 122 to the detailed information display control unit 123 to the selection / switching unit 121 of the additional information display control unit 12.

[0063] In step ST6, the detection unit determines whether the gaze position LPoV based on the gaze position information of the user's non-dominant eye from the gaze detection device 3 is within the position (range) of the pop-up P2 displayed by the detailed information display control unit 123 in step ST5.

[0064] When the gaze position LPoV of the non-dominant eye is within the position (range) of the pop-up P2, the pop-up P2 continues to be displayed. If the gaze position LPoV of the non-dominant eye falls outside the position (range) of the pop-up P2, the process proceeds to step ST7.

[0065] In step ST7, the simplified information display control section 122 determines whether or not it is necessary to display the pop-up P2, depending on the state of the main information image MV displayed on the display screen 2b. For example, if the work procedure signal 5 input to the simplified information display control unit 122 is the last, it is determined that there is no need to display the pop-up P2, and the process ends. If the work procedure signal 5 is not the last one, the process returns to step ST2, where the detection unit 11 and the simplified information display control unit 122 display the pop-up P1.

[0066] The operation button 41 of the instruction device 4 may be operated to return to the display of the pop-up P1 or to end the display of the pop-ups P1 and P2.

[0067] In the work support system according to the first embodiment, the case where a head-mounted display is used as the display device 2 has been mainly described, but the basic concept is the same even if the display device 2 is a display device having a screen display function.

[0068] As described above, the pop-up display control device 10 of the work support device 1 in the work support system according to embodiment 1 controls the display of an image based on additional information for an image based on primary information as a pop-up at a position of the user's blind spot on the display screen. Therefore, even if a pop-up is displayed on an image based on primary information displayed on the display screen, the pop-up does not become an eyesore for the user. Furthermore, by controlling the display of the popup to be at the blind spot of the user's dominant eye, the popup will not be an eyesore to the user.

[0069] The popup display control device 10 has two types of additional information to be displayed as a popup, namely, simplified information and detailed information, and controls the size of the popup as an image based on the displayed simplified information to fall within the range of the size of the user's blind spot, so that the popup based on the simplified information does not obstruct the view, and the user can know the detailed contents by the popup based on the detailed information when necessary.

[0070] The pop-up display control device 10 controls so that, when the gaze position of the user's non-dominant eye moves to the position of the blind spot of the user's dominant eye, a pop-up which is an image based on simplified information is switched to a pop-up based on detailed information, so that when the user wants to know the details, the details can be visually confirmed promptly and in a timely manner, without moving the gaze significantly, and with a minimum movement of the gaze.

[0071] The work support device 1 in the work support system according to the first embodiment has a blind spot setting unit for setting the position of the user's blind spot. Therefore, the setting unit 21 sets blind spot information for each user based on the position of the blind spot, which varies from person to person, and stores the information in the memory unit 22. The detection unit 11 uses the blind spot information stored in the memory unit to detect the position of the blind spot on the display screen of the display device 2 of the user. This makes it possible to reliably display the pop-up P1 at the position of the user's blind spot. It is simple and inexpensive.

[0072] It should be noted that the components in the first embodiment may be freely combined, any of the components in the embodiments may be modified, or any of the components in the embodiments may be omitted. [Industrial Applicability]

[0073] INDUSTRIAL APPLICABILITY A pop-up display control device according to the present disclosure is suitable for use in a task support system that uses an MR system to display task content on an MR display and perform tasks in a virtual reality environment. In particular, it is suitable for use in an MR display that uses a head-mounted display. [Explanation of symbols]

[0074] 1 work support device, 10 pop-up display control device, 11 detection unit, 12 additional information display control unit, 121 selection / switching unit, 122 simplified information display control unit, 123 detailed information display control unit, 20 blind spot setting unit, 30 content storage unit, 2 display device, 3 line of sight detection device, 4 instruction device.

Claims

1. A detection unit that detects the position of the blind spot of the dominant eye of the user on a display screen on which an image based on primary information is displayed, based on gaze position information indicating the gaze position of the dominant eye of the user on the display screen and blind spot setting information indicating the relationship between the gaze position of the user and the position of the blind spot; a simplified information display control unit that displays an image based on simplified information among additional information for an image based on the main information as a pop-up at the position of the blind spot of the dominant eye of the user on the display screen detected by the detection unit; and a detailed information display control unit that displays an image based on detailed information among the additional information as a pop-up instead of an image based on the simplified information, and when a gaze position on the display screen based on gaze position information of the user's non-dominant eye from the detection unit moves to the position of the blind spot of the dominant eye of the user on the display screen based on position information of the blind spot of the dominant eye of the user from the detection unit, an additional information display control unit that switches and displays the pop-up that is the image based on the simplified information by the simplified information display control unit to a pop-up that is the image based on detailed information by the detailed information display control unit. A pop-up display control device comprising:

2. 2. The popup display control device according to claim 1, wherein the simplified information display control unit controls the size of the popup as an image based on the simplified information to be displayed so as to fall within a range of the size of the user's blind spot. Display control device.

3. A pop-up display control device according to claim 1 or 2, a blind spot setting unit, and a content storage unit, the blind spot setting unit includes a setting unit that receives gaze position information indicating the gaze position of the user and position information indicating the position of the blind spot of the user, and generates blind spot setting information indicating a relationship between the gaze position of the user and the position of the blind spot, and a storage unit that stores the blind spot setting information of the user generated by the setting unit and from which the blind spot setting information of the user is read by a detection unit of the pop-up display control device; the content storage unit stores additional information indicating work support for the main information, and the stored additional information is read out by an additional information display control unit of the pop-up display control device. Work support equipment.

4. A work support device comprising the work support device according to claim 3, a display device, a gaze detection device, and an instruction device, the display device has a display screen on which an image based on the main information and an image based on the additional information are displayed, the line-of-sight detection device detects a line of sight of a user on the display screen of the display device, and outputs line-of-sight position information indicating the detected line of sight of the user on the display screen to a detection unit of a pop-up display control device in the work assistance device; The instruction device outputs gaze position information indicating a gaze position of the user and position information indicating a position of the blind spot of the user to a blind spot setting unit in the work assistance device. Work support system.

5. a detection unit detecting a position of the blind spot of the dominant eye of the user on a display screen on which an image based on main information is displayed, based on gaze position information indicating a gaze position of the dominant eye of the user on the display screen and blind spot setting information indicating a relationship between the gaze position of the user and a position of the blind spot; a step in which a simplified information display control unit of an additional information display control unit displays, as a pop-up, an image based on simplified information among the additional information for the image based on the main information at the position of the blind spot of the dominant eye of the user on the display screen detected by the detection unit; a step of instructing image switching based on the additional information when the detection unit detects that a gaze position of the non-dominant eye of the user on the display screen has been moved to a position of a blind spot of the dominant eye of the user on the display screen; a step of displaying, when a detailed information display control unit of the additional information display control unit receives an instruction to switch the image based on the additional information, an image based on the detailed information among the additional information for the image based on the main information as a pop-up at the position of the blind spot of the dominant eye of the user on the display screen detected by the detection unit, instead of a pop-up which is an image based on the simplified information; A popup display method comprising:

6. a step of detecting a position of the blind spot of the dominant eye of the user on a display screen on which main information is displayed, based on a gaze position of the dominant eye of the user on the display screen and blind spot setting information indicating a relationship between the gaze position of the dominant eye of the user and a position of the blind spot; a step of displaying an image based on simplified information among additional information for an image based on the main information as a pop-up at a position of the blind spot of the dominant eye of the user on the display screen; a step of instructing image switching based on the additional information when it is detected that a gaze position of the non-dominant eye of the user on the display screen has been moved to a position of the blind spot of the dominant eye of the user on the display screen; a step of displaying, when receiving an instruction to switch the image based on the additional information, an image based on detailed information among the additional information for the image based on the main information as a pop-up at a position of the blind spot of the dominant eye of the user on the display screen, instead of the pop-up which is the image based on the simplified information; A pop-up display program that you allow your computer to run.

7. a step of detecting a position of the blind spot of the dominant eye of the user on a display screen on which main information is displayed, based on a gaze position of the dominant eye of the user on the display screen and blind spot setting information indicating a relationship between the gaze position of the dominant eye of the user and a position of the blind spot; a step of displaying an image based on simplified information among additional information for the main information as a pop-up at a position of the blind spot of the dominant eye of the user on the display screen; a step of instructing image switching based on the additional information when it is detected that a gaze position of the non-dominant eye of the user on the display screen has been moved to a position of the blind spot of the dominant eye of the user on the display screen; a step of displaying, when receiving an instruction to switch the image based on the additional information, an image based on detailed information among the additional information for the image based on the main information as a pop-up at a position of the blind spot of the dominant eye of the user on the display screen, instead of the pop-up which is the image based on the simplified information; A recording medium that stores a program to be executed by a computer.

Images