Head-mounted display, control method, and program

Abstract

To provide a technique which enables a user to quickly avoid visible recognition of a reality object which is a mental obstacle.SOLUTION: A display device includes: display means for visually recognizing a reality space through a display surface; acquisition means for acquiring user information on the state of a user; and control means for controlling display of the display means so as to reduce saliency of a first object that is a reality object to which the user directs its line of sight or a reality object other than the first object, on the basis of the user information.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention is a method for visually recognizing real space through a display screen. head-mounted display Regarding. [Background technology]

[0002] XR (Cross Reality) is known as a technology that merges real space and virtual space. Hereinafter, objects that exist in real space will be referred to as real objects, and objects that do not exist in real space, such as CG models, will be referred to as virtual objects. In XR, various image processing such as superimposition of virtual objects is performed on real space or real objects. For example, Patent Document 1 discloses replacing physical obstacles (real objects that a user may come into physical contact with) with virtual objects. Patent Document 2 discloses using a technology called DR (Diminished Reality) to erase unnecessary real objects specified by a user from an image in real space (replacing the unnecessary real objects with similar real objects). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 5580855 [Patent Document 2] Japanese Patent Application Laid-Open No. 2017-212693 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the technology disclosed in Patent Document 1, although a user can avoid contact with a physical obstacle, when a mental obstacle (for example, a real object that evokes negative feelings in the user) comes into view, the user cannot avoid seeing the obstacle. Also, with the technology disclosed in Patent Document 2, when a mental obstacle comes into view, the user must designate the obstacle as an unwanted object, and cannot quickly avoid seeing the obstacle.

[0005] The present invention aims to provide a technique that enables a user to quickly avoid seeing a real object that is a mental obstacle. [Means for solving the problem]

[0006] A first aspect of the present invention is a display device having an imaging means for imaging a real space, a display means for displaying an image of the real space imaged by the imaging means so as to be visible to a user, an acquisition means for acquiring user information relating to a state of the user, and a control means for controlling, based on the user information, the display of the display means so as to reduce the salience of a first object which is a real object at which the user is looking, or a real object other than the first object. and when the user directs his / her gaze to a second object that is a real object of the same type as the first object after performing a first control for the first object based on the user information, the control means performs a second control for the second object without using the user information, and when the first control is a control to reduce the salience of the first object, the second control is a control to reduce the salience of the second object, and when the first control is a control to reduce the salience of a real object other than the first object, the second control is a control to reduce the salience of a real object other than the second object. The head-mounted display is characterized by:

[0008] A second aspect of the present invention is a control method for a head-mounted display having an imaging means for imaging a real space, and displaying an image of the real space imaged by the imaging means so that the image can be visually recognized by a user, the control method including: acquiring user information relating to a state of the user; and determining a real object at which the user is looking based on the user information. The first object is ,or, The first and a control step of controlling the display of the head-mounted display so as to reduce the salience of real objects other than the object. In the control step, after a first control for the first object is performed based on the user information, if the user directs his / her gaze to a second object that is a real object of the same type as the first object, a second control for the second object is performed without being based on the user information, and if the first control is a control to reduce the salience of the first object, the second control is a control to reduce the salience of the second object, and if the first control is a control to reduce the salience of a real object other than the first object, the second control is a control to reduce the salience of a real object other than the second object. The control method is characterized by the above.

[0009] The present invention3 The third aspect is a program for causing a computer to execute each step of the above-described control method. [Effects of the Invention]

[0010] According to the present invention, the user can quickly avoid seeing a real object that is a mental obstacle. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram illustrating an example of a hardware configuration of a display device according to a first embodiment. [Figure 2] FIG. 1 is a block diagram illustrating an example of a functional configuration of a display device according to a first embodiment. [Figure 3] 1 is a table showing an example of a correspondence relationship between emotion levels and image processing according to the first embodiment. [Figure 4] 10 is a flowchart illustrating an example of an operation of the display device according to the first embodiment. [Figure 5] 10 is a flowchart illustrating an example of an operation of the display device according to the second embodiment. [Figure 6] 10 is a table showing an example of a correspondence relationship between attention levels and image processing according to the third embodiment. [Figure 7] 10 is a flowchart showing an example of the operation of the display device according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Example 1 A first embodiment of the present invention will be described below. FIG. 1 is a block diagram showing an example of a hardware configuration of a display device 100 according to the first embodiment. The display device 100 is an XR (Cross The display device 100 is a display device that can be used for a variety of applications, including virtual reality (VR), such as a head-mounted display, smart glasses, a smartphone, a tablet terminal, etc. The display device 100 includes a CPU 101, a ROM 102, a RAM 103, an imaging unit 104, a display unit 105, a sensing unit 106, an operation unit 107, a communication unit 108, and a storage unit 109, which are connected to one another via a bus 110.

[0013] The CPU 101 is an arithmetic processing unit that performs overall control of the display device 100, and executes various programs recorded (stored) in the ROM 102 or the like to perform various processes. The ROM 102 is a read-only non-volatile memory that stores various programs and parameters that do not require modification, such as image processing programs and initial data. The RAM 103 is used as a working area for the CPU 101, and temporarily stores input information for the display device 100, calculation results in image processing, and the like.

[0014] The imaging unit 104 is a camera that captures images of the real space. The display unit 105 is a liquid crystal display or the like, and displays images captured of the real space and various graphics on the display surface. The graphics are, for example, virtual objects (objects that do not exist in the real space), characters, icons, etc. The user can view the real space through the display surface of the display unit 105. In the first embodiment, viewing an image captured of the real space and displayed on the display surface is viewing the real space through the display surface.

[0015] The sensing unit 106 is a sensor for detecting the user's state, such as facial expression and line of sight. The operation unit 107 includes various operation members such as a power button and accepts operations by the user. The communication unit 108 is a device that complies with communication standards such as Ethernet and IEEE802.11, and transmits and receives data to and from external devices such as external sensors that measure the user's sweat rate. The storage unit 109 stores various data. The storage unit 109 may be a hard disk or memory card fixed to the display device 100, or may be a memory card, flexible disk, IC card, or the like that is detachable from the display device 100.

[0016] FIG. 2 is a block diagram showing an example of the functional configuration of the display device 100. The functional units shown in FIG. 2 are realized, for example, by the CPU 101 executing a program. Note that the display device 100 may include a dedicated processing circuit that realizes the functional units shown in FIG. 2, separate from the CPU 101. All of the functional units shown in FIG. 2 may be realized by the CPU 101, or all of the functional units may be realized by dedicated processing circuits, or some of the functional units may be realized by the CPU 101 and the remaining functional units may be realized by dedicated processing circuits. One dedicated processing circuit may realize one functional unit, one dedicated processing circuit may realize multiple functional units, or multiple dedicated processing circuits may realize one functional unit.

[0017] The user information acquisition unit 201 acquires user information. For example, the user information acquisition unit 201 acquires at least a part of the user information from the sensing unit 106, or acquires at least a part of the user information from an external sensor via the communication unit 108. The user information is information about the user's condition, and includes, for example, facial expression and vital sign information such as heart rate, blood pressure, and sweat rate.

[0018] The attention object detection unit 202 detects an attention object, which is a real object (an object existing in real space) at which the user is directing his or her gaze. For example, the attention object detection unit 202 acquires information about the user's gaze from the sensing unit 106, and acquires an image of real space (image data (image data)) from the imaging unit 104. Then, the attention object detection unit 202 detects an attention object from the image of real space based on the gaze information. The image of real space captured by the imaging unit 104 can also be considered as an image of the user's field of vision.

[0019] The image processing target determination unit 203 determines a real object to be subjected to image processing based on the user information acquired by the user information acquisition unit 201. For example, the image processing target determination unit 203 determines whether the user has felt a negative emotion such as discomfort, fear, or agitation based on the user's vital signs. If the image processing target determination unit 203 determines that the user has felt a negative emotion, it determines the object of interest detected by the object of interest detection unit 202 as the object of image processing.

[0020] The image processing determination unit 204 determines image processing to be performed on the image processing target based on the user information acquired by the user information acquisition unit 201. The image processing determined by the image processing determination unit 204 is image processing that reduces the saliency of the image processing target, such as graphic superimposition processing, erasing processing of real objects, blurring processing of real objects, and mosaic processing of real objects. For example, the image processing determination unit 204 varies the type and intensity of image processing depending on the user information. Saliency may be considered as the degree of prominence or the magnitude of the mental stress (psychological stress) imposed on the user.

[0021] When the image processing target determination unit 203 determines the image processing target, it records (registers) information about the target object in a user characteristic DB (DataBase) 211 in the storage unit 109. When the image processing determination unit 204 determines the image processing, it records (registers) information about the image processing in the user characteristic DB 211 so as to associate the information with the target object. When the target object is determined as the image processing target, the information about the image processing target and the information about the image processing are associated with each other and recorded in the user characteristic DB 211. Note that the user characteristic DB 211 may previously store information about real objects (candidates for the target object) and information about the image processing corresponding to the real objects in association with each other.

[0022] The image processing unit 205 performs image processing on the image in real space captured by the imaging unit 104 so as to reduce the saliency of the image processing target determined by the image processing target determination unit 203. The image processing determined by the processing determination unit 204 is performed. However, the virtual object superimposition process is not performed by the image processing unit 205 but by the virtual object superimposition unit 206.

[0023] When the image processing determination unit 204 determines that a virtual object should be superimposed, the virtual object superimposition unit 206 performs superimposition processing of the virtual object on the image in real space captured by the imaging unit 104 so as to reduce the salience of the image processing target. In the virtual object superimposition processing, the size and position of the virtual object are adjusted to fit the area of ​​the image in real space that is the image processing target, and the virtual object is placed (superimposed) on the image in real space. A virtual object DB 212 in the storage unit 109 stores virtual objects (virtual object data (image data)) in advance, and the virtual object superimposition unit 206 reads and uses the virtual object from the virtual object DB 212. The virtual object is a real object, such as a dog, spider, or snake, that may evoke negative emotions in the user, virtualized so as not to evoke negative emotions in the user. Virtual objects created using an external CG modeling tool or the like can be added to the virtual object DB 212 as needed.

[0024] When the image processing target determination unit 203 determines an image processing target and the image processing determination unit 204 determines image processing, the display unit 105 displays the real space image after the image processing on the display surface. When the image processing target determination unit 203 does not determine an image processing target, the display unit 105 displays, for example, an image in real space that has not been subjected to image processing by the image processing unit 205 or the virtual object superimposition unit 206 on the display surface.

[0025] FIG. 3 is a table showing an example of the correspondence between the level of negative emotion (emotion level) based on the user's vital signs and the image processing determined by the image processing determination unit 204. The range of the intensity of negative emotion (emotion intensity) is divided into multiple emotion levels using multiple thresholds, with the stronger the emotion intensity, the higher the emotion level. The image processing determination unit 204, for example, determines the user's emotion level based on the user's vital signs, and determines image processing according to the determined emotion level so that the higher the determined emotion level, the stronger the degree of suppression of negative emotion. When a user experiences negative emotion, they show reactions such as agitation depending on the emotion. The emotion level can also be considered a reaction level. Note that the correspondence between emotion levels and image processing (e.g., the number of emotion level stages and candidate image processing) is not limited to that shown in FIG. 3.

[0026] According to the correspondence relationship in Fig. 3, in the case of emotion level 1, virtual object superimposition processing is performed. In the case of emotion level 2, virtual object superimposition processing is also performed. However, in the case of emotion level 2, texture modification processing of the virtual object is also performed so as to further suppress negative emotions. For example, when superimposition processing of a living virtual object is performed, the texture modification processing changes the color of the texture to a single color, or changes the transmittance or reflectance of the texture so as to reduce the lifelikeness of the virtual object.

[0027] At emotion level 3, blurring of the image processing target (the real object that is the object of interest) is performed. At emotion level 4, superimposition of a warning dialog notifying the user of the presence of a real object that the user dislikes is performed. At emotion level 5, elimination of the image processing target is performed. This elimination process uses, for example, a technology called DR (Diminished Reality). At emotion level 5, in order to notify the user of the presence of the real object (image processing target), a process of extracting and superimposing the contour of the real object is also performed (superimposition of the contour of the image processing target; emphasis of the contour of the image processing target). At emotion level 6, elimination of the image processing target is also performed. However, at emotion level 6, superimposition of the contour of the image processing target is not performed.

[0028] Fig. 4 is a flowchart showing an example of the operation of display device 100. For example, the operation of Fig. 4 is realized by CPU 101 executing a program. The operation of Fig. 4 is started, for example, when display device 100 is powered on, and is repeated at a predetermined interval until display device 100 is powered off.

[0029] First, in step S401, the user information acquisition unit 201 acquires information on facial expressions and vital signs such as heart rate, blood pressure, and sweat rate as user information.

[0030] Next, in step S402, the image processing target determination unit 203 determines whether the user has felt a negative emotion such as discomfort or fear based on the user information (user's vital signs) acquired in step S401. For example, the image processing target determination unit 203 determines the intensity of the negative emotion (emotion intensity) based on the vital signs, and determines whether the determined emotion intensity is higher than a predetermined threshold. If it is determined that the user has felt a negative emotion (if the emotion intensity is determined to be higher than the predetermined threshold), the process proceeds to step S403. If it is determined that the user has not felt a negative emotion (if the emotion intensity is determined to be equal to or lower than the predetermined threshold), this operation is terminated.

[0031] In step S403, the attention object detection unit 202 detects an attention object, which is a real object at which the user is directing his or her gaze, from the image of real space captured by the imaging unit 104. The attention object is detected using a known technique such as object recognition using deep learning.

[0032] Next, in step S404, the image processing target determination unit 203 determines that the user felt negative emotions when looking at the target object detected in step S403, and determines the target object as the target of image processing.

[0033] In step S403, the attention object detection unit 202 may detect an attention area, which is an area (range) where the user is directing their gaze, instead of an attention object. Then, in step S404, the image processing target determination unit 203 may detect a real object from the attention area and determine the detected real object as an object to be processed for image processing. The real object to be processed for image processing is detected using a known technique such as object recognition using deep learning.

[0034] Next, in step S405, the image processing determination unit 204 determines the image processing to be performed on the image processing target determined in step S404, based on the user information acquired in step S401. For example, information on the correspondence relationship (the correspondence relationship between emotion levels and image processing) as shown in Fig. 3 is pre-recorded in the ROM 103. The image processing determination unit 204 determines the emotion level corresponding to the emotion intensity determined in step S402. Then, the image processing determination unit 204 determines the image processing corresponding to the determined emotion level in the correspondence relationship recorded in ROM 103 as the image processing to be performed on the image processing target.

[0035] Next, in step S406, the image processing target determination unit 203 records information about the image processing target (object of interest) determined in step S404 in the user characteristic DB 211 in the storage unit 109. For example, character strings (label information) such as "Dog," "Spider," or "Snake" are recorded in the user characteristic DB 211. In addition, the image processing determination unit 204 records information about the image processing determined in step S405 in the user characteristic DB 211 so as to associate it with the image processing target (object of interest).

[0036] Next, in step S407, the image processing unit 205 and the virtual object superimposing unit 206 At least one of the image processing units 206 performs the image processing determined in step S405 on the real space image captured by the imaging unit 104. The image processing is performed so as to reduce the salience of the image processing target determined in step S404. In this way, the real space image after the image processing is displayed on the display surface of the display unit 105. A specific example when virtual object superimposition processing is determined will be described. The virtual object superimposition unit 206 searches the virtual object DB 212 for a virtual object corresponding to the image processing target. If a virtual object corresponding to the image processing target exists, the virtual object superimposition unit 206 performs superimposition processing of the virtual object. If a virtual object corresponding to the image processing target does not exist, the virtual object superimposition unit 206 performs superimposition processing of a warning dialog.

[0037] As described above, according to the first embodiment, when the attention object is considered to be a mental obstacle to the user based on the information about the user's state, the display of the display unit 105 is automatically controlled so as to reduce the salience of the attention object. This eliminates the need for user operation, and allows the user to quickly avoid viewing the real object that is a mental obstacle.

[0038] In the first embodiment, the visual recognition of the real space through the display surface is the visual recognition of an image of the real space captured and displayed on the display surface, but this is not limited to this. For example, if the display surface of the display unit 105 is transparent, like the lens part of smart glasses, the user can view the raw real space, not the captured image, through the display surface.

[0039] Although the first embodiment has been described with reference to the display device 100 to which the present invention is applied, the present invention can be applied to various devices capable of controlling the display of a display device. For example, the present invention can be applied to a PC (personal computer) or a game console that is separate from the display device.

[0040] <Example 2> A second embodiment of the present invention will be described below. Note that, in the following, a description of the same points (such as configurations and processes) as those in the first embodiment will be omitted, and differences from the first embodiment will be described. In the first embodiment, it is determined whether the user has felt negative emotions when looking at a target object based on the user's vital signs. If it is determined that the user has felt negative emotions, the display of the display unit 105 is controlled so as to reduce the salience of the target object. According to the method of the first embodiment, the user can quickly avoid viewing a real object that is a mental obstacle. However, in the method of the first embodiment, in order to reduce the salience of a target object that evokes negative emotions in the user, the user will experience negative emotions once. Therefore, in the second embodiment, the display of the display unit 105 is controlled in consideration of the user's characteristics (past operations of the display device 100) so as to reduce the user's negative emotions.

[0041] Fig. 5 is a flowchart showing an example of the operation of display device 100. For example, the operation of Fig. 5 is realized by CPU 101 executing a program. The operation of Fig. 5 is started, for example, when display device 100 is powered on, and is repeated at a predetermined interval until display device 100 is powered off.

[0042] First, in step S501, the user information acquisition unit 201 acquires vital sign information as user information. This process is the same as the process in step S401 in Fig. 4. Next, in step S502, the target object detection unit 202 detects a target object from an image in real space. This process is the same as the process in step S403 in Fig. 4.

[0043] Next, in step S503, the image processing target determination unit 203 records information on the real object of the same type as the target object detected in step S502 in the user characteristic DB 211. 4. Since the real objects recorded in the user characteristic DB 211 are mental obstacles, there is a high possibility that real objects of the same type as the target object detected in step S502 are also mental obstacles. Taking this into consideration, if it is determined that information on a real object of the same type as the target object detected in step S502 is recorded in the user characteristic DB 211, the process proceeds to step S504. If it is determined that information on a real object of the same type as the target object detected in step S502 is not recorded in the user characteristic DB 211, the process proceeds to step S507. The processes of steps S507 to S511 are the same as the processes of steps S402 and S404 to S407 in FIG. 4.

[0044] In step S504, the image processing target determination unit 203 determines that the user feels negative emotions when looking at the target object detected in step S502, and determines the target object as a target for image processing. Next, in step S505, the image processing determination unit 204 determines, in the user characteristic DB 211, image processing associated with a real object of the same type as the target object, as image processing to be performed on the target object for image processing determined in step S504. The processing in steps S504 and S505 is not based on the user information acquired in step S501. Next, in step S506, at least one of the image processing unit 205 and the virtual object superimposition unit 206 applies the image processing determined in step S505 to the image in real space captured by the imaging unit 104. The image processing is performed so as to reduce the salience of the target object for image processing determined in step S504. In this way, the image in real space after image processing is displayed on the display surface of the display unit 105.

[0045] As described above, according to the second embodiment, it is determined whether information on a real object (first object) of the same type as the target object (second object) is recorded in the user characteristic DB 211. If the first object is recorded in the user characteristic DB 211, the display on the display unit 105 is controlled in the same manner as when the user was directing his or her gaze at the first object, regardless of the user information. In this way, the user can easily identify a real object that once gave him or her negative feelings, and a real object of the same type as a pre-registered real object. but Visibility To be In this case, the viewer can avoid the view before it develops negative emotions.

[0046] Example 3 A third embodiment of the present invention will be described below. Note that, in the following, a description of the same points as in the first embodiment (such as configuration and processing) will be omitted, and differences from the first embodiment will be described. In the first and second embodiments, an example in which vital sign information is used as user information will be described. In the first and second embodiments, an example in which the salience of a target object is reduced will be described. In the third embodiment, an example in which information on the time a user directs their gaze toward a real object (attention time) will be described as user information will be described. When the user's attention time on a real object is long, it is considered that the user wants to concentrate on viewing the real object. In such a case, other real objects can become mental obstacles that reduce the user's concentration. Therefore, in the third embodiment, an example in which the salience of real objects other than the target object is reduced will be described.

[0047] FIG. 6 is a table showing an example of the correspondence between the level of attention time (attention level) of the user to a real object (target object) and the image processing determined by the image processing determination unit 204. It is assumed that the range of attention time is divided into a plurality of attention levels using a plurality of thresholds, and the longer the attention time, the higher the attention level. The image processing determination unit 204 determines the attention level based on the attention time, for example, and determines image processing according to the determined attention level so that the higher the determined attention level, the easier it is to concentrate on the target object. Note that the correspondence between attention levels and image processing (the number of stages of attention levels, candidate image processing, etc.) can be calculated using the following: It is not limited to what is shown in FIG.

[0048] According to the correspondence relationship in Fig. 6, at attention level 1, blurring of the image processing target (all or part of the real objects other than the target object) is performed. At attention level 2, superimposition of a virtual object is performed. At attention level 3, erasure of the image processing target is performed.

[0049] Fig. 7 is a flowchart showing an example of the operation of display device 100. For example, the operation of Fig. 7 is realized by CPU 101 executing a program. The operation of Fig. 7 is started, for example, when display device 100 is powered on, and is repeated at a predetermined interval until display device 100 is powered off.

[0050] First, in step S701, the attention object detection unit 202 detects an attention object from an image in real space. This process is the same as the process in step S403 in Fig. 4. Next, in step S702, the user information acquisition unit 201 acquires, as user information, information on the user's attention time on the attention object detected in step S701.

[0051] Next, in step S703, the image processing target determination unit 203 determines whether the attention time exceeds a predetermined threshold based on the user information (attention time) acquired in step S702. If it is determined that the attention time exceeds the predetermined threshold, the process proceeds to step S704, and if it is determined that the attention time does not exceed the predetermined threshold, the process ends.

[0052] Next, in step S704, the image processing target determination unit 203 determines that the user wants to concentrate on viewing the target object detected in step S701, and determines all real objects other than the target object as targets for image processing. Note that some real objects other than the target object may also be determined as targets for image processing.

[0053] Next, in step S705, the image processing determination unit 204 determines the image processing to be performed on the image processing target determined in step S704, based on the user information acquired in step S702. For example, information on the correspondence relationship (the correspondence relationship between attention level and image processing) as shown in FIG. 6 is pre-recorded in the ROM 103. The image processing determination unit 204 determines the attention level corresponding to the user's attention time. Then, the image processing determination unit 204 selects the image processing corresponding to the determined attention level in the correspondence relationship recorded in the ROM 103. , painting The image processing to be performed on the image processing target is determined.

[0054] Next, in step S706, at least one of the image processing unit 205 and the virtual object superimposition unit 206 performs the image processing determined in step S705 on the image of real space captured by the imaging unit 104. This processing is the same as the processing in step S407 in FIG.

[0055] As described above, according to the third embodiment, the object of interest is determined based on the information about the user's state. other than When a real object of interest is considered to be a mental obstacle to the user, the display of the display unit 105 is automatically controlled to reduce the salience of the real object. This eliminates the need for user operation, allowing the user to quickly avoid viewing the real object that is a mental obstacle. Furthermore, when the user wants to concentrate on viewing the target object, the user's concentration can be expected to improve, and the efficiency of the work the user performs while looking at the target object can be expected to improve.

[0056] It should be noted that Examples 1 to 3 are merely examples, and the structures of Examples 1 to 3 may be used within the scope of the gist of the present invention. The present invention also includes configurations obtained by appropriately modifying or changing the configuration of the first to third embodiments. The present invention also includes configurations obtained by appropriately combining the configurations of the first to third embodiments. For example, in the third embodiment, as in the second embodiment, the display of the display unit 105 may be controlled in consideration of the user's characteristics (past operations of the display device 100). [Explanation of symbols]

[0057] 100:Display device 101:CPU 105:Display unit 201: User information acquisition unit 202: Attention object detection unit 203: Image processing target determination unit 204: Image processing determination unit 205: Image processing unit 206: Virtual object superimposition unit

Claims

1. an imaging means for imaging a real space; a display means for displaying the image of the real space captured by the imaging means so that the image can be visually recognized by a user; acquiring means for acquiring user information relating to the user's status; a control means for controlling the display of the display means so as to reduce the salience of a first object, which is a real object at which the user is looking, or a real object other than the first object, based on the user information; and the control means performs a first control for the first object based on the user information, and then, when the user directs his or her gaze to a second object that is a real object of the same type as the first object, performs a second control for the second object without being based on the user information; when the first control is a control for reducing the salience of the first object, the second control is a control for reducing the salience of the second object; When the first control is a control for reducing the salience of a real object other than the first object, the second control is a control for reducing the salience of a real object other than the second object. A head-mounted display characterized by:

2. The user information is information on the user's vital signs.

2. The head-mounted display according to claim 1.

3. The vital signs include at least one of facial expression, heart rate, blood pressure, and sweat rate.

3. The head-mounted display according to claim 2.

4. The control means controls the display of the display means to reduce the salience of the first object based on the user information.

4. The head-mounted display according to claim 2 or 3.

5. The user information includes information about the time the user directs his or her gaze toward a real object.

2. The head-mounted display according to claim 1.

6. The control means controls the display of the display means to reduce the salience of real objects other than the first object based on the user information.

6. The head-mounted display according to claim 5.

7. The control means controls the image to be displayed after image processing based on the user information.

7. The head-mounted display according to claim 1, wherein the head-mounted display is a head-mounted display.

8. The control means changes the type of image processing depending on the user information.

8. The head-mounted display according to claim 7.

9. The image processing is a graphic overlay process, a real object erasure process, a real object blurring process, or a real object mosaic process.

9. The head-mounted display according to claim 7 or 8.

10. The graphic overlay processing is overlay processing of a virtual object or overlay processing of a warning dialog.

10. The head-mounted display according to claim 9.

11. The image processing includes a process of overlaying the virtual object and a process of changing the texture of the virtual object. The head-mounted display according to claim 10 .

12. The image processing includes a process of erasing the real object and a process of superimposing the contour of the real object.

10. The head-mounted display according to claim 9.

13. The control means performs the second control based on first information about the first object and second information about the first control, both of which are recorded in a predetermined database.

13. The head-mounted display according to claim 1.

14. The user information includes information about a time period during which the user directs his or her gaze at a real-world object; The control means When the time is a first time, the image is subjected to image processing for blurring real objects other than the first object, and the image is displayed; When the time is a second time longer than the first time, the image is subjected to image processing to superimpose a virtual object on a real object other than the first object, and the image is displayed; When the time is a third time longer than the second time, the image is subjected to image processing for erasing real objects other than the first object and is then displayed.

2. The head-mounted display according to claim 1.

15. The control means determining a negative emotion level according to the user information; When the negative emotion level is a first level, control is performed to display the image after performing image processing to superimpose a virtual object on the first object or a real object other than the first object, which is a processing target; When the negative emotion level is a second level higher than the first level, the image is subjected to image processing for superimposing a virtual object on the processing target and image processing for changing a texture of the virtual object, and the image is then displayed; When the negative emotion level is a third level higher than the second level, the image is subjected to image processing for blurring the processing target and then displayed; When the negative emotion level is a fourth level higher than the third level, the image is subjected to image processing for superimposing a warning dialog and then displayed; when the negative emotion level is a fifth level higher than the fourth level, control is performed so that the image is displayed after undergoing image processing for erasing the processing object and image processing for superimposing a contour line of the processing object; When the negative emotion level is a sixth level higher than the fifth level, the image is controlled so as to be displayed without image processing for superimposing the contour line of the processing object, but by image processing for erasing the processing object.

11. The head-mounted display according to claim 1.

16. The control means Controlling the image so that the image is subjected to image processing based on the user information and then displayed; When performing image processing to superimpose virtual objects, Obtaining a virtual object corresponding to the first object or a real object other than the first object, which is a processing target, from a predetermined database; If a virtual object corresponding to the processing target exists, image processing is performed to superimpose the virtual object corresponding to the processing target; If there is no virtual object corresponding to the processing target, image processing is performed to superimpose a warning dialog.

11. The head-mounted display according to claim 1.

17. A method for controlling a head-mounted display having an imaging means for imaging a real space, the head-mounted display displaying an image of the real space imaged by the imaging means so as to be visible to a user, the method comprising: an acquisition step of acquiring user information relating to a user's status; a control step of controlling display of the head-mounted display based on the user information so as to reduce the salience of a first object, which is a real object at which the user is looking, or a real object other than the first object; in the control step, after performing a first control for the first object based on the user information, when the user directs his / her gaze to a second object that is a real object of the same type as the first object, performing a second control for the second object without using the user information; when the first control is a control for reducing the salience of the first object, the second control is a control for reducing the salience of the second object; When the first control is a control for reducing the salience of a real object other than the first object, the second control is a control for reducing the salience of a real object other than the second object. A control method comprising:

18. A program for causing a computer to execute each step of the control method according to claim 17. Hmm.

Images