Display device, display method, and program

Abstract

To appropriately act on a predetermined person.SOLUTION: The display device includes a transmissive portion that transmits external light, a light-shielding portion that switches a transmittance of incident light through the transmissive portion from transmission to blocking for each region, a first imaging unit that captures a visual field range viewed from the transmissive portion and the light-shielding portion, a person extraction unit that specifies a region of a person from a captured image captured by the first imaging unit, and extracts a predetermined person who is present in a predetermined size or more and for a predetermined time or more from the region of the predetermined person, a state determination unit that determines whether a motion amount of a pixel in a region other than the predetermined person is equal to or greater than a threshold, and a light-shielding control unit that sets the transmissivity of a region other than the region of the predetermined person to be lower than the transmissivity of the region of the predetermined person when the motion amount is equal to or greater than the threshold.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a display device, a display method, and a program. [Background technology]

[0002] There are known technologies for displaying information on wearable devices. For example, Patent Document 1 discloses a technology that can display information about a caregiver on a smart glass display and control the transmittance of the display so as not to appear suspicious to the caregiver. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-40326 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology of Patent Document 1 does not take into consideration a situation where multiple people are present around the user wearing the smart glasses. Therefore, when multiple people are present and the surroundings are hectic, it is difficult to act appropriately around a specific person.

[0005] In view of the above-described problems, the present disclosure aims to provide a display device, a display method, and a program that can take appropriate action toward a specific person. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the display device of the present disclosure includes a transmission section that transmits external light, a shading section that switches the transmittance of incident light that passes through the transmission section from transmission to blocking for each region, a first imaging section that captures an image of the field of view visible through the transmission section and the shading section, a person extraction section that identifies a person's region from the captured image captured by the first imaging section and extracts a specified person that is equal to or larger than a specified size and that exists for a specified period of time from the person's region, a state determination section that determines whether the amount of movement of pixels in regions other than the specified person is equal to or larger than a threshold, and a shading control section that, if the amount of movement is equal to or larger than the threshold, reduces the transmittance of regions other than the specified person's region to be lower than the transmittance of the specified person's region.

[0007] In order to solve the above-mentioned problems and achieve the object, the display method of the present disclosure is a display device including a transparent section that transmits external light, a light-shielding section that switches the transmittance of incident light that passes through the transparent section from transparent to blocked for each region, and a first imaging section that captures an image of the field of view visible through the transparent section and the light-shielding section, and includes the steps of: identifying a person's region from an image captured by the first imaging section, and extracting a specific person that is equal to or larger than a specific size and that exists for a specific period of time from the person's region; determining whether the amount of movement of pixels in regions other than the specific person is equal to or larger than a threshold; and, if the amount of movement is equal to or larger than the threshold, lowering the transmittance of the region other than the specific person's region below the transmittance of the specific person's region.

[0008] In order to solve the above-mentioned problems and achieve the object, the program of the present disclosure causes a computer to execute the following steps in a display device including a transparent section that transmits light, a light-shielding section that switches the transmittance of incident light that passes through the transparent section from transparent to blocked for each region, and a first imaging section that captures the field of view visible through the transparent section and the light-shielding section: identifying a person's region from an image captured by the first imaging section, and extracting a specific person from the person's region that is greater than or equal to a predetermined size and that exists for more than a predetermined time; determining whether the amount of movement of pixels in regions other than the specific person is greater than or equal to a threshold; and, if the amount of movement is greater than or equal to the threshold, lowering the transmittance of the region other than the specific person's region below the transmittance of the specific person's region. [Effects of the Invention]

[0009] According to the present invention, it is possible to concentrate on a specific person. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram of a display device according to a first embodiment. [Figure 2] FIG. 2 is a schematic diagram of the display device according to the first embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a partitioned light blocking portion as seen from a user according to the first embodiment. [Figure 4] FIG. 4 is a block diagram showing an example of the configuration of the display device according to the first embodiment. [Figure 5] FIG. 5 is a flowchart showing an example of the flow of the display method according to the first embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a part of the user's field of view according to the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of light blocking according to the first embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of light blocking according to the first embodiment. [Figure 9] FIG. 9 is a diagram illustrating the user's eyes when light is blocked according to the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating the user's eyes when light is blocked according to the first embodiment. [Figure 11] FIG. 11 is a schematic diagram of a display device according to the second embodiment. [Figure 12] FIG. 12 is a block diagram showing an example of the configuration of a display device according to the second embodiment. [Figure 13] FIG. 13 is a flowchart showing an example of the flow of the display method according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to these embodiments. Furthermore, the components of these embodiments include those that are easily replaceable by those skilled in the art, or those that are substantially identical.

[0012] [First embodiment] (display device) A display device according to an embodiment will be described with reference to Fig. 1 to Fig. 4. Fig. 1 and Fig. 2 are schematic diagrams of a display device according to a first embodiment. Fig. 3 is a diagram showing an example of a partitioned light-shielding section as seen from a user according to the first embodiment. Fig. 4 is a block diagram showing an example of the configuration of a display device according to the first embodiment.

[0013] 1, 2, and 4, the display device 1 is worn by a user U, and includes a first imaging unit 12, a second imaging unit 14, a transmission unit 16, a light-shielding unit 18, a sensor 20, a storage unit 24, a communication unit 26, and a control unit 30. As shown in Fig. 1, the user U wearing the display device 1 faces in a direction where persons A, B, and C are present.

[0014] The display device 1 is a device worn on the head of a user U. The display device 1 is, for example, an eyeglass-type device such as AR (Augmented Reality) glasses, VR (Virtual Reality) glasses, or MR (Mixed Reality) glasses. The display device 1 may be, for example, a goggle-type device. The display device 1 may be, for example, a head-mounted device such as a VR head-mounted display. The display device 1 is a device that blocks light from partially covering the field of view of a user U wearing the display device 1. Hereinafter, the user U wearing the display device 1 will be referred to simply as user U, where appropriate.

[0015] The first imaging unit 12 captures an image of the field of view, which is the range of the user U's field of view. The first imaging unit 12 captures an image of the field of view that is visible through the transmission unit 16 and the light-shielding unit 18. The first imaging unit 12 is provided around the transmission unit 16. The first imaging unit 12, for example, captures an image that is within the field of view of the user U and generates a still image or a moving image. The first imaging unit 12 is, for example, a camera including an optical element and an imaging element. The first imaging unit 12 is, for example, a visible light camera or an infrared camera.

[0016] The second imaging unit 14 captures an image of the eyeball of the user U wearing the display device 1. The second imaging unit 14 is provided around the transmission unit 16. The second imaging unit 14 is, for example, a visible light camera or an infrared camera.

[0017] The transmission section 16 transmits external light. The transmission section 16 is a transparent display that transmits light and allows the view ahead to be seen through. The transmission section 16 is, for example, a self-luminous display such as an organic EL (Electro Luminescence) display. The transmission section 16 displays, for example, an image used for shading. The user U can see objects present in the field of view through the transmission section 16.

[0018] The light-shielding portion (transmittance adjusting portion) 18 is disposed in front of the transmissive portion 16. The light-shielding portion 18 is disposed opposite the transmissive portion 16. The light-shielding portion 18 can adjust the transmittance of light incident on the display device 1 for each region. The regions may be defined two-dimensionally to correspond to pixels, or may be divided horizontally into multiple regions. The light-shielding portion 18 switches between transmission and blocking by controlling the transmittance of light incident through the transmissive portion 16. The light-shielding portion 18 is composed of, for example, a liquid crystal element. In this case, the light-shielding portion 18 changes the orientation state of the liquid crystal in response to an applied voltage, thereby varying the transmittance of light incident on the display device 1 from transmission to blocking. The transmittance may be adjusted by adjusting the proportion of liquid crystals that change the light distribution state within the region, or by the light distribution state. The light-shielding portion 18 can change the orientation state of the liquid crystal at any location. In other words, the light-shielding portion 18 can control the incident light at any location from transmission to blocking. Figure 3 shows one of the light-shielding portions 18 as viewed from the front of the display device 1. As shown in FIG. 3, the light-shielding portion 18 is divided into a plurality of areas by the control unit 30. In this embodiment, the light-shielding portion 18 is divided into eight areas (N1 to N8). N indicates the number of areas divided by the light-shielding portion 18. Although FIG. 3 shows one of the light-shielding portions 18, the other light-shielding portion 18 is also divided into eight areas. The light-shielding portion 18 is not limited to being divided into eight areas (N=8), and may be divided into two or more areas. Furthermore, the light-shielding portion 18 does not have to be divided into areas. The light-shielding portion 18 may be integrated with the transmission portion 16.

[0019] The sensor 20 detects acceleration occurring in the transmission section 16. The sensor 20 is, for example, an acceleration sensor that detects acceleration in three axial directions: the X-axis direction, the Y-axis direction, and the Z-axis direction. The sensor 20 outputs the acceleration detection result to the control section 30.

[0020] The storage unit 24 stores various types of information. For example, the storage unit 24 stores images captured by the first imaging unit 12. The storage unit 24 stores information such as the contents of calculations performed by the control unit 30 and programs. The storage unit 24 includes at least one of, for example, a random access memory (RAM), a main storage device such as a read only memory (ROM), and an external storage device such as a hard disk drive (HDD).

[0021] The storage unit 24 stores, for example, information about the light blocking portion 18. The storage unit 24 stores the number of sections of the light blocking portion 18.

[0022] The communication unit 26 is a module that executes communication between the display device 1 and an external device.

[0023] The control unit 30 controls each unit of the display device 1. The control unit 30 has, for example, an information processing device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage device such as a RAM or a ROM. The control unit 30 executes a program that controls the operation of the display device 1 according to the present invention. The control unit 30 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 30 may be realized by a combination of hardware and software.

[0024] The control unit 30 includes a sensor control unit 32, an acceleration determination unit 34, a gaze detection unit 36, an imaging control unit 38, a person extraction unit 40, a movement information generation unit 42, a state determination unit 44, and a shading control unit 46.

[0025] The sensor control unit 32 controls the sensor 20. The sensor control unit 32 controls the sensor 20 to detect the acceleration occurring in the transmission unit 16. The sensor control unit 32 acquires the acceleration occurring in the transmission unit 16 from the sensor 20.

[0026] The acceleration determination unit 34 determines whether the acceleration occurring in the transmission unit 16 is equal to or greater than a predetermined value. The acceleration determination unit 34 determines whether the acceleration acquired by the sensor control unit 32 is equal to or greater than a predetermined value. If the acceleration determination unit 34 determines that the acceleration acquired by the sensor control unit 32 is equal to or greater than a predetermined value, it determines that the user U wearing the display device 1 is moving, and if the sensor control unit 32 determines that the acceleration acquired is less than a predetermined value, it determines that the user U wearing the display device 1 is stationary.

[0027] The gaze detection unit 36 ​​detects the gaze direction of the user U wearing the display device 1. The gaze detection unit 36 ​​detects the gaze direction of the user U, for example, by performing image recognition processing on the image captured by the second imaging unit 14 and identifying the direction in which the eyes of the user U are facing.

[0028] The imaging control unit 38 controls the first imaging unit 12 to capture an image of the field of view of the user U wearing the display device 1. The imaging control unit 38 acquires the captured image captured by the first imaging unit 12. The imaging control unit 38 controls the second imaging unit 14 to capture an image of the eyeballs of the user U wearing the display device 1. The imaging control unit 38 acquires the captured image captured by the second imaging unit 14.

[0029] The person extraction unit 40 identifies a person area from the captured image captured by the first imaging unit 12. The person extraction unit 40 extracts a predetermined person who is equal to or larger than a predetermined size and who has been present for a predetermined period of time or longer from the person area.

[0030] The movement information generation unit 42 calculates the amount of movement in the field of view of the user U. The movement information generation unit 42 may calculate the amount of movement using, for example, the first frame and the second frame. Information included in the amount of movement is, for example, movement of a person, movement of the person's face and mouth, etc.

[0031] The state determination unit 44 determines whether the amount of movement of pixels in an area other than a predetermined person is equal to or greater than a threshold. The state determination unit 44 determines whether the motion vector calculated by the motion information generation unit 42 is equal to or greater than a threshold. The state determination unit 44 determines the state of a person present in the field of view of the user U from the motion vector.

[0032] The shading section 18 is divided into multiple sections. The shading control section 46 applies a voltage to the shading section 18 to control the transmittance of the shading section 18. The shading control section 46 displays an image on the transmission section 16. When the amount of movement is equal to or greater than a threshold, the shading control section 46 reduces the transmittance of areas other than the area of ​​the specified person to be lower than the transmittance of the area of ​​the specified person. In other words, when the amount of movement is equal to or greater than a threshold, the shading control section 46 blocks a predetermined percentage or more of light in the shading section 18 in the area other than the area of ​​the specified person. When the determination result of the acceleration determination section 34 is less than a predetermined value and the person extraction section 40 has extracted the specified person, the shading control section 46 reduces the transmittance of areas other than the area of ​​the specified person to be lower than the transmittance of the area of ​​the specified person.

[0033] (Display method) The processing flow of the display method according to the embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the flow of the display method according to the first embodiment. The display device 1 performs processing when the acceleration of the user U wearing the display device 1 is less than a predetermined value. In other words, the display device 1 performs processing when the user U is standing still.

[0034] The gaze detection unit 36 ​​detects the gaze of the user U wearing the display device 1 (step S12). The imaging control unit 38 controls the second imaging unit 14 to capture an image of the eyeballs of the user U wearing the display device 1, and the gaze detection unit 36 ​​detects the gaze of the user U wearing the display device 1 based on the image captured by the second imaging unit 14. FIG. 6 is a diagram showing an example of a portion of a user's field of view according to the first embodiment. Here, one side of the field of view of the user U wearing the display device 1 is shown. Persons A, B, and C are present in one side of the field of view of the user U. In the example shown in FIG. 6, the gaze detection unit 36 ​​determines that the user U's gaze is directed in the direction in which persons A, B, and C are present.

[0035] The imaging control unit 38 controls the first imaging unit 12 to capture an image of the visual field of the line of sight of the user U wearing the display device 1 (step S10). The imaging control unit 38 acquires the captured image captured by the first imaging unit 12. Then, the process proceeds to step S14.

[0036] The person extraction unit 40 extracts people from the captured image captured by the first imaging unit 12 (step S14). The person extraction unit 40 identifies a person's area from the captured image captured by the first imaging unit 12 and extracts the people within that area. The person extraction unit 40 identifies an area for each person, for example, and extracts the people within that area. The person extraction unit 40 identifies the area of ​​the person's entire body and extracts the people. The person extraction unit 40 may also identify areas of parts of the person's body and extract those parts. For example, the person extraction unit 40 may identify areas such as the person's face or mouth, and extract the identified face, mouth, etc.

[0037] In the example shown in FIG. 6, the person extraction unit 40 identifies an area AR1 of person A, an area AR2 of person B, and an area AR3 of person C from the captured image captured by the first imaging unit 12, and extracts person A, person B, and person C.

[0038] The person extraction unit 40 determines whether or not a predetermined person exists in the person area that is equal to or larger than a predetermined size and that has been present for a predetermined period of time (step S16). If the person extraction unit 40 determines that a predetermined person exists that is equal to or larger than a predetermined size and that has been present for a predetermined period of time (step S16; No), the processing ends. If the person extraction unit 40 determines that a predetermined person exists that is equal to or larger than a predetermined size and that has been present for a predetermined period of time (step S16; Yes), the processing proceeds to step S18.

[0039] In the example shown in FIG. 6, person A and person C are moving, and person B is stationary. The person extraction unit 40 extracts person B as a predetermined person who is a predetermined size or larger and has been present for a predetermined time or longer. Note that the predetermined size refers to a certain size within the field of view. The predetermined time may be, for example, 10 seconds. Note that the predetermined time is not limited to 10 seconds.

[0040] The motion information generator 42 calculates the amount of motion in the field of view of the user U (step S18). The motion information generator 42 extracts a target area from the image data in the first frame and the image data in the second frame, and obtains a motion vector between the target area in the first frame and the target area in the second frame. The motion information generator 42 compares the images in the first and second frames to calculate the target area and the motion vector of the target area. In this embodiment, for example, the first frame may be the previous frame, and the second frame may be the current frame.

[0041] The motion information generation unit 42 compares the first frame with the second frame to extract a target region, and then calculates a motion vector for the target region based on the target region in the first frame and the target region in the second frame. In this embodiment, a target region in which a motion vector is generated is treated as a motion region, which is a region in which a motion vector (amount of motion) is generated. A motion vector is an index indicating the amount of movement (amount of movement of a motion region) of a motion region from the first frame to the second frame. The motion information generation unit 42 calculates the motion vector of the target region (motion region) from the position of the target region in the image of the first frame and the position of the target region in the image of the second frame. That is, for example, the motion information generation unit 42 calculates the difference between the position of the target region in the image of the first frame and the position of the target region in the image of the second frame in a coordinate system based on the image as the amount of movement (motion vector) of the target region (motion region). When there are multiple target regions (motion regions), the motion information generation unit 42 calculates a motion vector for each target region (motion region).

[0042] In the example shown in FIG. 6, the movement information generation unit 42 extracts an area AR1 of person A as the target area. The movement information generation unit 42 calculates a motion vector from the body movement of person A between the first frame and the second frame. The movement information generation unit 42 also calculates a motion vector for person C in the same way as for person A. The movement information generation unit 42 also calculates a motion vector for person B extracted by the person extraction unit 40. The movement information generation unit 42 extracts an area AR2 of person B as the target area. The movement information generation unit 42 calculates the motion vector from the facial movement, hand movement, etc. of person B between the first frame and the second frame. The movement information generation unit 42 may extract the face area as the target area and calculate the motion vector from the mouth movement.

[0043] The state determination unit 44 determines whether the amount of movement of pixels in an area other than the area of ​​the specified person is equal to or greater than a threshold (step S20). For example, the state determination unit 44 determines that the amount of movement is equal to or greater than a threshold when there is a person or object moving in an area other than the area of ​​the specified person. In this embodiment, the state determination unit 44 determines that the amount of movement of person A and person C is equal to or greater than a threshold. The state determination unit 44 determines the state of the person based on, for example, the motion vectors calculated by the motion information generation unit 42. For example, the state determination unit 44 determines that person A and person C are moving based on the motion vectors calculated by the motion information generation unit 42. Furthermore, for example, the state determination unit 44 may determine that person B is having a conversation with user U wearing the display device 1 based on the motion vectors calculated by the motion information generation unit 42.

[0044] If the state determination unit 44 determines that the amount of movement of pixels in an area other than the area of ​​the predetermined person is equal to or greater than the threshold (step S20; No), the process ends. If the state determination unit 44 determines that the amount of movement of pixels in an area other than the area of ​​the predetermined person is equal to or greater than the threshold (step S20; Yes), the process proceeds to step S22.

[0045] The shading control unit 46 reduces the transmittance of areas other than the area of ​​the specified person (step S22). FIGS. 7 and 8 are diagrams illustrating an example of shading according to the first embodiment. Here, an example of shading of one shading unit 18 of the user U is shown. As illustrated in FIG. 7, the shading control unit 46 divides the shading unit 18 into multiple sections, and reduces the transmittance of the areas of the shading unit 18 other than the area of ​​the specified person by making them darker than the area of ​​the specified person. In this embodiment, the shading control unit 46 reduces the transmittance of an area AR1 of person A and an area AR3 of person C, other than an area AR2 of person B. The shading control unit 46 reduces the transmittance of some sections of the shading unit 18 into which incident light enters the areas AR1 and AR2 of the shading unit 18. That is, the shading control unit 46 shades the areas N1 to N3 and N6 to N8 of the shading unit 18 divided by the shading control unit 46. 8, the shading control unit 46 may block light by displaying an image IM in an area of ​​the transmission unit 16 that corresponds to an area to be shaded by the shading control unit 46. In this embodiment, the shading control unit 46 shades light from areas of the transmission unit 16 that correspond to areas N1 to N3 and N6 to N8 of the shading unit 18 with the image IM.

[0046] 9 and 10 are diagrams illustrating the user's eyes when light is blocked according to the first embodiment. FIGS. 9 and 10 show the actual eye E of the user U when, for example, person B views the user U wearing the display device 1. Here, one of the user U's eyes is shown. As shown in FIG. 9 , person C can see the user U's eye E in an area AR11 that is not blocked by the shading control unit 46. Also, as shown in FIG. 10 , the shading control unit 46 may display an image EI of the user U's eye. In this case, for example, a transparent display 21 different from the transmission unit 16 is arranged in front of the shading unit 18, and the shading control unit 46 displays the eye image EI on the transparent display 21. The shading control unit 46 displays the eye image EI in an area of ​​the transparent display 21 that corresponds to the area blocked by the shading unit 18. In this embodiment, the shading control unit 46 displays the eye image EI1 and the eye image EI2 in the shading areas AR21 and AR22 of the shading unit 18 of the transparent display 21, respectively. The eye image EI may be captured in advance and stored in the storage unit 24.

[0047] In the first embodiment, the display device 1 blocks light from an area where the amount of movement is equal to or greater than a threshold. Therefore, when the amount of movement in the surrounding area is equal to or greater than a threshold, the display device 1 blocks light from the area where the amount of movement is equal to or greater than the threshold, allowing the user U to act appropriately toward a specific person. For example, the user U can concentrate on the conversation.

[0048] In the present embodiment, the display device 1 controls the shading so that the center of the shading portion 18 is transmitted, but the present invention is not limited thereto, and the shading may be controlled so that an area where a predetermined person is present is transmitted. For example, when the predetermined person is person A, the display device 1 may control the shading so that N1 to N3 of the shading portion 18 are transmitted. In the present embodiment, the shading process for one of the shading portions 18 is described, but in reality, both shading portions 18 are processed to shade areas other than the area where the predetermined person is present.

[0049] [Second embodiment] A display device 1A according to the second embodiment will be described with reference to Figs. 11 and 12. The second embodiment differs from the first embodiment in that the display device 1A recognizes voice. The same components as those in the first embodiment are given the same reference numerals, and descriptions thereof will be omitted. Fig. 11 is a schematic diagram of the display device according to the second embodiment. Fig. 12 is a block diagram showing an example of the configuration of the display device according to the second embodiment.

[0050] (display device) As shown in FIG. 11, the display device 1A differs from the display device 1 of the first embodiment in that it includes a microphone 27 and a control unit 30A.

[0051] The microphone 27 inputs voice. The microphone 27 is provided near the transmissive portion 16. The microphone 27 inputs voice around the user U wearing the display device 1A. The microphone 27 inputs, for example, the speech of a person within the field of view of the user U. The microphone 27 inputs the voice of the user U. The microphone 27 inputs, for example, the speech of the user U. The microphone 27 may be provided at any position on the display device 1A as long as it is a position where the surrounding voices and the speech of the user U can be input.

[0052] The control unit 30A differs from the control unit 30 of the first embodiment in that it includes a voice recognition unit 50 and a voice determination unit 52.

[0053] The voice recognition unit 50 recognizes voice input from the microphone 27. The voice recognition unit 50 recognizes voices around the user U wearing the display device 1A. The voice recognition unit 50 recognizes the number of voice inputs per predetermined time. The voice recognition unit 50 recognizes, for example, the speech of a person within the field of view of the user U. The voice recognition unit 50 recognizes the voice of the user U. The voice recognition unit 50 recognizes, for example, the speech of the user U.

[0054] The voice determination unit 52 determines the voice recognized by the voice recognition unit 50. The voice determination unit 52 determines whether the number of voice inputs recognized by the voice recognition unit 50 per predetermined time period is equal to or greater than a predetermined number. The voice determination unit 52 determines words per predetermined time period from the voice recognized by the voice recognition unit 50. For example, the voice determination unit 52 determines whether words of a predetermined level of difficulty or higher are included per predetermined time period from the voice recognized by the voice recognition unit 50.

[0055] The predetermined time is, for example, 10 seconds, but is not limited to 10 seconds.

[0056] For example, the voice determination unit 52 determines whether a person speaking speaks quickly. The voice determination unit 52 determines whether a person speaking quickly based on the number of voice inputs per 10 seconds. Words of a predetermined level of difficulty or higher may be, for example, words classified as highly difficult in technical terms or language tests. It is preferable that these words be stored in the storage unit 24 in advance.

[0057] In the second embodiment, the display device 1A recognizes the voice of the user U wearing the display device 1A and the voices of the surrounding area. Therefore, the state determination unit 44 may determine that the user U is conversing with a person from the voice recognized by the voice recognition unit 50. The shading control unit 46 also shades the shading unit 18 in response to the input voice. That is, when the voice determination unit 52 determines that the number of voice inputs is equal to or greater than a predetermined number, or when the amount of movement is equal to or greater than a threshold, the shading control unit 46 reduces the transmittance of the area other than the area of ​​the predetermined person to be lower than the transmittance of the predetermined person. The shading control unit 46 also reduces the transmittance of the area other than the area of ​​the predetermined person to be lower than the transmittance of the predetermined person in response to the word.

[0058] (Display method) Next, a display method of the display device 1A will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an example of the flow of the display method according to the second embodiment. The processes from step S10 to step S20 are the same as the processes from step 10 to step 20 shown in Fig. 5, respectively, and therefore description thereof will be omitted.

[0059] If the state determination unit 44 determines that the amount of movement of pixels in an area other than the area of ​​the predetermined person is equal to or greater than the threshold (step S20; Yes), the shading control unit 46 reduces the transmittance of the area other than the area of ​​the predetermined person (step S22). The process of step S22 is the same as the process of step S22 shown in FIG. 5, and therefore a description thereof will be omitted.

[0060] If the state determination unit 44 determines that the amount of movement of pixels in an area other than the area of ​​the specified person is equal to or greater than the threshold (step S20; No), the voice recognition unit 50 recognizes the number of voice inputs per predetermined time (step S30), and then proceeds to step S32.

[0061] The voice determination unit 52 determines whether the number of voices per predetermined time is equal to or greater than a predetermined number (step S32). If the voice determination unit 52 determines that the number of voices per predetermined time is not equal to or greater than the predetermined number (step S32; No), the voice determination unit 52 ends the process. If the voice determination unit 52 determines that the number of voices per predetermined time is equal to or greater than the predetermined number, the voice determination unit 52 reduces the transparency of areas other than the area of ​​the predetermined person (step S22).

[0062] In this embodiment, the display device 1A may lower the transmittance of the area outside the area of ​​the specified person depending on the difficulty of the words included in the voice, even if the amount of movement is less than the threshold. Also, even if the amount of movement is less than the threshold, the display device 1A may lower the transmittance of the area outside the area of ​​the specified person depending on the number of voice inputs and the difficulty of the words included in the voice. Also, although the display device 1A recognizes voice when the amount of movement is less than the threshold, it may also recognize voice when the amount of movement is equal to or greater than the threshold. Note that the display device 1A is not limited to recognizing voice and performing shading processing.

[0063] In the second embodiment, the display device 1A recognizes speech. The display device 1A also determines the number of speech inputs per predetermined time period and words of the recognized speech. Therefore, even if the surrounding motion vector is less than a threshold, that is, even if the surroundings are quiet, the user U can behave appropriately toward a specific person. For example, the user U can concentrate on the conversation.

[0064] [effect] As described above, the display device according to this embodiment includes a transmission unit 16 that transmits external light, a light-shielding unit that switches between transmitting and blocking incident light that has passed through the transmission unit 16, a first image capture unit 12 that captures an image of the field of view visible through the transmission unit 16 and the light-shielding unit 28, a person extraction unit 40 that identifies a person's area from the captured image captured by the first image capture unit 12 and extracts a predetermined person who is equal to or larger than a predetermined size and who has been present for a predetermined period of time from the person's area, a state determination unit 44 that determines whether the amount of motion of pixels in areas other than the predetermined person is equal to or larger than a threshold, and a light-shielding control unit 46 that, if the amount of motion is equal to or larger than the threshold, reduces the transmittance of areas other than the area of ​​the predetermined person to be lower than the transmittance of the area of ​​the predetermined person. Therefore, areas with an amount of motion above the threshold can be shaded, allowing the display device to act appropriately in relation to the predetermined person.

[0065] The display device according to this embodiment further includes a sensor 20 that detects acceleration occurring in the transmission section 16, and an acceleration determination section 34 that determines whether the acceleration occurring in the transmission section 16 is equal to or greater than a predetermined value, and when the determination result of the acceleration determination section 34 is less than the predetermined value and the person extraction section 40 has extracted the predetermined person, the shading control section 46 makes the transmittance of areas other than the area of ​​the predetermined person lower than the transmittance of the area of ​​the predetermined person. Therefore, when the user is standing still, the transmittance can be lowered.

[0066] The display device according to this embodiment further includes a microphone 27, a voice recognition unit 50 that recognizes the number of voice inputs per predetermined time period input to the microphone 27, and a voice determination unit 52 that determines whether the number of voice inputs per predetermined time period recognized by the voice recognition unit 50 is equal to or greater than a predetermined number, and if the number of voice inputs is equal to or greater than the predetermined number or the amount of movement is equal to or greater than a threshold, the shading control unit 46 reduces the transmittance of areas other than the area of ​​the specified person compared to the transmittance of the area of ​​the specified person. Therefore, by recognizing voice, the transmittance of the area of ​​the shading unit other than the area of ​​the specified person can be reduced, allowing the display device to act appropriately toward the specified person.

[0067] In the display device according to this embodiment, the voice determination unit 52 determines whether the voice recognized by the voice recognition unit 50 contains words of a predetermined level of difficulty or higher per predetermined time, and the shading control unit 46 reduces the transmittance of areas other than the area of ​​the predetermined person to be lower than the transmittance of the area of ​​the predetermined person depending on the words. Therefore, even if difficult words are contained, the display device can act appropriately toward the predetermined person.

[0068] The display method according to this embodiment is a display device including a transmission section 16 that transmits external light, a light-shielding section 18 that switches between transmitting and blocking incident light that has passed through the transmission section 16, and a first imaging section 12 that captures an image of the field of view visible through the transmission section 16 and the light-shielding section 18. The display method includes the steps of: identifying a person's area from an image captured by the first imaging section 12; extracting a predetermined person from the person's area that is equal to or larger than a predetermined size and that has been present for a predetermined period of time; determining whether the amount of motion of pixels in an area other than the predetermined person is equal to or larger than a threshold; and, if the amount of motion is equal to or larger than the threshold, lowering the transmittance of the area other than the predetermined person's area below the transmittance of the area of ​​the predetermined person. Therefore, the area with an amount of motion above the threshold can be blocked, allowing appropriate action to be taken in relation to the predetermined person.

[0069] The program according to this embodiment is a display device including a transmission unit 16 that transmits external light, a light-shielding unit 18 that switches between transmitting and blocking incident light that has passed through the transmission unit 16, and a first image capture unit 12 that captures a field of view visible through the transmission unit 16 and the light-shielding unit 18. The program causes a computer to execute the following steps in a display device: identifying a person's area from an image captured by the first image capture unit 12, extracting a specific person from the person's area that is equal to or larger than a specific size and that has been present for a specific period of time; determining whether the amount of motion of pixels in areas other than the specific person is equal to or larger than a threshold; and, if the amount of motion is equal to or larger than the threshold, lowering the transmittance of the area other than the specific person's area to be lower than the transmittance of the specific person's area. Therefore, the area with the amount of motion above the threshold can be blocked, allowing appropriate action to be taken in relation to the specific person.

[0070] The components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads and usage conditions. This distribution and integration configuration may also be performed dynamically.

[0071] Although the embodiments of the present invention have been described above, the present invention is not limited to the contents of these embodiments. Furthermore, the above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments. [Explanation of symbols]

[0072] 1, 1A display device 12 First imaging unit 14 Second imaging unit 16 Transparent part 18 Light blocking section 20 sensors 24 Memory section 26 Communications Department 27. Mike 30 Control Unit 32 Sensor control unit 34 Acceleration judgment section 36 Gaze detection unit 38 Imaging control unit 40 Person extraction part 42 Motion information generation unit 44 Status determination unit 46 Light blocking control section 50 Voice Recognition Unit 52 Audio determination unit A, B, C people AR1, AR2, AR3 area E eyes EI, EI1, EI2 Eye images IM Image U User

Claims

1. a transmission portion that transmits external light; a light-shielding portion that switches the transmittance of incident light that has passed through the light-transmitting portion from transmission to blocking for each region; a first imaging unit that captures an image of a field of view visible through the transmission unit and the light blocking unit; a person extraction unit that identifies a person area from the captured image captured by the first imaging unit and extracts a predetermined person who is equal to or larger than a predetermined size and who exists for a predetermined period of time from the person area; a state determination unit that determines whether the amount of movement of pixels in the predetermined non-person area is equal to or greater than a threshold; a light blocking control unit that, when the amount of movement is equal to or greater than a threshold, makes the transmittance of an area other than the area of ​​the specified person lower than the transmittance of the area of ​​the specified person; Equipped with Display device.

2. a sensor for detecting acceleration occurring in the transmission portion; an acceleration determination unit that determines whether the acceleration occurring in the transmission unit is equal to or greater than a predetermined value, when the determination result of the acceleration determination unit is less than a predetermined value and the person extraction unit has extracted the predetermined person, the shading control unit reduces the transmittance of an area other than the area of ​​the predetermined person to be lower than the transmittance of the area of ​​the predetermined person. The display device according to claim 1 .

3. a microphone; and a voice recognition unit that recognizes the number of voices input to the microphone per predetermined time period; a voice determination unit that determines whether the number of voice inputs recognized by the voice recognition unit per predetermined time period is equal to or greater than a predetermined number, the light blocking control unit, when the number of voice inputs is equal to or greater than a predetermined number, or when the amount of movement is equal to or greater than a threshold, makes the transmittance of an area other than the area of ​​the predetermined person lower than the transmittance of the area of ​​the predetermined person; The display device according to claim 1 .

4. the voice determination unit determines whether the voice recognized by the voice recognition unit contains words of a predetermined level of difficulty or higher per predetermined time period; the light blocking control unit reduces the transmittance of an area other than the area of ​​the specified person to be lower than the transmittance of the area of ​​the specified person, in accordance with the word; The display device according to claim 3 .

5. a transmission portion that transmits external light; a light-shielding portion that switches the transmittance of incident light that has passed through the light-transmitting portion from transmission to blocking for each region; a first imaging unit that images a field of view visible through the transmission unit and the light-shielding unit, identifying a person area from the captured image captured by the first imaging unit, and extracting a predetermined person who is equal to or larger than a predetermined size and who exists for a predetermined period of time from the person area; determining whether the amount of movement of pixels in the predetermined non-person area is equal to or greater than a threshold; If the amount of movement is equal to or greater than a threshold, setting the transmittance of an area other than the area of ​​the predetermined person to be lower than the transmittance of the area of ​​the predetermined person; Including, Display method.

6. a transmission portion that transmits external light; a light-shielding portion that switches the transmittance of incident light that has passed through the light-transmitting portion from transmission to blocking for each region; a first imaging unit that images a field of view visible through the transmission unit and the light-shielding unit, identifying a person area from the captured image captured by the first imaging unit, and extracting a predetermined person who is equal to or larger than a predetermined size and who exists for a predetermined period of time from the person area; determining whether the amount of movement of pixels in the predetermined non-person area is equal to or greater than a threshold; If the amount of movement is equal to or greater than a threshold, setting the transmittance of an area other than the area of ​​the predetermined person to be lower than the transmittance of the area of ​​the predetermined person; to the computer, program.

Images