Control device, control method, and control program

Abstract

To provide a control device, a control method, and a control program with which it is possible to maintain the visibility of projection images in a good condition and cut down the burden on a user's eyes.SOLUTION: Provided is a control device equipped with a processor to control the projection of a projection image to an eyeglass type display device which is provided with a transmission unit that provides a real image to a user's eyes and a projection unit that projects a projection image to the transmission unit in such a way as to be visually recognizable by the user's eyes. The processor of the control device acquires, with regard to the size of pupils of the user's eyes, a first size of the pupils when the projection unit is projecting the projection image with first brightness, acquires a second size of the pupils when the projection unit is projecting the projection image with second brightness that is brighter than the first brightness, and controls the brightness of the projection image that is projected to the transmission unit, in accordance with the comparison results of the first and second sizes.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present disclosure relates to a control device, a control method, and a control program.

Background Art

[0002] Conventionally, as a display device for displaying a stereoscopic image, there is known a glasses-type information display device such as an augmented reality (AR) device that displays an image in a state of being superimposed on the real world.

[0003] As a glasses-type information display device, there is a glasses-type information display device including a transmissive portion that provides a real image to the user's eyes and a projection portion that projects a projection image in a state visible to the user's eyes through the transmissive portion. By using the glasses-type information display device, the user may visually recognize the projection image, which places a burden on the eyes and may cause the eyes to become fatigued.

[0004] Therefore, in order to reduce the fatigue state of the user's eyes, for example, there is the technique described in Patent Document 1. Patent Document 1 describes a technique for measuring the size of the user's pupil and estimating the fatigue state of the user's eyes from the size of the pupil.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] According to the technique described in Patent Document 1, it is possible to estimate the fatigue state of the user's eyes and control the projection of the projection image in response to the fatigue state, but it is not sufficient to reduce the burden on the user's eyes itself. In addition, the visibility of the projection image is not considered, and the visibility of the projection image may decrease.

[0007] This disclosure is made in view of the above circumstances and aims to provide a control device, a control method, and a control program that can maintain good visibility of projected images and reduce the strain on the user's eyes. [Means for solving the problem]

[0008] To achieve the above objective, a control device according to a first aspect of this disclosure is a control device for a spectacle-type display device comprising a transparent section that provides a real image to the user's eye, and a projection section that projects a projected image onto the transparent section in a state that is visible to the user's eye, and the control device comprises a processor for controlling the projection of the projected image, wherein the processor obtains a first pupil size when the projection section projects the projected image at a first brightness, obtains a second pupil size when the projection section projects the projected image at a second brightness that is brighter than the first brightness, and controls the brightness of the projected image projected onto the transparent section according to the comparison result between the first and second sizes. Furthermore, if the first size is the same as the second size, control is performed to brighten the projected image.

[0009] A control device according to a second aspect of the present disclosure, in the control device according to the first aspect, the processor performs control to maintain the brightness of the projected image when the first size is larger than the second size and the difference between the first size and the second size is less than or equal to a predetermined threshold.

[0010] A control device according to a third aspect of the present disclosure, in the control device according to the first aspect, the processor performs control to darken the projected image when the first size is larger than the second size and the difference between the first size and the second size is greater than a predetermined threshold.

[0012] History of this disclosure 4 The control device in the following configuration is from the first configuration to the first configuration. 3 In a control device according to any one of the embodiments, the processor performs control to brighten the projected image when the first size is smaller than the second size.

[0013] History of this disclosure5 The control device in the following configuration is from the first configuration to the first configuration. 4 In a control device according to any one of the embodiments, the processor provides reference correspondence information representing the correspondence between the brightness of the image being viewed and the size of the pupil of the eye being viewed. Refer to the brightness of the projected image to be projected in order to obtain the second size, First size If we consider the pupil size of the eye to be the size of the eye that can see it Brightness of the corresponding visible image The decision will be made based on the following criteria. .

[0014] History of this disclosure 6 A control device in the manner of the first is 5 In a control device of the present invention, the processor uses information as reference correspondence information that represents the correspondence between the brightness of a projected image and the size of the user's pupil, obtained by projecting projected images with different brightness levels.

[0015] History of this disclosure 7 A control device in the manner of the first is 5 In the control device of the first or seventh embodiment, the processor updates the reference correspondence information with a value representing a post-control correspondence in which the brightness of the projected image after control, according to the comparison result of the first size and the second size, is the brightness of the image to be viewed, and the first size is the size of the pupil of the eye to be viewed.

[0016] History of this disclosure 8 A control device in the manner of the first is 7 In a control device of the form, the processor updates the old post-control correspondence The value that represents By sequentially excluding values ​​from the standard correspondence relationship information, the number of values ​​representing the controlled correspondence relationship included in the standard correspondence relationship information is controlled to be less than or equal to a predetermined number.

[0017] History of this disclosure 9 A control device in the manner of the first is 7 The manner or the 8 In a control device of the form described above, the processor controls the number of values ​​representing the post-control correspondence included in the reference correspondence information to be less than or equal to a first predetermined number for each predetermined range determined according to the size of the pupil of the eye being viewed in the reference correspondence information.

[0018] History of this disclosure10 The control device according to the aspect of 7 the aspect or the 8 In the control device according to the aspect of, the processor controls the number of values representing the post-control correspondence relationship included in the reference correspondence relationship information to be equal to or less than a second predetermined number for each predetermined range determined according to the brightness of the image to be visually recognized in the reference correspondence relationship information.

[0019] In the control device according to the 11 aspect of the present disclosure, 5 the aspect or the 6 In the control device according to the aspect of, the processor determines, for each predetermined range determined according to the size of the pupil of the eye to be visually recognized in the reference correspondence relationship information, for each predetermined range determined according to the brightness of the image to be visually recognized in the reference correspondence relationship information, or for each predetermined range determined according to the size of the pupil of the eye to be visually recognized and the Brightness in the reference correspondence relationship information, a representative value of the correspondence relationship between the brightness of the image to be visually recognized and the size of the pupil of the eye to be visually recognized, and derives the reference correspondence relationship information based on the representative value.

[0020] In the control device according to the 12 aspect of the present disclosure, 11 In the control device according to the aspect of, the processor uses the brightness of the projected image after control according to the comparison result between the first size and the second size as the brightness of the image to be visually recognized, and uses the first size as the size of the pupil of the eye to be visually recognized to represent the the value to update the representative value.

[0021] In the control device according to the aspect of the present disclosure, in the control device according to the aspect, the processor updates the representative value by weighting the value representing the post-control correspondence relationship. ru.

[0022] To achieve the above object, in the 14The control method for the embodiment of a spectacle-type display device comprises a transparent section that provides a real image to the user's eyes and a projection section that projects a projected image onto the transparent section in a state that is visible to the user's eyes. The control method is performed by a processor of a control device equipped with a processor that controls the projection of the projected image. The method involves obtaining a first size of the pupil of the user's eye when the projection section projects the projected image at a first brightness, and obtaining a second size of the pupil when the projection section projects the projected image at a second brightness that is brighter than the first brightness. The control method then controls the brightness of the projected image projected onto the transparent section according to the comparison result between the first and second sizes, and if the first size is the same as the second size, it controls the brightness of the projected image.

[0023] History of this disclosure 15 The control method for the aspect is the 14 state Mr. / Ms. In the control method, if the first size is larger than the second size, and the difference between the first size and the second size is less than or equal to a predetermined threshold, control is performed to maintain the brightness of the projected image.

[0024] To achieve the above objectives, the following disclosure 16 The control program in this embodiment is a control program executed by the processor of a control device equipped with a processor for controlling the projection of a projected image for a glasses-type display device comprising a transparent unit that provides a real image to the user's eyes and a projection unit that projects a projected image onto the transparent unit in a state that is visible to the user's eyes. The control program acquires a first size of the pupil of the user's eye when the projection unit projects the projected image at a first brightness, and a second size of the pupil when the projection unit projects the projected image at a second brightness that is brighter than the first brightness. The program controls the brightness of the projected image projected onto the transparent unit according to the comparison result between the first size and the second size, and controls the brightness of the projected image when the first size is the same as the second size.

[0025] History of this disclosure 17 The control program of the form is 16 state Mr. / Ms.In the control program, if the first size is larger than the second size, and the difference between the first size and the second size is less than or equal to a predetermined threshold, this control is used to maintain the brightness of the projected image. [Effects of the Invention]

[0026] According to this disclosure, it is possible to maintain good visibility of projected images while reducing the strain on the user's eyes. [Brief explanation of the drawing]

[0027] [Figure 1] This is a configuration diagram showing an example of the configuration of the eyeglass-type information display device according to the embodiment. [Figure 2] This is a perspective view showing an example of AR glasses according to an embodiment. [Figure 3] This diagram illustrates the pupil size detected by the pupil sensor. [Figure 4] This is a block diagram showing an example of the configuration of a smartphone according to the first embodiment. [Figure 5] This diagram illustrates the relationship between the brightness of a projected image relative to reality and the strain on the eye when viewing that projected image. [Figure 6A] This diagram illustrates the relationship between the pupil size of the eye that perceives the real image, the pupil size of the eye that perceives the projected image superimposed on the real image, and the brightness of the projected image. [Figure 6B] This diagram illustrates the relationship between the pupil size of the eye that perceives the real image, the pupil size of the eye that perceives the projected image superimposed on the real image, and the brightness of the projected image. [Figure 6C] This diagram illustrates the relationship between the pupil size of the eye that perceives the real image, the pupil size of the eye that perceives the projected image superimposed on the real image, and the brightness of the projected image. [Figure 6D] This diagram illustrates the relationship between the pupil size of the eye that perceives the real image, the pupil size of the eye that perceives the projected image superimposed on the real image, and the brightness of the projected image. [Figure 7]A block diagram showing an example of the hardware configuration of a smartphone according to the first embodiment. [Figure 8] This flowchart shows an example of projection control processing performed on a smartphone according to the first embodiment. [Figure 9] This is a block diagram showing an example of the configuration of a smartphone according to the second embodiment. [Figure 10] This figure shows an example of information related to compliance with standards. [Figure 11] A block diagram showing an example of the hardware configuration of a smartphone according to the second embodiment. [Figure 12] This flowchart shows an example of projection control processing performed on a smartphone according to the second embodiment. [Figure 13] This figure shows an example of reference correspondence information based on post-control correspondence. [Figure 14] This flowchart shows an example of the projection control process performed on a smartphone in the modified example 1. [Figure 15] This figure shows an example of the reference correspondence information for Modification Example 2. [Figure 16] This figure shows an example of the reference correspondence information for Modification Example 3. [Figure 17] This figure shows an example of the reference correspondence information for Modification Example 4. [Figure 18] This diagram illustrates an example of how reference correspondence information is derived in Modification Example 5. [Figure 19] This flowchart shows an example of the weight acquisition process performed on a smartphone according to the third embodiment. [Modes for carrying out the invention]

[0028] Hereinafter, with reference to the drawings, examples of embodiments for carrying out the technology of this disclosure will be described in detail.

[0029] [First Embodiment] Referring to Figure 1, the configuration of the display system 1 of this embodiment will be described. As shown in Figure 1, the display system 1 of this embodiment includes AR (Augmented Reality) glasses 10 and a smartphone 12. The AR glasses 10 of this embodiment are an example of the glasses-type information display device of this disclosure, and the smartphone 12 of this embodiment is an example of the control device of this disclosure.

[0030] AR glasses 10 are a device that enables a user to view a projected image superimposed on a real image, projected from an OLED (Organic Light Emitting Diode) 26. Figure 2 shows a perspective view of an example of AR glasses 10 according to this embodiment. As shown in Figures 1 and 2, AR glasses 10 comprises a pair of transparent sections 20L for the left eye and 20R for the right eye, an OLED 26, and a pupil sensor 27. The right eye transparent section 20R in this embodiment is an example of a transparent section in this disclosure. The OLED 26 in this embodiment is an example of a projection section in this disclosure.

[0031] The OLED26 projects a projected image onto the light guide plate24 so that the projected image is displayed superimposed on the real image seen by the user's right eye through the right-eye transmissive section20R.

[0032] The right-eye transmissive section 20R includes a right-eye lens 22R and a light guide plate 24. Light corresponding to the projected image projected from the OLED 26 is incident on one end of the light guide plate 24. The light propagating through the light guide plate 24 changes direction at an emission section (not shown) and is emitted towards the user's eye. The light corresponding to the projected image emitted from the light guide plate 24 passes through the right-eye lens 22R and is guided to the user's right eye. The user then perceives the real world as a real image through the right-eye lens 22R with their right eye.

[0033] Therefore, while an image is being projected from the OLED 26, the image perceived by the user's right eye is a superposition of the real-world image seen through the right-eye lens 22R and the projected image projected onto the light guide plate 24. Conversely, when no image is being projected from the OLED 26, the image perceived by the user is the real-world image seen through the right-eye lens 22R and the light guide plate 24.

[0034] On the other hand, the left eye's transparent section 20L includes the left eye's lens 22L. The user views the real world through the left eye's lens 22L with their left eye.

[0035] The pupil sensor 27 is a sensor that has the function of detecting the size of the pupil of the user's right eye. As shown in Figure 3, the human eye has a pupil 92, which is what is commonly called the black part of the eye. The pupil 92 changes in size (dilates and contracts) to adjust the amount of light entering the eye 90. The pupil sensor 27 is a sensor that detects the diameter S of the pupil 92 as the size of the pupil 92. The pupil sensor 27 is not particularly limited, and as an example, the pupil sensor 27 in this embodiment employs a camera that outputs a captured image of the eye 90 to the smartphone 12. The timing at which the pupil sensor 27 captures the eye 90 is not particularly limited, and for example, it may be the timing at which the first acquisition unit 50 and the second acquisition unit 52, which will be described later, each acquire the captured image.

[0036] On the other hand, the smartphone 12 is equipped with a processor 40. The processor 40 in this embodiment controls the OLED 26 to project an image onto the light guide plate 24.

[0037] Figure 4 shows a block diagram illustrating an example of the configuration of the functions related to the control of projected image projection in the smartphone 12. As shown in Figure 4, the smartphone 12 comprises a processor 40, memory 42, I / F (Interface) unit 43, storage unit 44, display 46, and input device 48. The processor 40, memory 42, I / F unit 43, storage unit 44, display 46, and input device 48 are connected to each other via a bus 49, such as a system bus or control bus, enabling the exchange of various types of information.

[0038] The processor 40 reads various programs, including the projection control program 45 stored in the memory unit 44, into the memory 42 and executes processing according to the read program. In this way, the processor 40 controls the projection of the image by the OLED 26. The memory 42 is a work memory for the processor 40 to execute processing.

[0039] The memory unit 44 stores the projection control program 45, image data of the projection image projected from the OLED 26 (not shown), and various other information. Specific examples of the memory unit 44 include HDDs (Hard Disk Drives) and SSDs (Solid State Drives).

[0040] The I / F unit 43 communicates various information with the OLED 26 and the pupil sensor 27 via wireless or wired communication. The display 46 and the input device 48 function as a user interface. The display 46 provides the user with various information regarding the projection of the projected image. The display 46 is not particularly limited and includes liquid crystal monitors and LED (Light Emitting Diode) monitors, etc. The input device 48 is operated by the user to input various instructions regarding the projection of the projected image. The input device 48 is not particularly limited and includes, for example, a keyboard, a stylus, and a mouse. In the smartphone 12, a touch panel display is used that integrates the display 46 and the input device 48.

[0041] Next, the functions of the smartphone 12 of this embodiment will be described. The processor 40 of the smartphone 12 of this embodiment has a function to control the brightness of the projected image projected onto the OLED 26 as a control of the projection of the projected image. Specifically, the smartphone 12 of this embodiment controls the brightness of the projected image projected onto the OLED 26 based on the size S of the pupil 92 of the user's eye 90 that views the projected image.

[0042] Here, we will explain the relationship between the brightness of the projected image, the size of the pupil 92 of the eye 90 that views the projected image, and the strain on the eye 90. Generally, excessive brightness is considered to place a greater burden on the eye 90. For example, if the projected image is too bright relative to the surrounding environment, the burden on the eye 90 increases. On the other hand, a brighter projected image is easier to see and has higher visibility. Therefore, as in the case of using AR glasses 10, when a user sees a real image versus when a projected image is superimposed on a real image, there is a relationship between the brightness of the projected image relative to the real image and the burden on the eye 90 that sees the projected image, as shown in Figure 5. As shown in Figure 5, if the projected image is very dark relative to the real image, the visibility of the projected image is poor, but the burden on the eye 90 is small. On the other hand, if the projected image is very bright relative to the real image, the visibility of the projected image is good, but the burden on the eye 90 increases. Thus, there is an inverse relationship between the visibility of the projected image and the burden on the eye 90 that sees the projected image.

[0043] On the other hand, as described above, the size S of the pupil 92 of the eye 90 expands and contracts in accordance with the amount of light entering the eye. Generally, when the surroundings are bright, the pupil 92 becomes smaller to reduce the amount of light entering the eye 90, and when the surroundings are dark, the pupil 92 becomes larger to increase the amount of light entering the eye 90. Therefore, the smartphone 12 of this embodiment controls the brightness of the projected image according to the size of the pupil 92 of the eye 90.

[0044] In Fig. 6A, a case is shown where the size S2 of the pupil 92 of the eye 90 viewing the projected image 62 superimposed on the real image 60 is larger than the size S1 of the pupil 92 of the eye 90 viewing the real image 60 (S1 < S2). In the case shown in Fig. 6A, since the projected image 62 is relatively darker than the real image 60 and the amount of light of the projected image 62 is small, the pupil 92 of the eye 90 viewing the projected image 62 is dilated. In this case, although the burden on the eye 90 is small, the visibility of the projected image 62 is poor. Therefore, the smartphone 12 performs control to brighten the projected image 62.

[0045] Also, in Fig. 6B, a case is shown where the size S1 of the pupil 92 of the eye 90 viewing the real image 60 is the same as the size S2 of the pupil 92 of the eye 90 viewing the projected image 62 superimposed on the real image 60 (S1 = S2). Here, "the same" means that an allowable degree of error is ignored. When the projected image 62 is too dark, for example, darker than the case shown in Fig. 6A, the size of the pupil 92 of the eye 90 tends to depend on the brightness of the real image 60 and may not respond to the brightness of the projected image 62. Therefore, in the case shown in Fig. 6B, it is presumed that the projected image 62 is very dark compared to the real image 60. In this case, although the burden on the eye 90 is small, the visibility of the projected image 62 is poor. Therefore, the smartphone 12 performs control to brighten the projected image 62.

[0046] Also, in Fig. 6C, a case is shown where the size S2 of the pupil 92 of the eye 90 viewing the projected image 62 superimposed on the real image 60 is smaller than the size S1 of the pupil 92 of the eye 90 viewing the real image 60, and the difference between the size S1 and the size S2 is larger than a predetermined threshold th (S1 > S2, S1 - S2 > th). In the case shown in Fig. 6C, since the projected image 62 is relatively brighter than the real image 60 and the amount of light of the projected image 62 is large, the pupil 92 of the eye 90 viewing the projected image 62 is constricted. In this case, although the visibility of the projected image 62 is good, the burden on the eye 90 is large. Therefore, the smartphone 12 performs control to darken the projected image 62.

[0047] Furthermore, Figure 6D shows the case where the pupil size S2 of the eye 90 viewing the projected image 62 superimposed on the real image 60 is smaller than the pupil size S1 of the eye 90 viewing the real image 60, and the difference between sizes S1 and S2 is less than or equal to a predetermined threshold th (S1 > S2, S1 - S2 ≤ th). In the case shown in Figure 6D, the projected image 62 is slightly brighter than the real image 60, and the amount of light in the projected image 62 is slightly greater, so the pupil 92 of the eye 90 viewing the projected image 62 is slightly constricted. In this case, the visibility of the projected image 62 is good, and the burden on the eye 90 is small. Therefore, since the brightness of the projected image 62 is appropriate, the smartphone 12 controls the brightness of the projected image 62.

[0048] The threshold th used to determine whether the brightness of the projected image 62 is appropriate can be set to a value experimentally derived based on the relationship between the brightness of the real image 60 and the projected image 62 and the degree of eye fatigue 90.

[0049] Figure 7 shows a functional block diagram illustrating an example of the configuration of the functions of the smartphone 12 in this embodiment. As shown in Figure 4, the smartphone 12 includes a first acquisition unit 50, a second acquisition unit 52, and a projection control unit 54. As an example, in the smartphone 12 of this embodiment, the processor 40 executes a projection control program 45 stored in the storage unit 44, thereby enabling the processor 40 to function as the first acquisition unit 50, the second acquisition unit 52, and the projection control unit 54.

[0050] The first acquisition unit 50 has the function of acquiring the size S1 of the pupil 92 based on the detection result of the pupil sensor 27. Specifically, the acquisition unit 52 acquires the size S1 of the pupil 92 of the user's eye 90 when the OLED 26 is not projecting an image. In other words, the second acquisition unit 52 acquires the size S1 of the pupil 92 of the user's eye 90 when the OLED 26 is projecting an image with a brightness of "zero". The size S1 of this embodiment is an example of the first size of this disclosure. The acquisition unit 52 outputs the acquired size S1 of the pupil 92 to the projection control unit 54.

[0051] The second acquisition unit 52 has the function of acquiring the size S2 of the pupil 92 based on the detection result of the pupil sensor 27. Specifically, the acquisition unit 52 acquires the size S2 of the pupil 92 of the user's eye 90 when the OLED 26 is projecting an image. In other words, the second acquisition unit 52 acquires the size S2 of the pupil 92 when the OLED 26 is projecting an image with a brightness of "zero" or more. The size S2 of this embodiment is an example of the second size of this disclosure. The acquisition unit 52 outputs the acquired size S2 of the pupil 92 to the projection control unit 54.

[0052] The projection control unit 54 has a function to control the brightness of the projected image projected onto the right eye transmissive unit 20R according to the comparison result of the pupil size S1 and size S2 of the pupil 92. In this embodiment, as described above with reference to Figure 6A, if the pupil size S2 is larger than the pupil size S1 of the pupil 92, the projection control unit 54 controls the brightness of the projected image projected from the OLED 26 to be brighter. Also, as described above with reference to Figure 6B, if the pupil size S1 and the pupil size S2 of the pupil 92 are the same, the projection control unit 54 controls the brightness of the projected image projected from the OLED 26 to be brighter. Also, as described above with reference to Figure 6C, if the pupil size S2 of the pupil 92 is smaller than the pupil size S1 of the pupil 92, and the difference between size S1 and size S2 is greater than a threshold th, the projection control unit 54 controls the brightness of the projected image projected from the OLED 26 to be darker. Furthermore, as described above with reference to Figure 6D, the projection control unit 54 performs control to maintain the brightness of the projected image projected from the OLED 26 when the size S2 of the pupil 92 is smaller than the size S1 of the pupil 92, and the difference between the sizes S1 and S2 is less than or equal to a threshold th.

[0053] The method for controlling the brightness of the projected image 62 can be appropriately chosen depending on the device that projects the projected image 62. In this embodiment, since an OLED 26 is used as the device that projects the projected image 62, the magnitude of the drive current applied to the pixels of the OLED 26 is adjusted. To increase the brightness of the projected image 62, the magnitude of the drive current is increased, and to decrease the brightness of the projected image 62, the magnitude of the drive current is decreased.

[0054] Next, the operation of the smartphone 12 of this embodiment will be explained. Figure 8 shows a flowchart illustrating an example of the projection control processing flow by the processor 40 of the smartphone 12 of this embodiment. As an example, in the smartphone 12 of this embodiment, when the user inputs a projection start command using the input device 48, the processor 40 executes the projection control program 45 stored in the memory unit 44, thereby executing the projection control processing shown as an example in Figure 8. At the time the projection control processing shown in Figure 8 is started, the user is viewing the real image with both their left and right eyes through the left eye translucent section 20L and the right eye translucent section 20R.

[0055] In step S100 of Figure 8, the first acquisition unit 50 acquires the size S1 of the pupil 92. As described above, the first acquisition unit 50 acquires the size S1 of the pupil 92 from the image captured by the pupil sensor 27 of the user's eye 90.

[0056] In the next step S102, the projection control unit 54 projects the projection image 62. Specifically, the projection control unit 54 outputs image data of the projection image 62 to the OLED 26 so that the projection image 62 is projected onto the light guide plate 24. When the OLED 26 receives image data of the projection image 62 from the smartphone 12, it projects the projection image 62 corresponding to the input image data onto the light guide plate 24. The brightness of the projection image 62 that the projection control unit 54 projects onto the OLED 26 in the first step S102, which is executed after starting the projection control process shown in Figure 8, is not particularly limited and can be a predetermined brightness. The predetermined brightness may be, for example, the brightness of a general-purpose projection image obtained in the design of the AR glasses 10.

[0057] In the next step S104, the second acquisition unit 52 acquires the size S2 of the pupil 92. As described above, the second acquisition unit 52 acquires the size S2 of the pupil 92 from the image captured by the pupil sensor 27 of the user's eye 90.

[0058] In the next step S106, the projection control unit 54 determines whether the size S1 obtained in the above step S100 is larger than the size S2 obtained in the above step S104 (S1 > S2). If the size S1 is not larger than the size S2, in other words, if the size S2 is greater than or equal to the size S1 (S1 ≤ S2), the determination in step S106 is a negative determination, and the process proceeds to step S108.

[0059] In step S108, the projection control unit 54 determines whether the size S1 is smaller than the size S2 (S1 < S2). If the size S1 is not smaller than the size S2, in other words, if the size S1 is the same as the size S2 (S1 = S2), the determination in step S108 is a negative determination, and the process proceeds to step S110. In this case, it corresponds to the case described above with reference to FIG. 6B. Therefore, in step S110, after the projection control unit 54 performs control to brighten the brightness of the projected projection image 62, the process returns to step S102.

[0060] On the other hand, in step S108, if the size S1 is smaller than the size S2 (S1 < S2), the determination is an affirmative determination, and the process proceeds to step S112. In this case, it corresponds to the case described above with reference to FIG. 6A. Therefore, in step S112, after the projection control unit 54 performs control to brighten the brightness of the projected projection image 62, the process returns to step S102.

[0061] Also, in the above step S106, if the size S1 is larger than the size S2 (S1 > S2), the determination is an affirmative determination, and the process proceeds to step S114.

[0062] In step S114, the projection control unit 54 determines whether the difference between size S1 and size S2 is less than or equal to the threshold th ((S1-S2)≦th). If the difference between size S1 and size S2 is not less than or equal to the threshold th, in other words, if the difference between size S1 and size S2 is greater than the threshold th ((S1-S2)>th), the determination in step S114 is negative, and the process proceeds to step S116. In this case, it corresponds to the case described above, referring to Figure 6C. Therefore, in step S116, the projection control unit 54 controls the brightness of the projected image 62 to be dimmed, and then returns to step S102.

[0063] On the other hand, in step S114, if the difference between size S1 and size S2 is less than or equal to the threshold th ((S1-S2)≦th), the result is positive. In this case, it corresponds to the case described above, as shown in Figure 6D. Therefore, the projection control unit 54 maintains the brightness of the projected image 62 at its current brightness. Consequently, when the result of step S114 is positive, the projection control process shown in Figure 8 is terminated.

[0064] In this embodiment, the projection control unit 54 of the smartphone 12 controls the brightness of the projected image 62 based on a comparison between the size of the pupil 92 of the user's eye 90 (S1) when the OLED 26 is not projecting the image 62 and the size of the pupil 92 (S2) when the OLED 26 is projecting the image 62. As a result, the smartphone 12 of this embodiment can maintain good visibility of the projected image 62 and reduce the burden on the user's eye 90.

[0065] [Second Embodiment] The configuration of the AR glasses 10 in this embodiment is the same as that of the AR glasses 10 in the first embodiment (see Figures 1 and 2), so no explanation is provided. On the other hand, as shown in Figure 9, the data stored in the storage unit 44 of the smartphone 12 in this embodiment is different from that of the storage unit 44 in the first embodiment (see Figure 4). As shown in Figure 9, the storage unit 44 in this embodiment further stores reference correspondence information 41.

[0066] The reference correspondence information 41 represents the correspondence between the brightness (luminance L) of the projected image 62 being viewed and the size of the pupil 92 of the eye 90 viewing it. Figure 10 shows an example of the reference correspondence information 41. The reference correspondence information 41 shown in Figure 10 can be obtained statistically, for example. There are no limitations on the form in which the reference correspondence information 41 may take. Figure 10 shows the reference correspondence information 41 in graph (table) format, representing the correspondence between the brightness (luminance L) of the shadow image and the size S of the pupil 92, but the format of the reference correspondence information 41 is not particularly limited. For example, the reference correspondence information 41 may be expressed as an equation that represents the correspondence between the brightness (luminance L) of the projected image 62 and the size S of the pupil 92.

[0067] Furthermore, as shown in Figure 11, the smartphone 12 of this embodiment has a different configuration of projection control unit 54 compared to the projection control unit 54 of the first embodiment (see Figure 7). The projection control unit 54 of this embodiment has a function to determine the brightness of the projection image 62 projected to obtain the size S2 of the pupil 92 based on reference correspondence information 41.

[0068] Furthermore, the projection control unit 54 of this embodiment includes a reference correspondence information update unit 55. The reference correspondence information update unit 55 has the function of updating the reference correspondence information 41 based on the correspondence between the brightness of the controlled projection image 62, which is determined by the comparison result of size S1 and size S2, and size S1. In other words, the projection control unit 54 updates the reference correspondence information 41 based on the correspondence between the brightness of the projection image 62 deemed appropriate and brightness S1.

[0069] Figure 12 shows a flowchart illustrating an example of the projection control process performed by the smartphone 12 of this embodiment. As shown in Figure 12, the projection control process of this embodiment differs from the projection control process of the first embodiment (see Figure 8) in that it includes step S101 between step S100 and step S102, and also includes step S118.

[0070] In step S101 of Figure 12, the projection control unit 54 refers to the reference correspondence information 41 and derives the brightness of the corresponding projection image 62, assuming that the size S of the pupil 92 is the size S1 obtained in step S100, as the initial value L0 of the brightness of the projection image 62 projected from the OLED 26. As a result, in the process of step S102, which is the first step executed when the projection control process shown in Figure 12 is started, the projection control unit 54 causes the projection image 62 with the initial value L0 brightness to be projected onto the OLED 26.

[0071] Furthermore, as shown in Figure 12, in the projection control process of this embodiment, if a positive determination is made in step S114, the process proceeds to step S118. That is, in this embodiment, after controlling the brightness of the projected image 62 to an appropriate brightness, the process of step S118 is executed.

[0072] In step S118, the reference correspondence information update unit 55 updates the reference correspondence information 41 using the brightness Lz of the final projected image 62 after control and the size S1 obtained in step S100. The correspondence between the brightness Lz of the final projected image 62 after control and the size S1 obtained in step S100 is called the "post-control correspondence." In the following, the value representing the post-control correspondence will be simply referred to as the "post-control correspondence value." When the processing in step S118 is completed, the projection control process of this embodiment shown in Figure 12 is completed. In this embodiment, as a method for updating the reference correspondence information 41, post-control correspondence values ​​may be accumulated each time the projection control process is executed, as shown in Figure 13, and an approximate curve or approximate formula representing the correspondence may be derived from the accumulated post-control correspondence values ​​and used as the reference correspondence information 41.

[0073] As described above, the projection control unit 54 of the smartphone 12 in this embodiment uses the reference correspondence relationship information 41 to derive the initial value L0 of the brightness of the projected image 62. As a result, the smartphone 12 in this embodiment can shorten the time required to control the brightness of the projected image 62 to an appropriate level.

[0074] Furthermore, the reference correspondence relationship information update unit 55 updates the reference correspondence relationship information 41 with a post-control correspondence relationship value corresponding to the comparison result of size S1 and size S2, and the projection control unit 54 uses the updated reference correspondence relationship information 41 to derive the initial value L0 of the brightness of the projected image 62. Although there are individual differences in the change in pupil size 92, in this embodiment, reference correspondence relationship information 41 based on the optimal brightness of the projected image 62 for the user can be obtained, so an appropriate initial value L0 of the brightness of the projected image 62 can be derived according to the individual user. Accordingly, with the smartphone 12 of this embodiment, the time required to control the brightness of the projected image 62 to an appropriate brightness can be further reduced.

[0075] (Variation 1) The reference correspondence information update unit 55 may control the number of post-control correspondence values ​​included in the reference correspondence information 41 to be less than or equal to a predetermined number by sequentially removing post-control correspondence values ​​with older update timings from the reference correspondence information 41. Figure 14 shows a flowchart illustrating an example of the projection control process performed on the smartphone 12 of this modified example. As shown in Figure 14, the projection control process of this modified example differs from the projection control process of the second embodiment (see Figure 13) in that it includes steps S107A and S107B before step S118.

[0076] In step S107A of Figure 14, the reference correspondence relationship information update unit 55 determines whether the total number of accumulated post-control correspondence relationship values, including the post-control correspondence relationship value obtained by the current projection control process, is less than or equal to a predetermined number. If the total number of post-control correspondence relationship values ​​is not less than or equal to the predetermined number, in other words, if the total number of post-control correspondence relationship values ​​exceeds the predetermined number, the determination in step S107A becomes a negative determination, and the process proceeds to step S107B. In step S107B, the reference correspondence relationship information update unit 55 removes the post-control correspondence relationship value with the oldest update timing from the reference correspondence relationship information 41, and then proceeds to step S118.

[0077] On the other hand, if the total number of correspondence values ​​after control in step S107A is less than or equal to a predetermined number, a positive determination is made, and the process proceeds to step S118.

[0078] Through these processes, in this embodiment, the reference correspondence relationship information 41 can be made into an approximate curve (see Figure 13) or the like, based on a predetermined number of post-control correspondence relationship values ​​with a new update timing. As a result, the reference correspondence relationship information update unit 55 of the smartphone 12 in this embodiment can suppress the number of post-control correspondence relationship values ​​to be stored, thereby suppressing the pressure on memory capacity caused by storing the reference correspondence relationship information 41. In addition, the amount of processing required for the reference correspondence relationship information update unit 55 to derive the approximate curve can be suppressed, and the processing time can be shortened.

[0079] The predetermined number of reference correspondence information used in the reference correspondence information 41 is not particularly limited and can be set as appropriate depending on whether it is possible to derive statistically significant approximation curves, etc., the limitations of the memory capacity required for storage, and the accuracy of the initial value L0 of the brightness of the projected image 62.

[0080] (Modification 2) This modified example describes a modification of controlling the number of post-controlled correspondence values ​​included in the reference correspondence information 41. The reference correspondence information update unit 55 in this modified example controls the number of values ​​representing post-controlled correspondences included in the reference correspondence information 41 to be less than or equal to a predetermined number for each predetermined range determined according to the size of the pupil 92 of the eye 90 in the reference correspondence information 41. The predetermined number in this modified example is an example of the first predetermined number in this disclosure.

[0081] Figure 15 shows an example of the reference correspondence information 41 when the reference correspondence information update unit 55 controls the number of controlled correspondence values ​​to "3" or less for each region, which is divided into regions 1 to 7 according to the size of the pupil 92 of the eye 90.

[0082] In this way, by excluding outdated post-control correspondence values ​​from the reference correspondence information 41, it is possible to prevent the pupil size 92 from being biased towards only specific post-control correspondence values. As a result, the reference correspondence information update unit 55 of this modified example can maintain the accuracy of the reference correspondence information 41.

[0083] (Variation 3) This modified example describes a modification of controlling the number of post-control correspondence values ​​included in the reference correspondence information 41. In this modified example, the reference correspondence information update unit 55 controls the number of values ​​representing post-control correspondences included in the reference correspondence information 41 to be less than or equal to a predetermined number for each predetermined range determined according to the brightness of the visible image in the reference correspondence information 41. The predetermined number in this modified example is an example of the second predetermined number in this disclosure.

[0084] Figure 16 shows an example of the reference correspondence information 41 when the reference correspondence information update unit 55 controls the number of controlled correspondence values ​​to "2" or less for each region, which is divided into regions 1 to 9 according to the brightness (luminance) of the image being viewed.

[0085] In this way, by excluding outdated post-control correspondence values ​​from the reference correspondence information 41, it is possible to prevent the brightness of the visible image from being biased towards only specific brightness post-control correspondence values. As a result, the reference correspondence information update unit 55 of this modified example can maintain the accuracy of the reference correspondence information 41.

[0086] (Modification 4) This modified example describes a modification of controlling the number of controlled correspondence values ​​included in the reference correspondence information 41. In this modified example, the reference correspondence information update unit 55 determines a representative value using the controlled correspondence value for each predetermined range determined according to the size of the pupil 92 of the eye 90 in the reference correspondence information 41, and derives the reference correspondence information 41 using the representative value.

[0087] Figure 17 shows an example of reference correspondence information 41 when the eye 90 is divided into regions 1 to 7 according to the size of the pupil 92, and a representative value is determined for each region from the post-controlled correspondence value. In the case shown in Figure 17, if multiple post-controlled correspondence values ​​are obtained in each of regions 1 to 7, the reference correspondence information update unit 55 determines one representative value from the multiple post-controlled correspondence values.

[0088] Therefore, when the reference correspondence relationship information update unit 55 acquires a new post-control correspondence relationship value, it updates the representative value by determining a new representative value from multiple post-control correspondence relationship values, including the new post-control correspondence relationship value. Figure 17 shows the case when the reference correspondence relationship information update unit 55 determines a new representative value after acquiring a new post-control correspondence relationship value included in region 4.

[0089] Furthermore, when determining representative values, it is preferable to weight the post-control correspondence relationship values ​​to determine the new representative values. For example, the reference correspondence relationship information update unit 55 calculates the new representative values ​​using the following equation (1). The weight W may be set in advance or changed as appropriate. For example, by making the value of W smaller, when determining the representative values, post-control correspondence relationship values ​​with older update timings will be given more weight, so it is possible to determine new representative values ​​that prioritize data continuity. Dn_new=W×D+Dn_old×(1-W) ···(1) Dn_new: New representative values ​​for region n (in Figure 17, n=1~7) Dn_old: Old representative value of region n D: Corresponding relationship value after new control W: Weight of the correspondence value after new control, W = 0 to 1

[0090] Furthermore, the reference correspondence information update unit 55 derives an approximate curve using representative values. If a new representative value is determined, the reference correspondence information update unit 55 re-derives the approximate curve based on the new representative value. The example shown in Figure 17 shows the case where a new approximate curve is derived based on the new representative value determined in region 4.

[0091] In this way, by using representative values ​​determined from post-control correspondence values ​​rather than the post-control correspondence values ​​themselves, it is possible to maintain the accuracy of the reference correspondence information 41 while reducing the processing required to derive the reference correspondence information 41 (in the example shown in Figure 17, the approximate curve).

[0092] In the above example, the reference correspondence information update unit 55 determined representative values ​​using controlled correspondence values ​​for each predetermined range determined according to the size of the pupil 92 of the eye 90 in the reference correspondence information 41. However, the method for determining representative values ​​is not limited to this embodiment. For example, the reference correspondence information update unit 55 may determine representative values ​​using controlled correspondence values ​​for each predetermined range determined according to the brightness of the image being viewed in the reference correspondence information 41. Alternatively, for example, the reference correspondence information update unit 55 may determine representative values ​​using controlled correspondence values ​​for each predetermined range determined according to the size of the pupil 92 of the eye 90 and the brightness of the image being viewed in the reference correspondence information 41.

[0093] (Variation 5) In each of the above embodiments, the initial stage of executing the projection control process involves using statistically obtained reference correspondence information 41 and updating the statistically obtained reference correspondence information 41. In this modified example, instead of the statistically obtained reference correspondence information 41, information representing the correspondence between the brightness of the projected image 62 and the size S1 of the pupil 92 of the user's eye 90, obtained by projecting projected images 62 with different brightness levels, is used.

[0094] The size of the pupil 92 varies from person to person, and the degree of change differs depending on factors such as aging. Therefore, in this modified example, reference correspondence information 41 specific to the individual user using the AR glasses 10 is used.

[0095] In this modified example, at predetermined timings, such as when the AR glasses 10 are put into use or before projection control processing is performed, the reference correspondence relationship information update unit 55 sequentially projects multiple projection images 62 with different brightness levels (luminance) using the OLED 26. The first acquisition unit 50 acquires the size S of the pupil 92 of the user's eye 90 that has viewed each projection image 62, and the reference correspondence relationship information update unit 55 derives reference correspondence relationship information 41 based on the size S of the pupil 92 of the user's eye 90 that has viewed each projection image 62. For example, in the example shown in Figure 18, the brightness is dark (1 cd / m²). 2 )From projection image 621, bright (10000 cd / m²) 2 This example shows how reference correspondence information 41 was derived from the size S obtained by sequentially projecting nine projection images 62 (621-629) up to projection image 629.

[0096] The size of the pupil 92 varies from person to person, and the degree of change differs depending on factors such as aging. Therefore, in this modified example, as described above, reference correspondence information 41 obtained from the relationship between the brightness of the projected image 62 actually viewed by the user using the AR glasses 10 and the size S of the pupil 92 is used. In this way, in this modified example, a reference correspondence information 41 optimized for each individual user is used, making it possible to obtain an initial value L0 with better accuracy.

[0097] [Third Embodiment] Generally, different colors have different levels of brightness perception, i.e., different spectral sensitivity characteristics. Specifically, the way the size of the pupil 92 changes differs depending on the color of the projected image 62 being viewed. Therefore, it is preferable to perform projection control processing using a projected image 62 in which the color has been adjusted to suit the user by weighting according to the spectral sensitivity characteristics of the user's pupil 92 for each color used in the projected image 62.

[0098] Figure 19 shows a flowchart illustrating an example of the weight acquisition process performed by the processor 40 of the smartphone 12 to obtain weights corresponding to the high-sensitivity characteristics. In this embodiment, the smartphone 12 performs the weight acquisition process shown in Figure 19 at predetermined timings, such as when the user first uses the AR glasses 10 or after a predetermined period of use has elapsed. In this embodiment, the case where the projected image 62 is an RGB color image is described as an example. Furthermore, it is preferable to perform the weight acquisition process in a dark environment in order to minimize the influence of the surrounding environment.

[0099] In step S10 of Figure 19, the projection control unit 54 projects a monochromatic red projection image 62. Specifically, the projection control unit 54 outputs image data of the projection image 62 to the OLED 26 so that the monochromatic red projection image 62 is projected onto the light guide plate 24. When the OLED 26 receives image data of the monochromatic red projection image 62 from the smartphone 12, it projects the monochromatic red projection image 62 corresponding to the input image data onto the light guide plate 24.

[0100] In the next step S12, the second acquisition unit 52 acquires the size S_r of the pupil 92. As described above, the second acquisition unit 52 acquires the size S_r of the pupil 92 from the image captured by the pupil sensor 27 of the user's eye 90.

[0101] In the next step S14, the projection control unit 54 obtains a luminance value corresponding to the size S_r obtained in step S12. The projection control unit 54 in this embodiment refers to the reference correspondence information 41 (see Figure 10, etc.) described above and obtains the luminance value corresponding to the size S_r as the luminance value L_r of the R monochromatic projection image 62.

[0102] In the next step S16, the projection control unit 54 projects a single-color G projection image 62. Specifically, the projection control unit 54 outputs image data of the projection image 62 to the OLED 26 so that the single-color G projection image 62 is projected onto the light guide plate 24. When the OLED 26 receives image data of the single-color G projection image 62 from the smartphone 12, it projects the single-color G projection image 62 corresponding to the input image data onto the light guide plate 24.

[0103] In the next step S18, the second acquisition unit 52 acquires the size S_r of the pupil 92. As described above, the second acquisition unit 52 acquires the size S_g of the pupil 92 from the image captured by the pupil sensor 27 of the user's eye 90.

[0104] In the next step S20, the projection control unit 54 obtains a luminance value corresponding to the size S_g obtained in step S18. The projection control unit 54 in this embodiment refers to the reference correspondence information 41 (see Figure 10, etc.) described above and obtains the luminance value corresponding to the size S_g as the luminance value L_g of the G monochromatic projection image 62.

[0105] In the next step S22, the projection control unit 54 projects a B-color monochromatic projection image 62. Specifically, the projection control unit 54 outputs image data of the projection image 62 to the OLED 26 so that the B-color monochromatic projection image 62 is projected onto the light guide plate 24. When the OLED 26 receives image data of the B-color monochromatic projection image 62 from the smartphone 12, it projects the B-color monochromatic projection image 62 corresponding to the input image data onto the light guide plate 24.

[0106] In the next step S24, the second acquisition unit 52 acquires the size S_b of the pupil 92. As described above, the second acquisition unit 52 acquires the size S_b of the pupil 92 from the image captured by the pupil sensor 27 of the user's eye 90.

[0107] In the next step S26, the projection control unit 54 acquires a luminance value corresponding to the size S_b acquired in step S20. The projection control unit 54 in this embodiment refers to the reference correspondence information 41 (see Figure 10, etc.) described above and acquires the luminance value corresponding to the size S_b as the luminance value L_b of the B-color monochromatic projection image 62.

[0108] In the next step S28, the projection control unit 54 derives the weights W_r corresponding to the R color, W_g corresponding to the G color, and W_b corresponding to the B color, and stores them in the storage unit 44. The projection control unit 54 derives the weights W_r corresponding to the R color, W_g corresponding to the G color, and W_b corresponding to the B color by normalizing the luminance value L_r obtained in step S14, the luminance value L_g obtained in step S20, and the luminance value L_b obtained in step S26. As an example, the projection control unit 54 of this embodiment derives the weights W_r, W_g, and W_b using the following equations (2) to (4). W_r=(L_r) / (L_r+L_g+L_b) ···(2) W_g=(L_g) / (L_r+L_g+L_b) ···(3) W_b=(L_b) / (L_r+L_g+L_b) ···(4) Note that W_r + W_g + W_b = 1.

[0109] According to equations (2) to (4) above, for example, if the spectral characteristics make the user perceive red as brighter, the value of weight W_r will be greater than the respective values ​​of weights W_g and W_b.

[0110] Once the process in step S28 is completed, the weight acquisition process shown in Figure 19 is finished.

[0111] In the smartphone 12 of this embodiment, projection control processing is performed using a projected image 62 whose color tone has been adjusted to suit the user by weighting W_r, W_g, and W_b according to the spectral sensitivity characteristics of the user's pupil 92 obtained in this way.

[0112] As a result, the smartphone 12 of this embodiment performs projection control processing using a projected image 62 that is adjusted so that colors that the user perceives as bright are darker, and colors that the user perceives as less bright are brighter. As a result, the smartphone 12 of this embodiment can control the brightness of the projected image 62 to a more appropriate level.

[0113] As described above, the smartphone 12 of this embodiment includes a processor 40 that controls the projection of the projected image 62 to the AR glasses 10, which are equipped with a right-eye transparent section 20R that provides a real image to the user's eye 90, and an OLED 26 that projects a projected image 62 onto the right-eye transparent section 20R in a state that is visible to the user's eye 90. The processor 40 obtains the size S1 of the pupil 92 of the user's eye 90 when the OLED 26 projects the projected image 62 at a first brightness, such as brightness "zero". The processor 40 also obtains the size S2 of the pupil 92 when the OLED 26 projects the projected image 62 at a second brightness that is brighter than the first brightness. Furthermore, the processor 40 controls the brightness of the projected image 62 projected onto the right-eye OLED 26R according to the comparison result of size S1 and size S2.

[0114] Thus, each of the above forms of smartphone 12 controls the brightness of the projected image 62 to an appropriate level according to the comparison result between the size S1 and size S2 of the user's pupil 92. Therefore, each of the above forms of smartphone 12 can maintain good visibility of the projected image 62 and reduce the strain on the user's eyes 90.

[0115] In the above configurations, when obtaining the size S1 of the user's pupil 92, the brightness of the projected image 62 was set to "zero" and the projected image 62 was not displayed. However, the configuration may also include the display of the projected image 62. Furthermore, although the above configurations described the case where the projected image is a still image, the projected image may also be a moving image.

[0116] Furthermore, the AR glasses 10 may have a dimming function to adjust the transmittance of the real image 60. For example, if the surrounding environment is too bright and the size S1 of the pupil 92 is too small, the dimming function may be used to adjust the transmittance of the real image 60 so that the size S1 is greater than or equal to a predetermined size. In each of the above embodiments, the transmittance of the right eye's transparent section 20R may be reduced. By adjusting the transmittance of the real image 60 in this way, the upper limit for the brightness of the projected image 62 can be kept low. This helps to suppress increases in power consumption, as well as increases in size and weight of the OLED 26.

[0117] Furthermore, although the above embodiment describes an AR glasses 10 in which the projected image 62 is viewed with the user's right eye 90, the AR glasses 10 are not limited to this embodiment and may be an AR glasses 10 in which the projected image 62 is viewed with the user's left eye 90. Alternatively, the AR glasses 10 may be an AR glasses 10 in which the projected image 62 is viewed with each of the user's left and right eyes 90. In this case, it is preferable that the smartphone 12 performs projection control processing for each of the left and right eyes 90.

[0118] Furthermore, although the second embodiment described above describes a configuration in which the reference correspondence relationship information 41 is stored in the memory unit 44 of the smartphone 12, the location where the reference correspondence relationship information 41 is stored is not limited to the smartphone 12. For example, the AR glasses 10 may store the reference correspondence relationship information 41.

[0119] Furthermore, in each of the above embodiments, the smartphone 12 is equipped with the functions of the first acquisition unit 50, the second acquisition unit 52, and the projection control unit 54. However, some or all of the functions of these units may be equipped with other devices. Other devices could be, for example, AR glasses 10 or a server computer located in the cloud.

[0120] Furthermore, in each of the above embodiments, the hardware structure of the processing unit that executes various processes, such as the first acquisition unit 50, the second acquisition unit 52, and the projection control unit 54, can be the various processors shown below. As mentioned above, these various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as programmable logic devices (PLDs), such as FPGAs (Field Programmable Gate Arrays), which are processors whose circuit configuration can be changed after manufacturing, and dedicated electrical circuits, such as ASICs (Application Specific Integrated Circuits), which are processors with circuit configurations specifically designed to execute specific processes.

[0121] A single processing unit may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, multiple processing units may be composed of a single processor.

[0122] Examples of configuring multiple processing units with a single processor include, firstly, a configuration where one or more CPUs and software combine to form a single processor, as exemplified by client and server computers, and this processor functions as multiple processing units. Secondly, a configuration using a processor that realizes the functions of the entire system, including multiple processing units, on a single IC (Integrated Circuit) chip, as exemplified by System-on-a-Chip (SoC). Thus, various processing units are configured, in terms of hardware structure, using one or more of the above-mentioned processors.

[0123] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits, which are combinations of circuit elements such as semiconductor devices.

[0124] Furthermore, although the above embodiments describe a configuration in which the projection control program 45 is pre-stored (installed) in the storage unit 44, the invention is not limited to this configuration. The projection control program 45 may be provided in the form of a recording medium such as a CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory), or USB (Universal Serial Bus) memory. Alternatively, the projection control program 45 may be provided in the form of a download from an external device via a network. [Explanation of symbols]

[0125] 1 Display System 10 AR Glasses 12 Smartphones 20L transparent section for left eye, 20R transparent section for right eye 22L lens for the left eye, 22R lens for the right eye 24 Light guide plate 26 OLED 27 Pupil sensor 40 processors 41. Information related to compliance with standards 42 memory 43 I / F section 44 Memory section 45 Projection control program 46 displays 48 Input devices 49 bus 50 First acquisition part 52 Second acquisition part 54 Projection Control Unit 55. Section for updating information related to compliance with standards. 60 Reality 62, 621-629 Projection Images 90 eyes 92 Pupil S, S1, S2 (pupil size)

Claims

1. A control device for a spectacle-type display device comprising a transparent section that provides a real image to the user's eyes, and a projection section that projects a projected image onto the transparent section in a state that is visible to the user's eyes, the control device comprising a processor for controlling the projection of the projected image, The aforementioned processor, Regarding the size of the pupil of the user's eye, the projection unit obtains the first size of the pupil when it projects the projection image at a first brightness. The second size of the pupil is obtained when the projection unit projects the projection image at a second brightness that is brighter than the first brightness. The brightness of the projected image projected onto the transparent portion is controlled according to the comparison result between the first size and the second size. If the first size is the same as the second size, control is performed to brighten the projected image. Control device.

2. The aforementioned processor, If the first size is larger than the second size, and the difference between the first size and the second size is less than or equal to a predetermined threshold, control is performed to maintain the brightness of the projected image. The control device according to claim 1.

3. The aforementioned processor, If the first size is larger than the second size, and the difference between the first size and the second size is greater than a predetermined threshold, control is performed to darken the projected image. The control device according to claim 1.

4. The aforementioned processor, If the first size is smaller than the second size, control is performed to brighten the projected image. The control device according to any one of claims 1 to 3.

5. The aforementioned processor, Referencing reference correspondence information that shows the correspondence between the brightness of the perceived image and the size of the pupil of the eye that is viewing it, To obtain the second size mentioned above, the brightness of the projected image is used. The first size is determined based on the brightness of the image being viewed, assuming that the size of the pupil of the eye being viewed is the size of the pupil of the eye being viewed. The control device according to any one of claims 1 to 4.

6. The aforementioned processor, As the aforementioned reference correspondence information, information representing the correspondence between the brightness of the projected image and the size of the user's pupil, obtained by projecting the projected image with varying brightness levels, is used. The control device according to claim 5.

7. The aforementioned processor, The reference correspondence information is updated using a value representing the controlled correspondence, where the brightness of the projected image after control, based on the comparison result between the first size and the second size, is defined as the brightness of the image being viewed, and the first size is defined as the size of the pupil of the eye being viewed. The control device according to claim 5 or claim 6.

8. The aforementioned processor, By sequentially removing values ​​representing the post-control correspondence that have an older update timing from the standard correspondence information, the number of values ​​representing the post-control correspondence included in the standard correspondence information is controlled to be less than or equal to a predetermined number. The control device according to claim 7.

9. The aforementioned processor, For each predetermined range determined according to the size of the pupil of the eye being viewed in the aforementioned reference correspondence information, the number of values ​​representing the controlled correspondence included in the aforementioned reference correspondence information is controlled to be less than or equal to a first predetermined number. The control device according to claim 7 or claim 8.

10. The aforementioned processor, For each predetermined range determined according to the brightness of the visible image in the aforementioned reference correspondence information, the number of values ​​representing the controlled correspondence included in the reference correspondence information is controlled to be less than or equal to a second predetermined number. The control device according to claim 7 or claim 8.

11. The aforementioned processor, For each predetermined range determined according to the size of the pupil of the eye being viewed in the reference correspondence information, for each predetermined range determined according to the brightness of the image being viewed in the reference correspondence information, or for each predetermined range determined according to the size of the pupil of the eye being viewed and the brightness of the image being viewed in the reference correspondence information, a representative value representing the correspondence between the brightness of the image being viewed and the size of the pupil of the eye being viewed is determined. The aforementioned representative values ​​are used to derive the aforementioned reference correspondence information. The control device according to claim 5 or claim 6.

12. The aforementioned processor, The representative value is updated using a value representing the controlled correspondence, where the brightness of the projected image after control, based on the comparison result between the first size and the second size, is defined as the brightness of the image being viewed, and the first size is defined as the size of the pupil of the eye being viewed. The control device according to claim 11.

13. The aforementioned processor, The representative value is updated by weighting the value representing the post-control correspondence mentioned above. The control device according to claim 12.

14. A control method by a processor of a control device equipped with a processor for controlling the projection of a projected image, wherein the device comprises a transparent section that provides a real image to the user's eyes, and a projection section that projects a projected image onto the transparent section in a state that is visible to the user's eyes, Regarding the size of the pupil of the user's eye, the projection unit obtains the first size of the pupil when it projects the projection image at a first brightness. The second size of the pupil is obtained when the projection unit projects the projection image at a second brightness that is brighter than the first brightness. The brightness of the projected image projected onto the transparent portion is controlled according to the comparison result between the first size and the second size. If the first size is the same as the second size, control is performed to brighten the projected image. Control method.

15. If the first size is larger than the second size, and the difference between the first size and the second size is less than or equal to a predetermined threshold, control is performed to maintain the brightness of the projected image. The control method according to claim 14.

16. A control program is executed by the processor of a control device equipped with a processor for controlling the projection of a projected image, for a spectacle-type display device comprising a transparent section that provides a real image to the user's eyes and a projection section that projects a projected image onto the transparent section in a state that is visible to the user's eyes. Regarding the size of the pupil of the user's eye, the projection unit obtains the first size of the pupil when it projects the projection image at a first brightness. The second size of the pupil is obtained when the projection unit projects the projection image at a second brightness that is brighter than the first brightness. The brightness of the projected image projected onto the transparent portion is controlled according to the comparison result between the first size and the second size. If the first size is the same as the second size, control is performed to brighten the projected image. Control program.

17. If the first size is larger than the second size, and the difference between the first size and the second size is less than or equal to a predetermined threshold, control is performed to maintain the brightness of the projected image. The control program according to claim 16.

Images