Control device, control method, and control program

Abstract

To provide a control device, a control method, and a control program with which it is possible to provide a first image suitable for observation by a first user and a second image suitable for observation by a second user to each.SOLUTION: Provided is a control device that includes at least one processor. When a first user observes a captured image that is captured by a camera of an eyeglass type display device and a second user observes the captured image with a display device that is different from the eyeglass type display device, the processor exercises control to output a first image in which imaging is controlled based on a first region in the captured image, to the eyeglass type display device and output a second image in which imaging is controlled based on a second region in the captured image, to the eyeglass type display device.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] The present disclosure relates to a control device, a control method, and a control program.

Background Art

[0002] Conventionally, there has been known a technique in which an image observed by a specific user using a glasses-type display device can also be observed by a user other than the specific user. For example, Patent Document 1 describes a technique for sharing content within virtual reality content visually recognized by a player wearing a head-mounted display with other viewers. In the technique described in Patent Document 1, the image provided to the viewer is scaled up or down according to the position of the viewer.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] According to the technique described in Patent Document 1, although the size of the image displayed according to the position of the viewer becomes an appropriate size, there are cases where it cannot be said that the image is appropriate for the viewer to observe.

[0005] The present disclosure has been made in view of the above circumstances, and an object thereof is to provide a control device, a control method, and a control program capable of providing a first image suitable for observation by a first user and a second image suitable for observation by a second user, respectively.

Means for Solving the Problems

[0006] To achieve the above objective, a control device according to a first aspect of the present disclosure includes at least one processor, which controls the output of a first image obtained by controlling the shooting based on a first region in the captured image to the glasses-type display device and a second image obtained by controlling the shooting based on a second region in the captured image to the display device, when a first user observes the captured image with the glasses-type display device and a second user observes the captured image with a display device different from the glasses-type display device.

[0007] A control device in a second aspect of the present disclosure, in the control device of the first aspect, wherein the processor performs, as a shooting control, shooting of a subject in a first area and shooting of a subject in a second area. Both types of filming The system controls the camera to capture a single image with a dynamic range suitable for the conditions, and outputs a first image and a second image corresponding to the captured image.

[0008] A control device according to a third aspect of the present disclosure, in the control device according to a second aspect, has a processor that generates a first image by adjusting the brightness value of one captured image for a spectacle-type display device, and generates a second image by adjusting the brightness value of one captured image for a display device.

[0009] A control device according to a fourth aspect of the present disclosure, in the control device according to a first aspect, the processor sets the focus position of the camera's optical system between a first focus position in which the subject in the first region is in focus and a second focus position in which the subject in the second region is in focus, as part of the shooting control. The subject in the first region and the subject in the second region are included in the depth of field. The system controls the camera to capture a single image with the focus position set, and then controls it to output a first image and a second image corresponding to the captured image.

[0010] A control device according to a fifth aspect of the present disclosure, in the control device according to the first aspect, the processor controls the exposure of the camera based on a subject in a first region to capture a first image, and controls the exposure of the camera based on a subject in a second region to capture a second image.

[0011] A control device according to a sixth aspect of the present disclosure is a control device according to any one of the first to fifth aspects, wherein the second image is a partial image obtained by cutting out a second region from a captured image.

[0012] A control device according to a seventh aspect of the present disclosure, in a control device according to any one of the first to sixth aspects, the processor determines a first region based on the gaze of a first user detected by a gaze detection device.

[0013] To achieve the above objective, the control method of the eighth aspect of the present disclosure is a control method by a processor provided in a control device, wherein when a first user observes an image captured by a camera of a glasses-type display device using the glasses-type display device, and a second user observes the captured image using a display device different from the glasses-type display device, the method outputs a first image obtained by controlling the capture based on a first region in the captured image to the glasses-type display device, and outputs a second image obtained by controlling the capture based on a second region in the captured image to the display device.

[0014] To achieve the above objective, the control program of the ninth aspect of this disclosure is an image processing program executed by a processor in a control device, which controls the output of a first image obtained by controlling the shooting based on a first region in the captured image to the glasses-type display device and a second image obtained by controlling the shooting based on a second region in the captured image to the display device, when a first user observes the captured image with the glasses-type display device and a second user observes the captured image with a display device different from the glasses-type display device. [Effects of the Invention]

[0015] According to this disclosure, a first image suitable for observation by a first user and a second image suitable for observation by a second user can be provided, respectively. [Brief explanation of the drawing]

[0016] [Figure 1] It is a configuration diagram showing an example of the configuration of the image display system of the embodiment. [Figure 2] It is a configuration diagram showing an example of the configurations of the AR glasses and the smartphone. [Figure 3] It is a perspective view showing an example of the AR glasses of the embodiment. [Figure 4] It is a diagram for explaining a photographed image viewed through the AR glasses. [Figure 5] It is a block diagram showing an example of the hardware configuration of the smartphone of the embodiment. [Figure 6] It is a diagram for explaining the viewing of the real world by the display of the AR glasses and the display of the image display device. [Figure 7] It is a block diagram showing an example of the configuration of the smartphone of the embodiment. [Figure 8] It is a diagram for explaining the first image and the second image of the first embodiment. [Figure 9] It is a block diagram showing an example of the hardware configuration of the image display device of the embodiment. [Figure 10] It is a block diagram showing an example of the configuration of the image display device of the embodiment. [Figure 11] It is a flowchart showing an example of the control process executed by the smartphone of the first embodiment and the display control process executed by the image display device. [Figure 12] It is a diagram for explaining the first image and the second image of the second embodiment. [Figure 13] It is a flowchart showing an example of the control process executed by the smartphone of the second embodiment and the display control process executed by the image display device. [Figure 14] It is a diagram for explaining the first image and the second image of the third embodiment. [Figure 15] It is a flowchart showing an example of the control process executed by the smartphone of the third embodiment and the display control process executed by the image display device.

Mode for Carrying Out the Invention

[0017] Hereinafter, with reference to the drawings, examples of embodiments for carrying out the technology of this disclosure will be described in detail.

[0018] [First Embodiment] Referring to Figure 1, the configuration of the image display system 1 of this embodiment will be described. As shown in Figure 1, the image display system 1 of this embodiment comprises AR (Augmented Reality) glasses 10, a smartphone 12, and an image display device 14. The smartphone 12 and the image display device 14 are connected via a network 19 by wired or wireless communication. The AR glasses 10 of this embodiment are an example of the glasses-type display device of this disclosure, and the smartphone 12 of this embodiment is an example of the control device of this disclosure. Furthermore, the image display device 14 of this embodiment is an example of a display device different from the glasses-type display device of this disclosure.

[0019] The image display system 1 of this embodiment has the function of displaying images captured by the camera 27 of the AR glasses 10 of the real world on the display 20 of the AR glasses 10 and the display 56 of the image display device 14. In other words, according to the image display system 1 of this embodiment, the user of the image display device 14 can also see the real world that the user of the AR glasses 10 sees. Hereinafter, the user using the AR glasses 10 will be referred to as the "first user," the user using the image display device 14 will be referred to as the "second user," and when referring to both together without distinction, they will simply be referred to as "user."

[0020] Referring to Figure 2, the configuration of the AR glasses 10 and smartphone 12 in this embodiment will be described. The AR glasses 10 is a device that enables the user to view images corresponding to each projected image by projecting a left-eye projection image from an OLED (Organic Light Emitting Diode) 26L onto a left-eye lens 22L, and projecting a right-eye projection image from a right-eye OLED 26R onto a right-eye lens 22R. In the following, the "left-eye projection image" and the "right-eye projection image" will be collectively referred to as the "projected image".

[0021] Figure 3 shows a perspective view of an example of the AR glasses 10 of this embodiment. As shown in Figures 2 and 3, the AR glasses 10 comprises a display 20, a left-eye OLED 26L, a right-eye OLED 26R, a camera 27, and an eye-tracking sensor 28. The display 20 also includes a left-eye lens 22L and a left-eye light guide plate 24L, and a right-eye lens 22R and a right-eye light guide plate 24R.

[0022] Light corresponding to the left-eye projection image projected from the left-eye OLED 26L is incident on one end of the left-eye light guide plate 24L. The light propagating through the left-eye light guide plate 24L changes direction at the output section (not shown) and is emitted towards the left eye of the first user. The light corresponding to the left-eye projection image emitted from the left-eye light guide plate 24L passes through the left-eye lens 22L and is guided to the left eye of the first user. Similarly, light corresponding to the right-eye projection image projected from the right-eye OLED 26R is incident on one end of the right-eye light guide plate 24R. The light propagating through the right-eye light guide plate 24R changes direction at the output section (not shown) and is emitted towards the right eye of the first user. The light corresponding to the right-eye projection image emitted from the right-eye light guide plate 24R passes through the right-eye lens 22R and is guided to the right eye of the first user. Therefore, while projected images are being projected from the left-eye OLED26L and the right-eye OLED26R, the first user's left and right eyes can see the corresponding display images, such as images captured by the camera 27, for each projected image.

[0023] As described above, the left-eye OLED 26L and the right-eye OLED 26R each project an image corresponding to the display image shown on the display 20 onto the right-eye light guide plate 24R and the right-eye OLED 26R. The first user views the left-eye projection image projected from the left-eye OLED 26L onto the left-eye light guide plate 24L with their left eye, and the right-eye projection image projected from the right-eye OLED 26R onto the right-eye light guide plate 24R with their right eye. As a result, the displayed images are synthesized in the first user's brain, and they are in a state of viewing the displayed image. For example, as shown in Figure 4, when displaying a captured image 101 of the real world 100 captured by a camera 27 on the display 20 as a display image, the left-eye OLED 26L projects the left portion of the captured image 101 as a left-eye projection image 101L onto the left-eye light guide plate 24L, and the right-eye OLED 26R projects the right portion of the captured image 101 as a right-eye projection image 101R onto the right-eye light guide plate 24R. The first user can view the captured image 101 by viewing the left-eye projection image 101L projected onto the left-eye light guide plate 24L with their left eye, and the right-eye projection image 101R projected onto the right-eye light guide plate 24R with their right eye.

[0024] Camera 27 is a camera that captures the real world observed by the first user. Examples of camera 27 include digital cameras such as CMOS (Complementary Metal Oxide Semiconductor) cameras. In this embodiment, camera 27 is capable of capturing color images in order to provide the user with an image representing the real world. Furthermore, camera 27 in this embodiment is a camera using a so-called fisheye lens or wide-angle lens, capable of capturing an area equivalent to the field of view of the first user. While in this embodiment the shooting range of camera 27 is equivalent to the field of view of the first user, this embodiment is not limited to this configuration; it may be a wider or narrower area than the field of view of the first user. It is preferable that the shooting range of camera 27 be equal to or greater than the field of view of the first user in order to at least provide the first user with the ability to see the real world. Image data of the captured image taken by camera 27 is output to the smartphone 12. In this embodiment, "captured image" refers to the image captured by camera 27.

[0025] The gaze detection sensor 28 is a sensor that detects the gaze of the first user. Known sensors can be used as the gaze detection sensor 28, such as a sensor that detects the direction of the first user's gaze based on the position of the iris or pupil. For example, the AR glasses 10 of this embodiment detect the gaze of the first user's right eye. The detection result of the gaze detection sensor 28 is output to the smartphone 12.

[0026] On the other hand, the smartphone 12 is equipped with a processor 40 that controls the AR glasses 10. Figure 5 shows a block diagram illustrating an example of the configuration of the functions of the smartphone 12 in this embodiment. As shown in Figure 5, the smartphone 12 is equipped with a processor 40, memory 42, I / F (Interface) unit 43, storage unit 44, display 46, and input device 48. The processor 40, memory 42, I / F unit 43, storage unit 44, display 46, and input device 48 are connected to each other via a bus 49 such as a system bus or control bus, enabling the exchange of various types of information.

[0027] The processor 40 reads various programs, including the control program 45 stored in the memory unit 44, into the memory 42 and executes processing according to the read program. The memory 42 is a work memory for which the processor 40 executes processing.

[0028] The memory unit 44 stores the control program 45, various image data (not shown) such as image data of images captured by the camera 27, and other various information. Specific examples of the memory unit 44 include HDDs (Hard Disk Drives) and SSDs (Solid State Drives).

[0029] The I / F unit 43 communicates various information with the left-eye OLED 26L, the right-eye OLED 26R, the camera 27, the gaze detection sensor 28, and the image display device 14 via wireless or wired communication. The display 46 and the input device 48 function as a user interface. The display 46 provides the user with various information regarding the projection of the projected image. The display 46 is not particularly limited and includes liquid crystal monitors and LED (Light Emitting Diode) monitors, etc. The input device 48 is operated by the user to input various instructions. The input device 48 is not particularly limited and includes, for example, a keyboard, a stylus, and a mouse. In the smartphone 12, a touch panel display is used that integrates the display 46 and the input device 48.

[0030] Next, the functions of the smartphone 12 of this embodiment will be described. In this embodiment, when a first user observes an image captured by the camera 27 using the AR glasses 10 and a second user observes the same image using the image display device 14, the smartphone 12 controls the shooting process based on a first region within the captured image and outputs the resulting first image to the AR glasses 10. The smartphone 12 also controls the shooting process based on a second region within the captured image and outputs the resulting second image to the image display device 14.

[0031] Now, referring to Figure 6, we will explain the first image 120 that the smartphone 12 outputs to the AR glasses 10 and the second image 140 that the image display device 14 outputs. Figure 6 shows a case where the first user is using the AR glasses 10 to observe the real world 100 and is fixating on an object 102 that exists in the real world 100.

[0032] The smartphone 12 acquires a captured image 110 of the real world 100 using its camera 27. The image display device 14 acquires the captured image 110 from the smartphone 12 and displays it on the display 56. This allows the second user to observe the real world 100 through the captured image 110. Figure 6 shows a case where the second user observes a subject 104 in the real world 100 by fixating on a subject image 105 in the captured image 110.

[0033] In the real world, subjects 102 and 104 exist in a specific location, and depending on factors such as the position of the light source, the way in which subject image 103 (corresponding to subject 102) and subject image 105 (corresponding to subject 104) appear in the captured image 110 may differ. For example, in some cases, the captured image 110 may have appropriate brightness and focus for only one of subject images 103 or 105.

[0034] Therefore, the smartphone 12 of this embodiment controls the camera 27 to take a picture based on a first region 112 containing the subject image 103 in the captured image 110, thereby obtaining a first image 120 with the image of the first region 112 appropriately adjusted and outputting it to the AR glasses 10. The first image 120 is displayed on the display 20 of the AR glasses 10 and viewed by the first user. The smartphone 12 also controls the camera 27 to take a picture based on a second region 114 containing the subject image 105 in the captured image 110, thereby obtaining a second image 140 with the image of the second region 114 appropriately adjusted and outputting it to the image display device 14. The second image 140 is displayed on the display 56 of the image display device 14 and viewed by the second user.

[0035] As a result, the smartphone 12 of this embodiment can provide a first image 120 suitable for the first user's viewing to a first user using the AR glasses 10, and a second image 140 suitable for the second user's viewing to a second user using the image display device 14.

[0036] Figure 7 shows a functional block diagram representing an example of the configuration of the functions of the smartphone 12 of this embodiment. As shown in Figure 7, the smartphone 12 includes a shooting control unit 60, a captured image acquisition unit 62, a display control unit 64, an image transmission unit 66, a gaze detection unit 68, a first area determination unit 70, a second area information receiving unit 72, a second area determination unit 74, a shooting condition determination unit 76, a first image generation unit 78, and a second image generation unit 80. As an example, in the smartphone 12 of this embodiment, the processor 40 executes a control program 45 stored in the storage unit 44, so that the processor 40 functions as the shooting control unit 60, the captured image acquisition unit 62, the display control unit 64, the image transmission unit 66, the gaze detection unit 68, the first area determination unit 70, the second area information receiving unit 72, the second area determination unit 74, the shooting condition determination unit 76, the first image generation unit 78, and the second image generation unit 80.

[0037] The shooting control unit 60 has the function of controlling the capture of images by the camera 27. While the first user is observing the real world 100 through the image displayed on the display 20 of the AR glasses 10, the shooting control unit 60 controls the camera 27 to capture multiple frames of images as a moving image.

[0038] The image acquisition unit 62 has the function of acquiring images captured by the camera 27. Specifically, the image acquisition unit 62 acquires image data representing the captured image 110 from the camera 27 via the I / F unit 43. The image acquisition unit 62 outputs the acquired captured image 110 to the display control unit 64, the image transmission unit 66, the first image generation unit 78, and the second image generation unit 80 as appropriate.

[0039] The display control unit 64 has the function of controlling the display of the captured image 110 or the first image 120 on the display 20 of the AR glasses 10.

[0040] The image transmission unit 66 has the function of transmitting the captured image 110 and the second image 140 to the image display device 14. Specifically, the image transmission unit 66 transmits image data representing the captured image 110 acquired by the captured image acquisition unit 62 to the image display device 14 via the I / F unit 43. The image transmission unit 66 also transmits image data representing the second image 140 generated by the second image generation unit 80 to the image display device 14 via the I / F unit 43.

[0041] The gaze detection unit 68 has the function of detecting the direction of the gaze of the first user's right eye based on the detection result of the gaze detection sensor 28. The gaze detection unit 68 outputs the detection result to the first region determination unit 70.

[0042] The first region determination unit 70 has the function of determining a first region 112 within the captured image 110 based on the detection result of the gaze detection unit 68. In the case shown in Figure 7, the gaze detection unit 68 detects the direction in which the subject 102 is located as the gaze direction based on the detection result of the gaze detection sensor 28. As an example, in this embodiment, a region of a predetermined size with the position in the captured image 110 corresponding to the gaze direction detected by the gaze detection sensor 28 as its center is determined as the first region 112. The predetermined size to be determined as the first region 112 is not particularly limited and may be predetermined in advance, for example, in the design of the AR glasses 10 or experimentally. In the example shown in Figure 7, the first region 112 determined by the first region determination unit 70 includes the subject image 103 corresponding to the subject 102. The first region determination unit 70 outputs information representing the determined first region 112 to the shooting condition determination unit 76 and the first image generation unit 78.

[0043] The second region information receiving unit 72 has the function of receiving information representing the second region 114 from the image display device 14. Specifically, the second region information receiving unit 72 receives information representing the second region 114 from the image display device 14 via the I / F unit 43. The second region information receiving unit 72 outputs the received information representing the second region 114 to the second region determination unit 74.

[0044] The second region determination unit 74 has the function of determining the second region 114 in the captured image 110 based on information representing the second region 114. For example, in the case shown in Figure 7, the second region information receiving unit 72 receives information representing the second region 114 from the image display device 14. The second region determination unit 74 determines the second region 114 in the captured image 110 based on the information representing the second region 114 received by the second region information receiving unit 72. The second region determination unit 74 outputs the determined information representing the second region 114 to the shooting condition determination unit 76 and the second image generation unit 80.

[0045] As shown in Figure 8, the shooting condition determination unit 76 has the function of determining the shooting conditions for the camera 27 to capture the captured image 111, which will be the basis for the first image 120 and the second image 140, based on the first region 112 and the second region 114 of the captured image 110. In this embodiment, the shooting condition determination unit 76 determines as shooting conditions the dynamic range suitable for capturing the subject 102 corresponding to the subject image 103 included in the first region 112 of the captured image 110, and the subject 104 corresponding to the subject image 105 included in the second region 114. The shooting condition determination unit 76 outputs the determined shooting conditions to the shooting control unit 60. For simplicity, below, the subject 102 corresponding to the subject image 103 included in the first region 112 will be referred to as "subject 102 included in the first region 112," and the subject 104 corresponding to the subject image 105 included in the second region 114 will be referred to as "subject 104 included in the second region 114."

[0046] The first image generation unit 78 has the function of generating a first image 120 from the captured image 111 taken by the camera 27 according to the shooting conditions determined by the shooting condition determination unit 76, based on the first region 112 determined by the first region determination unit 70. The first image generation unit 78 outputs the generated first image 120 to the display control unit 64.

[0047] The second image generation unit 80 has the function of generating a second image 140 from the captured image 111 taken by the camera 27 according to the shooting conditions determined by the shooting condition determination unit 76, based on the second region 114 determined by the second region determination unit 74. The second image generation unit 80 outputs the generated second image 140 to the image transmission unit 66.

[0048] On the other hand, as described above, the image display device 14 is used by a second user to observe the captured image 110 and the second image 140 taken by the camera 27 of the AR glasses 10.

[0049] Figure 9 shows a functional block diagram illustrating an example of the configuration related to the functions of the image display device 14. As shown in Figure 9, the image display device 14 comprises a processor 50, memory 52, I / F unit 53, storage unit 54, display 56, and input device 58. The processor 50, memory 52, I / F unit 53, storage unit 54, display 56, and input device 58 are connected to each other via a bus 59, such as a system bus or control bus, enabling the exchange of various types of information.

[0050] The processor 50 reads various programs, including the display control program 55 stored in the memory unit 54, into the memory 52 and executes processing according to the read program. The memory 52 is a work memory for which the processor 50 executes processing.

[0051] The storage unit 54 stores the display control program 55, image data of various images received from the smartphone 12, and other various information. Specific examples of the storage unit 54 include HDDs and SSDs.

[0052] The I / F unit 53 communicates various types of information with the smartphone 12 via wireless or wired communication. The display 56 and input device 58 function as a user interface. The display 56 displays various images received from the smartphone 12. The display 56 is not particularly limited and includes liquid crystal monitors and LED monitors. The input device 58 is operated by a second user to input various instructions related to the second area 114. The input device 58 is not particularly limited and includes, for example, a keyboard, a stylus, and a mouse. A touch panel display integrating the display 56 and the input device 58 may also be used.

[0053] Figure 10 shows a functional block diagram illustrating an example of the configuration of the image display device 14 according to this embodiment. As shown in Figure 10, the image display device 14 comprises an image receiving unit 90, a display control unit 92, a second area information receiving unit 94, and a second area information transmission unit 96. As an example, in this embodiment, the image display device 14 functions as the image receiving unit 90, display control unit 92, second area information receiving unit 94, and second area information transmission unit 96 when the processor 50 executes a display control program 55 stored in the storage unit 54.

[0054] The image receiving unit 90 has the function of receiving the captured image 110 and the second image 140 from the smartphone 12. Specifically, the image receiving unit 90 receives image data representing the captured image 110 or image data representing the second image 140 from the smartphone 12 via the I / F unit 53. The image receiving unit 90 outputs the received captured image 110 and the second image 140 to the display control unit 92.

[0055] The display control unit 92 has the function of controlling the display 56 to show the captured image 110 or the second image 140 received by the image receiving unit 90.

[0056] The second domain information receiving unit 94 has the function of receiving information representing the second domain 114 input by a second user using the input device 58. The second domain information receiving unit 94 outputs the received information representing the second domain 114 to the second domain information transmission unit 96.

[0057] The second domain information transmission unit 96 has the function of outputting information representing the second domain 114 to the smartphone 12. Specifically, the second domain information transmission unit 96 outputs information representing the second domain 114, which is input from the second domain information reception unit 94, to the smartphone 12 via the I / F unit 53.

[0058] Next, the operation of the smartphone 12 and image display device 14 in this embodiment will be explained. Figure 11 shows a flowchart illustrating an example of the control processing performed by the smartphone 12 and the display control processing performed by the image display device 14 in this embodiment. The smartphone 12 executes the control processing shown in Figure 11 by executing the control program 45 stored in the storage unit 44. The image display device 14 also executes the display control processing shown in Figure 11 by executing the display control program 55 stored in the storage unit 54. As an example, in this embodiment, when the smartphone 12 receives a display instruction from the first user via the input device 48 to display a captured image on the AR glasses 10 and the image display device 14, the control processing and display control processing shown in Figure 11 are executed.

[0059] In step S100 of Figure 11, the shooting control unit 60 instructs the camera 27 to start shooting the captured image 110. The camera 27 starts shooting in response to the instruction to start shooting input from the smartphone 12. In this case, the camera 27 shoots the captured image 110 at a predetermined frame rate based on predetermined shooting conditions such as AE (Automatic Exposure) and AF (Autofocus).

[0060] In the next step S102, the image acquisition unit 62 starts acquiring the captured image 110 from the smartphone 12, as described above.

[0061] In the next step S104, the display control unit 64 outputs an instruction to start displaying the captured image to the left eye OLED 26L and right eye OLED 26R of the AR glasses 10, in order to start displaying the captured image 110 acquired in step S102 on the display 20 of the AR glasses 10.

[0062] Specifically, in order to display the captured image 110 on the display 20 of the AR glasses 10, the display control unit 64 outputs the left-eye projection image, which is a partial image of the captured image 110, to the left-eye OLED 26L, and outputs the right-eye projection image, which is a partial image of the captured image 110, to the right-eye OLED 26R.

[0063] In the next step S106, the image transmission unit 66 transmits the captured image 110 acquired in step S100 to the image display device 14. As will be described in detail later, the image display device 14 displays the captured image 110 received from the smartphone 12 on the display 56, receives the second region 114 within the captured image 110 from the second user, and transmits information representing the second region 114 to the smartphone 12 (step S206).

[0064] Therefore, in the next step S108, the second region information receiving unit 72 receives information representing the second region 114 transmitted from the image display device 14. The information representing the second region 114 may include, for example, coordinate information representing the position and size of the second region 114 within the image display device 14.

[0065] In the next step S110, the second region determination unit 74 determines the second region 114 in the captured image 110 based on the information representing the second region 114 received by the second region information receiving unit 72.

[0066] In the next step S112, the gaze detection unit 68 detects the gaze direction of the first user based on the detection result of the gaze detection sensor 28.

[0067] In the next step S114, the first region determination unit 70 determines the first region 112 in the captured image 110, as described above, based on the gaze direction of the first user detected in step S112.

[0068] In the next step S116, the shooting condition determination unit 76 determines the shooting conditions for capturing the captured image 111, based on the first region 112 and the second region 114 within the captured image 110. As described above, the shooting condition determination unit 76 in this embodiment determines the shooting conditions for shooting with a dynamic range suitable for both capturing the subject 102 in the first region 112 and capturing the subject 104 in the second region 114.

[0069] In the next step S118, the shooting control unit 60 instructs the camera to start shooting the captured image 111 under the shooting conditions determined in step S116. The camera 27, upon receiving the instruction to start shooting from the smartphone 12, sets the instructed shooting conditions, starts shooting the captured image 111, and captures the captured image 111 at a predetermined frame rate.

[0070] In the next step S120, the image acquisition unit 62 acquires the captured image 111 taken by the camera 27 in response to the shooting start instruction in step S118.

[0071] In the next step S122, the first image generation unit 78 generates a first image 120 from the captured image 111 acquired in step S120, based on the first region 112. As an example, the first image generation unit 78 in this embodiment generates the first image 120 by adjusting the brightness values ​​of the pixels in the captured image 111 for use with the AR glasses 10, based on the brightness values ​​of the pixels in the first region 112 of the captured image 111. Specifically, the first image generation unit 78 in this embodiment derives the minimum magnification value required to make the brightness values ​​of the pixels in the first region 112 of the captured image 111 equal to or greater than a predetermined brightness value for the AR glasses 10, and generates the first region 112 by multiplying the brightness value of each pixel in the captured image 111 by the derived minimum value.

[0072] In the next step S124, the display control unit 64 issues an instruction to display the first image 120 generated in step S122 on the display 20 of the AR glasses 10. Specifically, in order to display the first image 120 on the display 20 of the AR glasses 10, the display control unit 64 outputs the left-eye projection image, which is a partial image of the first image 120, to the left-eye OLED 26L, and outputs the right-eye projection image, which is a partial image of the first image 120, to the right-eye OLED 26R.

[0073] In the next step S126, the second image generation unit 80 generates a second image 140 from the captured image 111 acquired in step S120, based on the second region 114 determined in step S110. As an example, the second image generation unit 80 of this embodiment generates the second image 140 by adjusting the brightness values ​​of the pixels in the captured image 111 for use with the image display device 14, based on the brightness values ​​of the pixels in the second region 114 of the captured image 111. Specifically, the second image generation unit 80 of this embodiment derives a minimum magnification value to make the brightness values ​​of the pixels in the second region 114 of the captured image 111 equal to or greater than a predetermined brightness value for the image display device 14, and generates a captured image for the image display device 14 in which the brightness value of each pixel in the captured image 111 is multiplied by the derived minimum value. Also, as an example, the second image 140 of this embodiment is a partial image obtained by cutting out the portion of the second region 114 from the captured image 111. Therefore, the second image generation unit 80 generates the second image 140 by cutting out the portion of the second region 114 from the captured image 111 for the image display device 14.

[0074] In the next step S128, the image transmission unit 66 transmits the second image 140 generated in step S126 to the image display device 14. As will be described in detail later, the image display device 14 displays the second image 140 received from the smartphone 12 on the display 56 (S210).

[0075] In the next step S130, the captured image acquisition unit 62 determines whether or not to terminate the control process shown in Figure 11. In this embodiment, the control process shown in Figure 11 is terminated if a predetermined termination condition is met. A predetermined termination condition is, for example, when the smartphone 12 receives an instruction from the first user via the input device 48 to terminate the display of the first image 120 by the AR glasses 10. Until the predetermined termination condition is met, the determination in step S130 is negative, and the process returns to step S120 and steps S120 to S128 are repeated. On the other hand, if the predetermined termination condition is met, the determination in step S130 is positive, and the process proceeds to step S132.

[0076] In step S132, the image transmission unit 66 transmits termination information to the image display device 14, indicating that the display of the first image 120 in the AR glasses 10 has ended.

[0077] In the next step S134, the display control unit 64 outputs an instruction to the AR glasses 10 to end the display of the first image 120 shown on the display 20 of the AR glasses 10. When the processing in step S134 is completed, the control process shown in Figure 11 is completed.

[0078] Meanwhile, in the image display device 14, in step S200 of Figure 11, the image receiving unit 90 starts receiving the captured image 110 transmitted from the smartphone 12 by the process of step S106 of the control process executed in the smartphone 12 described above.

[0079] In the next step S202, the display control unit 92 starts displaying the captured image 110 acquired in step S200 on the display 56. The second user observes the captured image 110 displayed on the display 56 and uses the input device 58 to specify the second region 114 within the captured image 110.

[0080] Therefore, in the next step S204, the second area information receiving unit 94 determines whether or not it has received the designation of the second area 114 made using the input device 58. Until the designation of the second area 114 is received, the determination in step S204 will be negative. On the other hand, if the designation of the second area 114 is received, the determination in step S204 will be positive, and the process will proceed to step S206.

[0081] In step S206, the second domain information transmission unit 96 transmits the information representing the second domain 114 received by the second domain information reception unit 94 to the smartphone 12. As a result, in step S108 of the control process, the second domain information reception unit 72 receives the information representing the second domain 114.

[0082] In the next step S208, the image receiving unit 90 receives the second image 140 transmitted from the smartphone 12 as a result of the control process in step S128 described above.

[0083] In the next step S210, the display control unit 92 displays the second image 140 received in step S208 on the display 56. As an example, in this embodiment, the display control unit 92 displays the second image 140 together with the captured image 110 on the display 56, as shown in Figures 6 and 8.

[0084] In the next step, S212, the image receiving unit 90 determines whether or not it has received the termination information transmitted from the smartphone 12 as a result of the control process described in step S132. If the termination information has not been received, the determination in step S212 becomes negative, and the process returns to step S208, repeating steps S208 and S210. On the other hand, if the termination information has been received, the determination in step S212 becomes positive, and the process proceeds to step S214.

[0085] In step S214, the display control unit 92 determines whether or not to terminate the display of the second image 140 and the captured image 110 displayed on the display 56. As an example, in this embodiment, the display control unit 92 terminates the display of the captured image 110 and the second image 140 when it receives an instruction from the second user via the input device 58 to terminate the display of the captured image 110 and the second image 140. Therefore, the process in step S214 is negative until the display control unit 92 receives an instruction from the second user via the input device 58 to terminate the display. On the other hand, when the display control unit 92 receives an instruction from the second user via the input device 58 to terminate the display, the process in step S214 becomes positive, and the process proceeds to step S216. In step S216, the display control unit 92 terminates the display of the captured image 110 and the second image 140 on the display 56. When the process in step S216 is completed, the display control process shown in Figure 11 is completed.

[0086] As described above, the shooting condition determination unit 76 of the smartphone 12 in this embodiment determines a dynamic range suitable for shooting the subject 102 in the first region 112 and the subject 104 in the second region 114 as the shooting condition. The shooting control unit 60 controls the camera 27 to take one image 111 under the determined shooting condition. The first image generation unit 78 generates a first image 120 from the one image 111, and the display control unit 64 controls the display 20 of the AR glasses 10 to display the first image 120. The second image generation unit 80 generates a second image 140 from the one image 111, and the image transmission unit 66 outputs the second image 140 to the image display device 14.

[0087] As a result, the smartphone 12 of this embodiment can provide a first image 120 suitable for observation by a first user, and a second image 140 suitable for observation by a second user.

[0088] [Second Embodiment] The configuration of the image display device 14 in this embodiment is the same as that of the image display device 14 in the first embodiment (see Figures 9 and 10), so its description will be omitted. Similarly, the configuration of the AR glasses 10 is the same as that of the AR glasses 10 in the first embodiment (see Figures 1 and 2), so its description will be omitted. On the other hand, the hardware configuration of the smartphone 12 in this embodiment is the same as that of the smartphone 12 in the first embodiment (see Figure 5). On the other hand, the configuration of the functions of the smartphone 12 is the same as in the first embodiment, comprising a shooting control unit 60, a captured image acquisition unit 62, a display control unit 64, an image transmission unit 66, a gaze detection unit 68, a first area determination unit 70, a second area information receiving unit 72, a second area determination unit 74, a shooting condition determination unit 76, a first image generation unit 78, and a second image generation unit 80 (see Figure 7), but some of its functions are different, so the different functions will be described.

[0089] As shown in Figure 12, the shooting condition determination unit 76 of this embodiment sets the focus position of the camera 27's optical system to a common focus position between a first focus position where the subject 102 in the first region 112 (not shown in Figure 12) of the captured image 110 is in focus, and a second focus position where the subject 104 in the second region 114 (not shown in Figure 12) of the captured image 110 is in focus. The shooting condition determination unit 76 then controls the camera 27 to capture one image 121 with the common focus position set. In other words, the shooting condition determination unit 76 sets the depth of field of the camera 27 to include at least the subject 102 and the subject 104.

[0090] The first image generation unit 78 obtains the first image 120 from the captured image 121, which is captured by the camera 27 with a common focus position set and acquired by the captured image acquisition unit 62. As an example, in this embodiment, the first image generation unit 78 uses the captured image 121 as the first image 120.

[0091] The second image generation unit 80 generates the second image 140 by extracting the second region 114 from the captured image 121, which is captured by the camera 27 with a common focus position set and acquired by the captured image acquisition unit 62.

[0092] Figure 13 shows a flowchart illustrating an example of the control process performed by the smartphone 12 and the display control process performed by the image display device 14 in this embodiment.

[0093] As shown in Figure 13, the display control processing performed by the image display device 14 of this embodiment is the same as the display control processing performed by the image display device 14 of the first embodiment (see Figure 11), so no explanation is given. On the other hand, the control processing performed by the smartphone 12 of this embodiment differs in part from the control processing performed by the smartphone 12 of the first embodiment (see Figure 11), as shown in Figure 13. The control processing performed by the smartphone 12 of this embodiment differs in part from the control processing performed by the smartphone 12 of the first embodiment (see Figure 11) in that it includes step S116A instead of step S116, does not include step S122, and includes step S127 instead of step S126.

[0094] In step S106A of Figure 13, the shooting condition determination unit 76 determines the shooting conditions for capturing the captured image 111, based on the first region 112 and the second region 114 within the captured image 110. As described above, the shooting condition determination unit 76 in this embodiment determines the shooting conditions so that the focus position of the optical system of the camera 27 is a common focus position between the first focus position where the subject 102 is in focus and the second focus position where the subject 104 is in focus.

[0095] Furthermore, in the control process of this embodiment, the captured image 121 obtained in step S120, which is taken by the camera 27 with a common focus position set, is used as the first image 120. Therefore, as described above, the process of step S122, which was performed in the control process of the first embodiment, is omitted.

[0096] Furthermore, in step S127, the second image generation unit 80 generates the second image 140 by cutting out the portion of the second region 114 from the captured image 121 acquired in step S120. As a result, in the next step S128, the second image 140 cut out from the captured image 121 is transmitted to the image display device 14.

[0097] As described above, the shooting condition determination unit 76 of the smartphone 12 in this embodiment determines the common focus position of the optical system of the camera 27 as a shooting condition, which is between a first focus position where the subject 102 in the first region 112 is in focus and a second focus position where the subject 104 in the second region 114 is in focus. The shooting control unit 60 controls the camera 27 to take one image 121 with the common focus position set. The first image generation unit 78 makes the captured image 121 the first image 120. The display control unit 64 controls the display 20 of the AR glasses 10 to display the first image 120. The second image generation unit 80 extracts the second image 140 from the captured image 121, and the image transmission unit 66 outputs the second image 140 to the image display device 14.

[0098] As a result, the smartphone 12 of this embodiment can provide a first image 120 suitable for observation by a first user, and a second image 140 suitable for observation by a second user.

[0099] In this embodiment as well, the captured image 121 may be subjected to image processing for the display of the AR glasses 10 to generate the first image 120, and the image may also be subjected to image processing for the display 56 of the image display device 14 to generate the second image 140.

[0100] [Third Embodiment] The configuration of the image display device 14 in this embodiment is the same as that of the image display device 14 in the first embodiment (see Figures 9 and 10), so its description will be omitted. Similarly, the configuration of the AR glasses 10 is the same as that of the AR glasses 10 in the first embodiment (see Figures 1 and 2), so its description will be omitted. On the other hand, the hardware configuration of the smartphone 12 in this embodiment is the same as that of the smartphone 12 in the first embodiment (see Figure 5). On the other hand, the configuration of the functions of the smartphone 12 is the same as in the first embodiment, comprising a shooting control unit 60, a captured image acquisition unit 62, a display control unit 64, an image transmission unit 66, a gaze detection unit 68, a first area determination unit 70, a second area information receiving unit 72, a second area determination unit 74, a shooting condition determination unit 76, a first image generation unit 78, and a second image generation unit 80 (see Figure 7), but some of its functions are different, so the different functions will be described.

[0101] As shown in Figure 14, the shooting condition determination unit 76 of this embodiment determines a first shooting condition that controls the exposure of the camera 27 based on the subject 102, and a second shooting condition that controls the exposure of the camera 27 based on the subject 104, as shooting conditions.

[0102] The first image generation unit 78 determines the captured image 1311 taken by the camera 27 according to first shooting conditions that control the exposure of the camera 27 based on the subject 102 as the first image 120. The captured image 1311 in this embodiment is an example of the first captured image of this disclosure.

[0103] The second image generation unit 80 generates a second image 140 by extracting a second region 114 from the captured image 1312 taken by the camera 27 based on a second shooting condition that controls the exposure of the camera 27 based on the subject 104. The captured image 1312 in this embodiment is an example of the second captured image of this disclosure.

[0104] Figure 15 shows a flowchart illustrating an example of the control process performed by the smartphone 12 and the display control process performed by the image display device 14 in this embodiment.

[0105] As shown in Figure 15, the display control processing performed by the image display device 14 of this embodiment is the same as the display control processing performed by the image display device 14 of the first embodiment (see Figure 11), so no explanation is given. On the other hand, the control processing performed by the smartphone 12 of this embodiment differs in part from the control processing performed by the smartphone 12 of the first embodiment (see Figure 11), as shown in Figure 15. The control processing performed by the smartphone 12 of this embodiment differs in part from the control processing performed by the smartphone 12 of the first embodiment (see Figure 11) in that it includes step S111 between step S110 and step S112, step S115 instead of step S116, steps S117 and S119 instead of steps S116 to S122, and steps S125A, S125B, and S127 instead of S126.

[0106] In step S111 of Figure 15, the shooting condition determination unit 76 determines a second shooting condition that controls the exposure of the camera 27 based on the subject 104 included in the second region 114 of the captured image 110, as described above.

[0107] Furthermore, in step S115 of Figure 15, the shooting condition determination unit 76 determines a first shooting condition that controls the exposure of the camera 27 based on the subject 102 included in the first region 112 of the captured image 110, as described above.

[0108] In the next step S117, the shooting control unit 60 instructs the camera to take the image 1311 (first shot) under the first shooting conditions determined in step S115. The camera 27, in response to the instruction for the first shot input from the smartphone 12, sets the instructed first shooting conditions and takes the image 1311.

[0109] In the next step S119, the captured image acquisition unit 62 acquires the captured image 1311 taken by the camera 27 in response to the first shooting instruction in step S117. Also, as described above, the first image generation unit 78 converts the captured image 1311 into the first image 120. As a result, in the next step S124, the display control unit 64 issues an instruction to display the first image 120, which is the captured image 1311, on the display 20 of the AR glasses 10.

[0110] Furthermore, in step S125A of Figure 15, the shooting control unit 60 instructs the camera to take the image 1312 (second shooting) under the second shooting conditions determined in step S111. The camera 27, in response to the instruction for second shooting input from the smartphone 12, sets the instructed second shooting conditions and takes the image 1312.

[0111] In the next step, S125B, the image acquisition unit 62 acquires the captured image 1312 taken by the camera 27 in response to the second shooting instruction in step S125A.

[0112] In the next step S127, the second image generation unit 80 generates the second image 140 by cutting out the portion of the second region 114 from the captured image 1312 acquired in step S125B, as described above. As a result, in the next step S128, the second image 140 cut out from the captured image 1312 is transmitted to the image display device 14.

[0113] As described above, the shooting condition determination unit 76 of the smartphone 12 in this embodiment determines the exposure of the camera 27 based on the subject 102 that is captured in the first region 112 as the first shooting condition. The shooting control unit 60 controls the camera 27 to capture the first image 1311 based on the first shooting condition. The first image generation unit 78 converts the first image 1311 into the first image 120, and the display control unit 64 controls the display of the AR glasses 10 to display the first image 120.

[0114] Furthermore, the shooting condition determination unit 76 determines the exposure of the camera 27 based on the subject 104 captured in the second region 114 as a second shooting condition. The shooting control unit 60 controls the camera 27 to capture the second image 1312 according to the second shooting condition. The second image generation unit 80 extracts the second image 140 from the second image 1312, and the image transmission unit 66 outputs the second image 140 to the image display device 14.

[0115] As a result, the smartphone 12 of this embodiment can provide a first image 120 suitable for observation by a first user, and a second image 140 suitable for observation by a second user.

[0116] As described above, the smartphone 12 of this embodiment includes a processor 40. When a first user observes an image captured by the camera 27 of the AR glasses 10 using the AR glasses 10, and a second user observes the captured image using an image display device 14 different from the AR glasses 10, the processor 40 performs control to output a first image 120 obtained by controlling the shooting based on a first region 112 in the captured image 110 to the AR glasses 10, and a second image 140 obtained by controlling the shooting based on a second region 114 in the captured image 110 to the image display device 14.

[0117] For example, the environment surrounding subjects 102 and 104 may be different, such as when the subject 102 observed by the first user and the subject 104 observed by the second user are located far apart, resulting in different appearances of the first region 112 and the second region 114 of the captured image 110. Even in such cases, the smartphone 12 of this embodiment can provide a first image 120 suitable for observation by the first user using the AR glasses 10, and a second image 140 suitable for observation by the second user using the image display device 14.

[0118] In each of the above embodiments, an embodiment using AR glasses 10 was described as an example of the glasses-type display device of this disclosure. However, the glasses-type display device is not limited to AR glasses 10 and can be applied to various forms such as head-mounted displays.

[0119] Furthermore, the term "glasses" in "glasses-type display device" functionally refers to the realization of a first sensor (human eye) that acquires an image, and a second sensor (camera 27 of AR glasses 10) that includes the field of view of the first sensor and acquires the image in approximately the same line of sight direction as the first sensor. The shape of the glasses-type display device is not limited to the shape, use, or mounting location of ordinary glasses. The glasses-type display device may be monocular or bicular, and although the above embodiments show a configuration in which the projected image is viewed with both eyes, it may also be a configuration in which the projected image is viewed with one eye. It may also be a shape in which the left and right sides are connected, like goggles. Furthermore, it is not limited to those that are worn on the human head, such as so-called head-mounted displays (for example, if a robot that mimics human functions but has the external form of a dog has the functions of a human eye realized by a camera on the robot's knee, then the display device of this disclosure would be worn on the knee. Such a display device is also included in the technology of this disclosure.

[0120] Furthermore, while the above embodiments describe a configuration in which the AR glasses 10 is equipped with one camera 27, and the one camera 27 captures images 110 and 111 corresponding to the entire field of view of the first user, the cameras equipped in the AR glasses 10 are not limited to these embodiments. For example, the AR glasses 10 may be equipped with two cameras, one for capturing the projection image for the left eye and one for capturing the projection image for the right eye. In this case, if the second region 114 is included in either the projection image for the left eye or the projection image for the right eye, for example, in the third embodiment, only the projection image that includes the second region 114 may be captured.

[0121] Furthermore, although the above configurations describe cases where the first user and the second user are different, the first user and the second user may be the same. Also, although the above configurations describe cases where the first user and the second user are each individual, the first user and the second user may each be multiple individuals.

[0122] Furthermore, although the above descriptions describe a configuration in which the image display system 1 is equipped with only one image display device 14, the number of image display devices 14 equipped with the image display system 1 is not limited. If the image display system 1 is equipped with multiple image display devices 14, the actions that the smartphone 12 performed on the image display device 14 as described in the above descriptions can be performed sequentially on each image display device 14 equipped with the image display system 1.

[0123] Furthermore, in each of the above embodiments, the second image 140 is an image extracted from each of the captured images 111, 121, and 1312, respectively. However, the embodiment is not limited to this one, and the captured images 111, 121, and 1312 may also be used as the second image 140.

[0124] Furthermore, in each of the above embodiments, the smartphone 12 is equipped with the functions of the shooting control unit 60, the captured image acquisition unit 62, the display control unit 64, the image transmission unit 66, the gaze detection unit 68, the first area determination unit 70, the second area information receiving unit 72, the second area determination unit 74, the shooting condition determination unit 76, the first image generation unit 78, and the second image generation unit 80. However, some or all of the functions of these units may be equipped with other devices. Other devices may include, for example, an image display device 14 or a server computer located on the cloud. For example, in each of the above embodiments, the smartphone 12 generates the second image 140. However, the smartphone 12 may transmit the information for generating the second image 140 and each of the captured images 111, 121, and 1312 to the image display device 14, and the image display device 14 generates the second image 140 from the captured images 111, 121, and 1312 based on the information for generating the second image 140 that it has received.

[0125] Furthermore, in each of the above embodiments, the hardware structure of the processing unit that performs various processes, such as the shooting control unit 60, captured image acquisition unit 62, display control unit 64, image transmission unit 66, gaze detection unit 68, first area determination unit 70, second area information receiving unit 72, second area determination unit 74, shooting condition determination unit 76, first image generation unit 78, and second image generation unit 80 of the smartphone 12, or the image receiving unit 90, display control unit 92, second area information receiving unit 94, and second area information transmission unit 96 of the image display device 14, can be any of the following types of processors. As mentioned above, the above types of processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), which is a processor whose circuit configuration can be changed after manufacturing, and a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration specifically designed to perform a particular process.

[0126] A single processing unit may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, multiple processing units may be composed of a single processor.

[0127] Examples of configuring multiple processing units with a single processor include, firstly, a configuration where one or more CPUs and software combine to form a single processor, as exemplified by client and server computers, and this processor functions as multiple processing units. Secondly, a configuration using a processor that realizes the functions of the entire system, including multiple processing units, on a single IC (Integrated Circuit) chip, as exemplified by System-on-a-Chip (SoC). Thus, various processing units are configured, in terms of hardware structure, using one or more of the above-mentioned processors.

[0128] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits, which are combinations of circuit elements such as semiconductor devices.

[0129] Furthermore, although the above embodiments describe a configuration in which the control program 45 and the display control program 55 are pre-stored (installed) in the storage unit 44, the invention is not limited to this configuration. The control program 45 and the display control program 55 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory), or USB (Universal Serial Bus) memory. Alternatively, the control program 45 and the display control program 55 may be provided in a form that can be downloaded from an external device via a network. [Explanation of Symbols]

[0130] 1. Image display system 10 AR Glasses 12 Smartphones 14 Image display device 19 Network 20 displays 22L lens for the left eye, 22R lens for the right eye 24L light guide plate for left eye, 24R light guide plate for right eye 26L OLED for left eye, 26R OLED for right eye 27 Cameras 28 Eye-tracking sensor 40, 50 processors 42, 52 memory 43, 53 I / F section 44, 54 Storage section 45 Control Program 46, 56 displays 48, 58 Input devices Buses 49 and 59 55 Display control program 60. Image capture control unit 62 Image acquisition unit 64 Display Control Unit 66 Image transmission unit 68 Eye-line detection unit 70 1st area determination section 72 Second Area Information Receiving Unit 74 Second area determination section 76 Shooting Condition Determination Unit 78 First Image Generation Unit 80 Second Image Generation Unit 90 Image receiving unit 92 Display Control Unit 94 Second Area Information Reception Department 96 Second Domain Information Transmission Unit 100 The Real World Images taken at 101, 110, 111, 121, 1311, and 1312. 101L Projected image for left eye, 101R Projected image for right eye 102, 104 Subject 103, 105 Subject images 112 1st area 114 Second area 120 Image 1 140 Image 2

Claims

1. It includes at least one processor, The aforementioned processor, When a first user observes an image captured by the camera of a glasses-type display device using the glasses-type display device, and a second user observes the captured image using a display device different from the glasses-type display device, A first image, obtained by controlling the shooting process based on a first region within the captured image, is output to the glasses-type display device. The system performs control to output a second image, which has been captured using a second region within the aforementioned captured image as a reference, to the display device. Control device.

2. The aforementioned processor, As part of the aforementioned shooting control, the camera is controlled to capture a single image with a dynamic range suitable for both capturing a subject in the first region and capturing a subject in the second region. Control is performed to output the first image and the second image corresponding to the single captured image. The control device according to claim 1.

3. The aforementioned processor, The brightness value of the aforementioned single captured image is adjusted for the glasses-type display device to generate the first image. The second image is generated by adjusting the brightness value of the first captured image for the display device. The control device according to claim 2.

4. The aforementioned processor, As part of the aforementioned shooting control, the focus position of the camera's optical system is set between a first focus position in which the subject in the first region is in focus and a second focus position in which the subject in the second region is in focus, and the camera is controlled to capture a single image with the focus position set such that the subject in the first region and the subject in the second region are included in the depth of field. Control is performed to output the first image and the second image corresponding to the single captured image. The control device according to claim 1.

5. The aforementioned processor, The first captured image is defined as the first image, obtained by controlling the exposure of the camera based on the subject in the first region. The second image is a second captured image obtained by controlling the exposure of the camera based on the subject captured in the second region. The control device according to claim 1.

6. The second image is a partial image obtained by cutting out the second region from the captured image. The control device according to any one of claims 1 to 5.

7. The aforementioned processor, Based on the gaze of the first user detected by the gaze detection device, the first region is determined. The control device according to any one of claims 1 to 6.

8. A control method by a processor provided in a control device, When a first user observes an image captured by the camera of a glasses-type display device using the glasses-type display device, and a second user observes the captured image using a display device different from the glasses-type display device, A first image, obtained by controlling the shooting process based on a first region within the captured image, is output to the glasses-type display device. The system controls the output of a second image, which has been captured using a second region within the aforementioned captured image as a reference, to the display device. Control method.

9. An image processing program executed by a processor in a control device, When a first user observes an image captured by the camera of a glasses-type display device using the glasses-type display device, and a second user observes the captured image using a display device different from the glasses-type display device, A first image, obtained by controlling the shooting process based on a first region within the captured image, is output to the glasses-type display device. The system controls the output of a second image, which has been captured using a second region within the aforementioned captured image as a reference, to the display device. Control program.

Images