Display control device, display control method, and display control program

Abstract

Provided are a display control device, a display control method, and a display control program that facilitate convenience. A processor of the display control device executes: an image acquisition process for acquiring an image to be displayed on a monitor; a first operation instruction acquisition process for acquiring a first operation instruction that is input through an operation on an operation unit of an imaging device different from a wearable device, and is input through a magnification changing operation on the imaging device to change the display magnification of the image; and a display control process for changing the display magnification of the image in accordance with the first operation instruction.

Description

Display control device, display control method, and display control program

[0001] The technology of the present disclosure relates to a display control device, a display control method, and a display control program.

[0002] In recent years, wearable monitors such as head-mounted displays (HMDs) have been developed as monitors worn on the user's head. Furthermore, unlike conventional HMDs, smart glasses have emerged, which use a small LCD (liquid crystal display) projector to display information in front of the user's eyes. These smart glasses are smaller and lighter than conventional HMDs, and are roughly the same size or slightly larger than eyeglasses. These smart glasses can superimpose a monitor display image with transmitted light or display it in part of the user's field of view. These types of smart glasses are expected to become the mainstream of wearable monitors in the future. Such wearable monitors are an example of a wearable device with a monitor. A display control device has been proposed that can scroll images captured using a camera on the monitor of a wearable device (see, for example, Patent Document 1). Patent Document 1 describes scrolling an image displayed on the monitor of a wearable device in response to the tilt of a smartphone.

[0003] International Publication No. 2014 / 185146

[0004] The technology disclosed herein provides a display control device, a display control method, and a display control program with improved convenience.

[0005] The display control device of the present disclosure is a display control device for a wearable device having a monitor, and is equipped with at least one processor and memory built into or connected to the processor, and the processor executes an image acquisition process to acquire an image to be displayed on the monitor, a first operation instruction acquisition process to acquire a first operation instruction input through an operation on an operation unit of an imaging device separate from the wearable device to change the display magnification of the image, the first operation instruction being input through a magnification change operation on the imaging device, and a display control process to change the display magnification of the image in accordance with the first operation instruction.

[0006] The magnification change operation may be a zoom magnification change operation performed when taking a photograph using a photographing device.

[0007] In addition, in the display control process, the processor may control the amount of change in the display magnification of the image in accordance with the amount of change in magnification relative to the amount of magnification change operation.

[0008] In addition, the processor may further execute a second operation instruction acquisition process that acquires a change in the attitude of the photographing device detected by an attitude detection unit that detects the attitude of the photographing device as a second operation instruction, and in the display control process, scroll the image in accordance with the second operation instruction.

[0009] In addition, if the photographing device is equipped with a touch panel that functions as an operation unit, and the magnification change operation when photographing with the photographing device is a swipe operation within a first area on the touch panel, the processor may detect a swipe operation within a second area on the touch panel that includes the first area and is larger than the first area as an operation to change the display magnification of the image.

[0010] The processor may also detect an arc-shaped swipe operation on the touch panel with a radius of 3 cm to 8 cm as a magnification change operation.

[0011] The image may also be an image taken by a photographing device.

[0012] In addition, the processor may be mounted on the wearable device, and in the image acquisition process, the processor may acquire an image from an external server, and in the first operation instruction acquisition process, the processor may acquire a first operation instruction input through an operation on the operation unit of the photographing device via the server.

[0013] In addition, the processor may be mounted on the wearable device, and in the image acquisition process, the processor may acquire an image from an external server, and in the first operation instruction acquisition process, the processor may acquire a first operation instruction input through an operation on an operation unit of the photographing device directly from the photographing device.

[0014] The image may also include information on the tilt angle, which is the inclination angle of the imaging device relative to the horizontal direction when the image was captured, and the processor may change the initial position of the center position when changing the display magnification of the image based on the tilt angle during the display control process.

[0015] The image may also include information on the roll angle, which is the angle of rotation relative to the horizontal direction around the optical axis of the image capturing device when the image was captured, and the processor may perform horizontal correction of the image based on the roll angle during the display control process.

[0016] In addition, the image includes magnification change operation related information regarding the amount of change in magnification relative to the amount of operation of the magnification change operation when photographing the image using the photographing device that photographed the image, and the processor may, in the display control processing, match the amount of change in magnification relative to the amount of operation when photographing the image with the amount of change in magnification relative to the amount of operation when changing the display magnification of the image, based on the magnification change operation related information.

[0017] In addition, the image may include angle of view information regarding the angle of view of the imaging device when the image was captured, and the processor may correct the amount of scrolling when scrolling the image in response to the second operation instruction based on the angle of view information in the display control process.

[0018] The display control method disclosed herein is a display control method for a wearable device having a monitor, and includes an image acquisition processing step for acquiring an image to be displayed on the monitor, a first operation instruction acquisition processing step for acquiring a first operation instruction input through an operation on an operation unit of an imaging device separate from the wearable device to change the display magnification of the image, the first operation instruction being input through a magnification change operation on the imaging device, and a display control processing step for changing the display magnification of the image in accordance with the first operation instruction.

[0019] The display control program of the present disclosure is a display control program for a wearable device having a monitor, and causes a computer to execute an image acquisition processing step for acquiring an image to be displayed on the monitor, a first operation instruction acquisition processing step for acquiring a first operation instruction input through an operation on an operation unit of an imaging device separate from the wearable device to change the display magnification of the image, the first operation instruction being input through a magnification change operation on the imaging device, and a display control processing step for changing the display magnification of the image in accordance with the first operation instruction.

[0020] 16 is a schematic configuration diagram of an image display system including a display control device according to a first embodiment. FIG. 17 is a block diagram showing the hardware configuration of a smartphone. FIG. 18 is a functional block diagram of a smartphone. FIG. 19 is an external view of smart glasses. FIG. 20 is a block diagram showing the hardware configuration of smart glasses. FIG. 21 is a diagram for explaining a user's field of view seen through smart glasses. FIG. 22 is a diagram showing the state of a user when capturing an image. FIG. 23 is a flowchart for explaining processing when capturing an image in the first embodiment. FIG. 24 is a diagram showing an example of a capturing screen displayed on a touch panel. FIG. 25 is a diagram showing the state of a user when an image is displayed. FIG. 26 is a flowchart for explaining processing when an image is displayed in the first embodiment. FIG. 27 is a diagram showing an example of a display screen displayed on a touch panel. FIG. 28 is a graph showing the relationship between swipe amount and magnification in a zoom magnification change operation and a display magnification change operation. FIG. 29 is a diagram for explaining a swipe operation on a display screen. FIG. 29 is a diagram showing a state in which an image displayed on smart glasses is enlarged from an initial display state. FIG. 29 is a diagram showing a state in which an image displayed on smart glasses is scrolled to the right from the state shown in FIG. 15. FIG. 29 is a diagram showing a state in which an image displayed on smart glasses is scrolled up from the state shown in FIG. 15. FIG. 29 is a schematic configuration diagram of an image display system including a display control device according to a second embodiment. FIG. 29 is a flowchart for explaining processing when an image is displayed in the second embodiment. FIG. 10 is a flowchart illustrating processing when an image is displayed in a third embodiment. FIG. 11 is a diagram illustrating the structure of an image file in a fourth embodiment. FIG. 12 is a diagram illustrating a tilt angle in the technology of the present disclosure. FIG. 13 is a diagram illustrating the initial position of the center position when changing the display magnification. FIG. 14 is a graph showing the relationship between the tilt angle and the amount of change in the Y-axis center position. FIG. 14 is a diagram illustrating the structure of an image file in a fifth embodiment. FIG. 15 is a diagram illustrating a roll angle in the technology of the present disclosure. FIG. 16 is a diagram illustrating an image taken with the smartphone tilted. FIG. 17 is a diagram illustrating a state in which an image taken with the smartphone tilted has been corrected. FIG. 18 is a diagram illustrating the structure of an image file in a sixth embodiment. FIG. 19 is a graph showing the relationship between the swipe amount and magnification on a touch panel. FIG. 19 is a diagram illustrating an example of a display screen displayed on a touch panel.FIG. 10 is an external view of a smartphone with a smartphone cover attached, viewed from the front. FIG. 11 is an external view of a smartphone with a smartphone cover attached, viewed from the back. FIG. 12 is a diagram illustrating the structure of an image file in a seventh embodiment. FIG. 13 is a graph showing the relationship between the amount of rotation of a zoom ring or a ring-type controller and magnification. FIG. 14 is a diagram illustrating the structure of an image file in an eighth embodiment. FIG. 15 is a diagram illustrating the relationship between the angle of view and the tilt angle of the smartphone. FIG. 16 is a diagram illustrating an example of an image displayed on smart glasses when capturing an image.

[0021] [First embodiment] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. Fig. 1 is a schematic configuration diagram of an image display system including a display control device according to a first embodiment of the present disclosure. The image display system 1 shown in Fig. 1 includes a smartphone 10 and smart glasses 50. By causing the smart glasses 50 to function as an external display of the smartphone 10, it is possible to display images that can be displayed on the smartphone 10 on the smart glasses 50. In the image display system 1, as described below, it is possible to change the display magnification and scroll of an image displayed on the smart glasses 50 by operating the smartphone 10.

[0022] 1 , in the image display system 1, a smartphone 10 and smart glasses 50 are wirelessly connected, for example. The smartphone 10 is an example of an imaging device in the technology of the present disclosure. The smart glasses 50 are glasses-type wearable computers, or so-called wearable monitors. The wearable monitor is an example of a wearable device that is worn on the body of a user U and has a monitor that displays images within the field of view of the user U.

[0023] As is well known, the smartphone 10 is a mobile terminal that functions as both a mobile computer and a mobile phone. The smartphone 10 includes a flat housing 11. A touch panel 24 is disposed on one side of the housing 11, and a photographing lens 26a is disposed on a side 11b of the housing 11 opposite the side on which the touch panel 24 is disposed. For convenience, in the following description, the side on which the touch panel 24 is disposed of the housing 11 of the smartphone 10 will be referred to as the front side 11a (see FIG. 8 ), and the side on which the photographing lens 26a is disposed will be referred to as the back side 11b.

[0024] 2, the smartphone 10 includes a CPU (Central Processing Unit) 21, a memory 22, a storage 23, a touch panel 24, a communication unit 25, an image capturing unit 26, and an attitude detection unit 27. Each component is connected to each other via a bus 28 so as to be able to communicate with each other.

[0025] The CPU 21 executes a control program and the like to comprehensively control each unit of the smartphone 10. The memory 22 is a working memory and is configured, for example, by a RAM (Random Access Memory). The CPU 21 reads the control program from the storage 23 into the memory 22 and executes the control program using the memory 22 as a working area. The CPU 21 controls each of the above components and performs various processes in accordance with the control program.

[0026] The CPU 21 is an example of a processor in the technology of the present disclosure. The memory 22 is an example of a memory in the technology of the present disclosure. The smartphone 10 including the CPU 21 and the memory 22 also functions as a display control device in the technology of the present disclosure.

[0027] The storage 23 stores control programs including an operating system, various programs such as application programs, and various data including image data. The storage 23 is configured, for example, with a non-volatile memory such as a flash memory. In this embodiment, the storage 23 stores a display control program PG as one of the various programs.

[0028] The touch panel 24 functions as a display unit that displays various images and as an operation unit that accepts touch input operations.

[0029] The communication unit 25 is an interface for the smartphone 10 to communicate with the smart glasses 50 and other devices, and uses standards such as Wi-Fi (registered trademark) and Bluetooth (registered trademark), for example.

[0030] The photographing unit 26 includes a photographing lens 26a and an image sensor (not shown), etc. The photographing unit 26 photographs a subject to obtain image data of the subject.

[0031] The attitude detection unit 27 is for detecting the attitude of the smartphone 10, and for example, a gyro sensor is used.

[0032] As shown in FIG. 3, the CPU 21 executes the display control program PG stored in the storage 23 to function as an image acquisition processing unit 21a, an operation instruction acquisition processing unit 21b, and a display control processing unit 21c.

[0033] The image acquisition processing unit 21 a executes image acquisition processing to acquire an image to be displayed on the smart glasses 50 .

[0034] The operation instruction acquisition processing unit 21b executes a first operation instruction acquisition process to acquire a first operation instruction input through an operation on an operation unit of the smartphone 10 separate from the smart glasses 50 in order to change the display magnification of an image, the first operation instruction being input through a magnification change operation on the smartphone 10. As an example, the first operation instruction is the same operation as a zoom magnification change operation when taking a picture using the smartphone 10, and is a swipe operation on the touch panel 24.

[0035] In addition, the operation instruction acquisition processing unit 21b executes a second operation instruction acquisition process that acquires a change in the attitude of the smartphone 10 detected by the attitude detection unit 27 that detects the attitude of the smartphone 10, i.e., a change in the orientation of the photographing lens 26a of the smartphone 10, as a second operation instruction.

[0036] In response to the first operation instruction, the display control processing unit 21c executes processing to change the display magnification of the image to be displayed on the smart glasses 50. In addition, in response to the second operation instruction, the display control processing unit 21c executes processing to scroll the image to be displayed on the smart glasses 50.

[0037] As shown in FIG. 4, the smart glasses 50 are a glasses-type wearable monitor that includes a frame 51, a lens 52 that is positioned in front of the eyes and is large enough to cover both eyes, and a display unit 64 that is positioned on the lens 52 at a position corresponding to the right eye. As an example, the display unit 64 is a retinal projection display unit that displays an image IM in a portion of the field of view of the right eye, as shown in FIG. 5, by directly projecting display light of the image onto the retina. FIG. 5 is a schematic diagram showing the state in which the image IM appears in the field of view of a user U wearing the smart glasses 50. To the user U, the image IM appears as if it were projected onto the lens 52.

[0038] 6 , the smart glasses 50 include a CPU 61, a memory 62, a storage 63, a display unit 64, and a communication unit 65. Each component is connected to each other via a bus 66 so as to be able to communicate with each other.

[0039] The CPU 61 controls each part of the smart glasses 50 by executing a control program and the like. The memory 62 is a working memory and is configured, for example, by a RAM. The storage 63 stores various programs and various data. The storage 63 is configured, for example, by a flash memory. The CPU 61 reads the control program from the storage 63 into the memory 62 and executes the control program using the memory 62 as a working area. The CPU 61 controls each of the above components and performs various processes in accordance with the control program.

[0040] The display unit 64 projects display light of an image onto the pupil of the user U to display the image in a part of the field of view of the user U, and includes an image display element and a projection lens (not shown). The resolution of the image display element is, for example, 640 dots in the horizontal direction and 360 dots in the horizontal direction.

[0041] The communication unit 65 is an interface for the smart glasses 50 to communicate with the smartphone 10 and other devices, and uses standards such as Wi-Fi (registered trademark) and Bluetooth (registered trademark), for example.

[0042] [Processing Flow] First, a description will be given of the processing flow when the user U captures an image using the smartphone 10. As shown in Fig. 7 , when capturing an image, as an example, the user U does not wear the smart glasses 50, but captures the image using only the smartphone 10.

[0043] FIG. 8 is a flowchart illustrating a process for capturing an image using the smartphone 10.

[0044] When the user U inputs an instruction to switch to image capture mode, such as an instruction to start the camera, the CPU 21 of the smartphone 10 transitions to image capture mode (step SP1).

[0045] When the image capture mode is entered, the CPU 21 displays a capture screen IP (see FIG. 9 ) on the touch panel 24 (step SP2). As shown in FIG. 9 , the capture screen IP includes a zoom magnification input unit 30, which is a GUI (Graphical User Interface) for adjusting the zoom magnification, a shutter button 31, and a live view display unit 32. The live view display unit 32 displays a live view of the image IM to be captured. In FIG. 9 , to avoid cluttering the illustration, the image IM is shown outside the margin of the live view display unit 32. However, the image IM is actually displayed within the live view display unit 32, and the image IM and the zoom magnification input unit 30 are displayed superimposed on each other.

[0046] The zoom magnification input unit 30 includes, for example, a magnification display section 30a that displays the current zoom magnification and a magnification designation line 30b for designating a zoom magnification from the wide-angle end to the telephoto end. The zoom magnification is displayed based on the focal length of the wide-angle end. That is, the zoom magnification at the wide-angle end is the reference, and is therefore set to 1x. The zoom magnification at the telephoto end is, for example, 5x. The magnification designation line 30b is an arc-shaped curve. One end (the bottom end in FIG. 9 ) of the magnification designation line 30b represents the zoom magnification at the wide-angle end, which is set to 1x in this example, and the other end (the top end in FIG. 9 ) of the magnification designation line 30b represents the zoom magnification at the telephoto end, which is set to 5x in this example. The zoom magnification in the middle portion of the magnification designation line 30b ranges from 1x to 5x and is set depending on the distance from the one end. In this example, the magnification display section 30a is circular and is large enough to enclose numbers indicating the zoom magnification, such as 1x and 5x.

[0047] The display position of the circular magnification display section 30a is fixed. Meanwhile, the magnification designation line 30b moves along an arc-shaped trajectory relative to the magnification display section 30a in response to a swipe operation by the user U. Numbers indicating the zoom magnification, such as 1x (×1.0) and 5x (×5.0), displayed at both ends of the magnification designation line 30b are movable along with the magnification designation line 30b. As the magnification designation line 30b moves in response to a swipe operation by the user U, the intersection position of the magnification designation line 30b with respect to the magnification display section 30a changes. The zoom magnification corresponding to the intersection position of the magnification designation line 30b is displayed within the circular magnification display section 30a. FIG. 9 shows a state in which 2.5x (×2.5) is displayed within the magnification display section 30a. In this manner, the zoom magnification corresponding to the intersection position of the magnification designation line 30b with respect to the magnification display section 30a is input.

[0048] The zoom magnification input area 24a is an area including the magnification display unit 30a and the magnification specification line 30b, and is set in an arc shape along the magnification specification line 30b, which is an arc-shaped curve. The zoom magnification input area 24a indicates a range of movement of the finger of the user U operating the zoom magnification input unit 30. The zoom magnification input area 24a is a first area in the technology of the present disclosure.

[0049] The CPU 21 detects a swipe operation that moves the magnification specification line 30b within the zoom magnification input area 24a on the touch panel 24 as a zoom magnification change operation performed when photographing using the smartphone 10. When a zoom magnification change operation is detected, the CPU 21 changes the zoom magnification of the photographing lens 26a or the zoom magnification of the electronic zoom. In response to this change in zoom magnification, the image IM being displayed on the live view display unit 32 is zoomed. That is, as the zoom magnification approaches the telephoto end, the subject in the image IM is enlarged while the angle of view is narrowed, and as the zoom magnification approaches the wide-angle end, the subject in the image IM is reduced while the angle of view is widened.

[0050] In the smartphone 10, the optical axis direction Z of the photographing lens 26a provided on the back surface 11b of the housing 11 is the photographing direction of the image. The user U moves the smartphone 10 to adjust the orientation of the photographing lens 26a of the smartphone 10, thereby adjusting the photographing direction.

[0051] The user U adjusts the shooting direction and zoom magnification, and when ready to shoot, touches the shutter button 31. When the CPU 21 detects the touching of the shutter button 31 (step SP3), it controls the photographing unit 26 to take a photograph (step SP4) and acquires image data (step SP5). The CPU 21 saves the image data acquired by the photographing unit 26 in the storage 23 and ends the process.

[0052] Next, a description will be given of a processing flow when the user U checks an image in the image display system 1. As shown in Fig. 10 , when the user U checks an image IM using the smart glasses 50, the user U wears the smart glasses 50.

[0053] FIG. 11 is a flowchart illustrating the process of image display in the image display system 1.

[0054] When an instruction to switch to the image display mode is input from the user U, the CPU 21 of the smartphone 10 transitions to the image display mode (step SP11).

[0055] When the mode is switched to the image display mode, the CPU 21 acquires information about the resolution of the display unit 64 of the smart glasses 50 connected to the smartphone 10 (step SP12). The information about the resolution of the display unit 64 is stored in the storage 23 together with model information of the smart glasses 50, for example, in the settings when wirelessly connecting the smart glasses 50 to the smartphone 10. When the mode is switched to the image display mode, the CPU 21 acquires the information about the resolution of the display unit 64 by reading it from the storage 23.

[0056] When the user U specifies an image to be displayed from among images captured by the smartphone 10, the CPU 21 executes an image acquisition process to acquire the image from the storage 23 (step SP13). Here, as an example, a case will be described in which an image IM of a landscape containing a person, as shown in Fig. 9, etc., is displayed. As an example, the resolution of the image IM captured by the smartphone 10 is 4096 dots horizontally and 3072 dots vertically.

[0057] In this example, the resolution of the display unit 64 of the smart glasses 50 is lower than the resolution of the image IM, so the CPU 21 adjusts the resolution of the image IM to match the resolution of the display unit 64 of the smart glasses 50. Adjusting the resolution of the image IM is equivalent to adjusting the image size. In the initial state, the image IM is sized so that the entire image IM fits on the entire screen of the display unit 64. The CPU 21 adjusts the image IM to the initial image size and displays it on the smart glasses 50 (step SP14).

[0058] Next, as shown in FIG. 12 , the CPU 21 of the smartphone 10 displays a display screen IV on the touch panel 24, and enters a state in which a first operation instruction for changing the display magnification of the image IM displayed on the smart glasses 50 and a second operation instruction for scrolling the image IM are accepted from the user U (step SP15). The display screen IV is an operation screen that is displayed on the touch panel 24 when the image IM is displayed on the smart glasses 50. In this example, the image IM is not displayed on the display screen IV of the smartphone 10, and the image IM is displayed only on the smart glasses 50. Note that when the image IM is displayed on the smart glasses 50, the image IM may also be displayed on the display screen IV.

[0059] When a first operation instruction for changing the display magnification of the image IM or a second operation instruction for scrolling the image IM is input via the display screen IV (determined Yes in step SP16), the CPU 21 acquires the input operation instruction. Step SP16 is an example of an operation instruction acquisition process for acquiring the first operation instruction or the second operation instruction. Next, the CPU 21 executes a display control process for changing the display magnification and scrolling the image IM in accordance with the acquired operation instruction, generates the image IM with the changed display mode, and displays it on the smart glasses 50 (step SP18).

[0060] The above-described process of changing the display magnification of the image IM and scrolling (steps SP16 to SP18) is repeated until the user U inputs an instruction to end the image display.

[0061] When the user U has finished checking the image IM, he / she inputs an instruction to end the image display to the smartphone 10. The instruction to end the image display is input, for example, by using a physical button (not shown) provided on the smartphone 10 or a back button (not shown) on the touch panel 24. When the instruction to end the image display is input (determined Yes in step SP19), the CPU 21 stops the display of the image IM and ends the process.

[0062] Furthermore, after the CPU 21 displays the display screen IV (step SP15), if an instruction to end image display is input (determined Yes in step SP19) when no operation instruction for changing the display magnification or scrolling of the image IM is input (determined No in step SP16), the CPU 21 stops displaying the image IM in the initial state and ends the processing.

[0063] Here, the process of changing the display magnification of the image IM and the process of scrolling will be described in detail separately.

[0064] First, changing the display magnification of the image IM will be described. As shown in Fig. 12, the zoom magnification input section 30 is displayed on the display screen IV. In the image display mode, the zoom magnification input section 30 is displayed as an operation section for changing the display magnification of the image IM. The zoom magnification input section 30 is displayed, for example, at approximately the same position as the shooting screen IP. This makes it possible to change the display magnification by the same operation as the zoom magnification change operation performed when shooting.

[0065] The magnification input area 24b is an area that accepts input of a magnification change operation for changing the display magnification. The magnification input area 24b is set to a rectangular shape that is wider than the zoom magnification input area 24a. The magnification input area 24b is a second area in the technology of the present disclosure.

[0066] The CPU 21 detects a swipe operation that moves the magnification specification line 30b within the magnification input area 24b on the touch panel 24 as a magnification change operation for changing the display magnification. At this time, when a finger touches a position away from the magnification specification line 30b within the magnification input area 24b, the CPU 21 determines that the finger is touching a position on the magnification specification line 30b that is closest to the finger contact position, and detects the swipe operation on the magnification specification line 30b as a magnification change operation.

[0067] FIG. 13 is a graph showing the relationship between the swipe amount (i.e., the operation amount) and the magnification in a magnification change operation. The state where the lower limit value of the magnification indicated on the magnification specification line 30b of the zoom magnification input unit 30 is superimposed on the magnification display unit 30a is set as the reference position (i.e., swipe amount 0), and graph R1 shows the relationship between the change in the zoom magnification and the swipe amount from the reference position. As described above, in the zoom magnification input unit 30, the reference position indicates the zoom magnification at the wide-angle end (1x) in the image capture mode. On the other hand, in the image display mode, the reference position indicates that the display magnification of the image IM is 1x (1x). 1x (1x) display is a state in which the entire image IM in its initial state that matches the resolution of the touch panel 24 is displayed on the smart glasses 50, as shown in FIG. 5 .

[0068] In graph R1, the magnification changes in proportion to the amount of swiping. In image capture mode, when a swipe operation is performed, the CPU 21 determines the zoom magnification corresponding to the amount of swiping from the relationship shown in graph R1. The greater the amount of swiping, the greater the zoom magnification. In image display mode, when a swipe operation is performed, the CPU 21 determines the display magnification corresponding to the amount of swiping from the relationship shown in graph R1. The greater the amount of swiping, the greater the display magnification. The greater the display magnification, the more enlarged the image IM is displayed. The relationship between the amount of operation and magnification shown in graph R1 is used for both changing the zoom magnification in image capture mode and changing the display magnification in image display mode. Therefore, in image display mode, when a swipe operation of the same swipe amount as the swipe operation in image capture mode is performed, the CPU 21 changes the display magnification of the image IM by the same amount as the change in the zoom magnification in image capture mode.

[0069] 14 , the swipe operation for changing the magnification in the image display mode is an arc-shaped swipe operation, which is similar to the zoom magnification change operation in the image capture mode. The CPU 21 detects such an arc-shaped swipe operation as a magnification change operation. The arc-shaped swipe operation on the touch panel 24 is, for example, an arc-shaped swipe operation with a radius of 3 cm to 8 cm. This is a shape and range that allows a user U to comfortably perform a swipe operation with their thumb while holding the smartphone 10 in their hand.

[0070] When a swipe operation is input in the magnification input area 24b (see step SP16 in FIG. 11), the CPU 21 acquires this swipe operation as a first operation instruction.

[0071] The CPU 21 executes a display control process to change the display magnification of the image IM to be displayed on the smart glasses 50 in response to the zoom magnification change operation, which is the first operation instruction (see step SP17 in FIG. 11 ). At this time, the CPU 21 controls the change amount of the display magnification of the image IM in response to the change amount of the magnification relative to the operation amount of the zoom magnification change operation.

[0072] Also, as an example, the center position when changing the display magnification of the image IM is the center position of the image IM currently displayed on the smart glasses 50. As shown in FIG. 15 , in the initial state, the center position of the entire image IM becomes the initial position C0, which is the center position when changing the display magnification. In other words, when enlarging the image IM according to the display magnification, the image IM is enlarged with the initial position C0 as the center. The area DA indicates the area displayed on the smart glasses 50 when the image IM is enlarged with the initial position C0 as the center.

[0073] The center position can be changed from the initial position C0. For example, the user U may be allowed to touch any position on the image IM to specify the center position. This allows the image IM to be enlarged with the area of ​​interest of the user U at the center.

[0074] Next, scrolling of the image IM will be described. When the display magnification of the image IM is changed and the entire image IM is not displayed on the smart glasses 50, the image IM can be scrolled. For example, as shown in FIG. 15 , when an area DA, which is a part of the image IM, is displayed, the area DA displayed within the image IM can be changed by scrolling the image IM.

[0075] For scrolling, the user U moves the smartphone 10 in the same way as when taking a photo, and adjusts the orientation of the photographing lens 26a of the smartphone 10 to specify the scroll direction. For example, if the user U wants to scroll the image IM to the right, the user tilts the orientation of the photographing lens 26a of the smartphone 10 to the right from the reference direction. Also, if the user U wants to scroll the image IM up, the user tilts the orientation of the photographing lens 26a of the smartphone 10 up from the reference direction.

[0076] Regarding the reference direction, the reference direction in the left-right direction is set to, for example, the orientation of the photographing lens 26a of the smartphone 10 at the time of transition to the image display mode in step SP11 shown in Fig. 11. Furthermore, the reference direction in the up-down direction is set to, for example, the direction in which the orientation of the photographing lens 26a of the smartphone 10 is horizontal.

[0077] The CPU 21 constantly monitors the orientation of the smartphone 10 detected by the orientation detection unit 27, and acquires a change in the orientation of the smartphone 10 as a second operation instruction (see step SP16 in FIG. 11).

[0078] The CPU 21 executes a display control process to scroll the image IM to be displayed on the smart glasses 50 in response to the second operation instruction (step SP17).

[0079] For example, when it is detected that the orientation of the photographing lens 26a of the smartphone 10 has been tilted to the right from the reference direction, the CPU 21 scrolls the image IM to the right from the state shown in FIG. 15 as shown in FIG. 16. As a result, the area DA displayed on the smart glasses 50 in the image IM is changed to the right part of the image IM. Furthermore, when the orientation of the photographing lens 26a of the smartphone 10 has been tilted upward from the reference direction, the CPU 21 scrolls the image IM upward from the state shown in FIG. 15 as shown in FIG. 17. As a result, the area DA displayed on the smart glasses 50 in the image IM is changed to the upper side of the image IM.

[0080] [Effects] In this embodiment, the CPU 21 (corresponding to a processor) of the smartphone 10 (corresponding to an imaging device and a display control device) executes an image acquisition process that acquires an image to be displayed on the smart glasses 50 (corresponding to a wearable monitor), a first operation instruction acquisition process that acquires a first operation instruction that is input through an operation on an operation unit of the smartphone 10 separate from the smart glasses 50 in order to change the display magnification of the image, the first operation instruction being input through a magnification change operation on the smartphone 10, and a display control process that changes the display magnification of the image in accordance with the first operation instruction.

[0081] This allows the user U to change the display magnification of the image displayed on the smart glasses 50 by performing a magnification change operation on the smartphone 10, making it a highly convenient display control device.

[0082] In general, the display resolution of a wearable monitor such as the smart glasses 50 is lower than the resolution of an image captured by an imaging device such as the smartphone 10. Therefore, in order to check for noise and other issues in the details of an image on the wearable monitor, the operation of changing the display magnification of the image to enlarge it is frequently performed.

[0083] According to the technology disclosed herein, such frequently operated image display magnification change operations can be performed with the same feel as the zoom magnification change operations performed during image capture, as in this example, allowing the user U to easily change the magnification even when wearing a wearable monitor such as smart glasses 50 and partially blocking the field of view. This is because both changing the zoom magnification during image capture and changing the display magnification during image display involve enlarging or reducing the displayed image, and the user U is likely to be familiar with changing the zoom magnification operations on an image capture device such as the smartphone 10. This makes it possible to provide a display control device that is easy to operate and highly convenient for the user.

[0084] In this example, the magnification change operation in the imaging device is a zoom magnification change operation during imaging on smartphone 10, which is an example of an imaging device, and the display magnification of the image on the wearable monitor is changed by the same operation as the zoom magnification change operation. However, this is not limited to this. Changing the zoom magnification is just one example of the magnification change operation of the present invention, and it can also be applied to changing the magnification of an extender lens of a TV lens, etc. The magnification change operation in the imaging device may also be a display magnification change operation that changes the display magnification of an image when displaying an image on smartphone 10, which is an example of an imaging device. In other words, the display magnification of the image on the wearable monitor may be changed by the same operation as the display magnification change operation of the image on the imaging device.

[0085] Furthermore, in the display control process, the CPU 21 controls the amount of change in the display magnification of the image IM in accordance with the amount of change in magnification relative to the amount of magnification change operation. This is preferable because it allows the display magnification of the image displayed on the smart glasses 50 to be changed with the same operational feel as the magnification change operation on the smartphone 10.

[0086] The CPU 21 further executes a second operation instruction acquisition process for acquiring, as a second operation instruction, a change in the attitude of the smartphone 10 detected by the attitude detection unit 27 that detects the attitude of the smartphone 10, and scrolls the image in accordance with the second operation instruction in the display control process. By adopting such an embodiment, the scrolling operation can be performed with the same feeling as tilting the smartphone 10 toward a desired subject when photographing.

[0087] Furthermore, the smartphone 10 includes a touch panel 24. When the zoom magnification change operation during image capture with the smartphone 10 is a swipe operation within the zoom magnification input area 24a on the touch panel, the CPU 21 detects a swipe operation within a magnification input area 24b on the touch panel 24 that includes the zoom magnification input area 24a and is wider than the zoom magnification input area 24a as an operation to change the display magnification of the image. By adopting this configuration, even if the user U performs the display magnification change operation without looking at the touch panel 24 and the position of the finger performing the swipe operation moves away from the magnification designation line 35b, the swipe operation can be detected. This realizes a more convenient display control device.

[0088] Furthermore, the CPU 21 detects an arc-shaped swipe operation on the touch panel 24 with a radius R of 3 cm to 8 cm as a display magnification change operation. An arc-shaped swipe operation with a radius R of 3 cm to 8 cm is a shape that allows the user U to perform a swipe operation comfortably when holding the smartphone 10 in his / her hand and swiping with his / her thumb. By adopting such an embodiment, a more convenient display control device can be achieved. Note that if the radius R of the arc-shaped swipe operation is less than 3 cm, the thumb must be bent excessively to perform the operation, making the swipe operation difficult. Also, if the radius R of the arc-shaped swipe operation is greater than 8 cm, the thumb may not be able to reach the target even when fully extended, making the swipe operation difficult.

[0089] Furthermore, the image IM displayed on the smart glasses 50 is an image captured by the smartphone 10. That is, capturing the image IM and changing the display magnification when displaying the image IM can be performed using the same smartphone 10. Therefore, a highly convenient display control device is realized.

[0090] Second Embodiment Next, a second embodiment of the present disclosure will be described. An image display system 2 according to the second embodiment differs from the first embodiment in that the device functioning as a display control device is changed from the smartphone 10 to smart glasses 50. That is, as will be described later, the smart glasses 50 execute an image acquisition process, a first operation instruction acquisition process, and a display control process. Furthermore, in the second embodiment, unlike the first embodiment, the image displayed on the smart glasses 50 is acquired from an external server 70, not from the smartphone 10.

[0091] The hardware configuration of the smartphone 10 and the smart glasses 50 is the same as that of the first embodiment, and therefore a description of the content that overlaps with the first embodiment will be omitted. Fig. 18 is a schematic configuration diagram of an image display system including a display control device according to the second embodiment.

[0092] 18 , the image display system 2 includes a smartphone 10 and smart glasses 50. The CPU 61 of the smart glasses 50 is an example of a processor in the technology of the present disclosure. The memory 62 is an example of a memory in the technology of the present disclosure. The smart glasses 50 including the CPU 61 and the memory 62 also function as a display control device in the technology of the present disclosure. The CPU 61 of the smart glasses 50 acquires images to be displayed on the smart glasses 50 from an external server 70.

[0093] The smartphone 10 and the smart glasses 50 are directly connected wirelessly. The smartphone 10 and the server 70 are connected wirelessly via a network 71. Similarly, the smart glasses 50 and the server 70 are connected wirelessly via the network 71.

[0094] [Processing Flow] Next, the processing performed in this embodiment will be described. Fig. 19 is a flowchart illustrating the processing performed when an image is displayed in the image display system 2.

[0095] When an instruction to switch to the image display mode is input from the user U, the CPU 21 of the smartphone 10 transitions to the image display mode (step SP21).

[0096] When the user U specifies an image to be displayed from among the images stored in the server 70, the CPU 21 transmits image specification information indicating the specified image to the server 70 (step SP22).

[0097] When the server 70 receives the image designation information from the smartphone 10 (step SS21), the server 70 transmits the designated image to the smart glasses 50 (step SS22).

[0098] The CPU 61 of the smart glasses 50 receives an image from the server 70 and executes an image acquisition process to acquire an image IM to be displayed on the display unit 64 (step SG21). Next, the CPU 61 adjusts the resolution of the acquired image to match the resolution of the display unit 64 of the smart glasses 50. In the initial state, the image is sized so that the entire image fits on the entire screen of the display unit 64. The CPU 61 adjusts the size of the image IM to the initial image size and displays it on the display unit 64 (step SG22).

[0099] Next, in step SP23, the CPU 21 of the smartphone 10 displays the display screen IV shown in Figure 12 on the touch panel 24, and enters a state in which it can accept instructions from the user U to change the display magnification and scroll the image displayed on the smart glasses 50.

[0100] When a first operation instruction for changing the display magnification of an image or a second operation instruction for scrolling the image IM is input (determined Yes in step SP24), the CPU 21 transmits these operation instructions to the server 70 (step SP25).

[0101] When the server 70 receives either the first operation instruction or the second operation instruction from the smartphone 10 (step SS23), the server 70 transfers this operation instruction to the smart glasses 50 (step SS24).

[0102] The CPU 61 of the smart glasses 50 receives the operation instruction from the server 70 and executes an operation instruction acquisition process to acquire the operation instruction as an instruction to change the display mode (step SG23). Next, the CPU 61 executes a display control process to change the display magnification or scroll the image IM in response to the operation instruction, and displays the image IM whose display mode has been changed by changing the display magnification or scrolling on the display unit 64 (step SG24).

[0103] In the smartphone 10, the above-described process of changing the display magnification of the image and scrolling is repeated until an instruction to end the image display is input from the user U (steps SP24 to SP26). In response to the process of the smartphone 10, the smart glasses 50 and the server 70 also repeat the process shown in FIG.

[0104] When the user U has finished checking the image, he / she inputs an instruction to end the image display to the smartphone 10. The instruction to end the image display is input, for example, by using a physical button (not shown) provided on the smartphone 10 or a back button (not shown) on the touch panel 24. When the instruction to end the image display is input (determined Yes in step SP26), the CPU 21 of the smartphone 10 transmits an instruction to end the display to the server 70 (step SP27) and ends the process.

[0105] When the server 70 receives the display end instruction from the smartphone 10 (step SS25), the server 70 transfers the display end instruction to the smart glasses 50 (step SS26).

[0106] When the CPU 61 of the smart glasses 50 receives the display end instruction from the server 70, it ends the display of the image on the display unit 64 (step SG25).

[0107] [Effects] In the present embodiment, the smart glasses 50, which are an example of a wearable monitor in the technology of the present disclosure, also function as a display control device in the technology of the present disclosure. In addition, the CPU 61 of the smart glasses 50 acquires an image IM from an external server 70 in an image acquisition process, and acquires a first operation instruction and a second operation instruction via the server 70 in an operation instruction acquisition process.

[0108] By adopting such an embodiment, images other than those captured by the smartphone 10 can also be acquired from the server 70 and displayed on the smart glasses 50.

[0109] Furthermore, when the smart glasses 50 display an image IM, all data acquired by the smart glasses 50 is acquired from the server 70. Therefore, when displaying an image, the smart glasses 50 do not need to communicate with the smartphone 10, and only need to communicate with the server 70, thereby reducing the load of communication processing between the smart glasses 50 and the smartphone 10.

[0110] [Third Embodiment] Next, a third embodiment of the present disclosure will be described. The image display system according to the third embodiment differs from the second embodiment in that the CPU 61 of the smart glasses 50 directly acquires the first operation instruction and the second operation instruction in the display control process, and the display end instruction when terminating the image display, from the smartphone 10 without going through the server 70 (see steps SP35, SP37, and steps SG33, SG35). The hardware configurations of the smartphone 10 and the smart glasses 50 are the same as those of the second embodiment, and therefore will not be described.

[0111] [Processing Flow] Next, the processing performed in this embodiment will be described. Fig. 20 is a flowchart illustrating the processing performed when an image is displayed in the image display system of this embodiment.

[0112] When an instruction to switch to the image display mode is input from the user U, the CPU 21 of the smartphone 10 transitions to the image display mode (step SP31).

[0113] When the user U specifies an image to be displayed from among the images stored in the server 70, the CPU 21 transmits image specification information indicating the specified image to the server 70 (step SP32).

[0114] When the server 70 receives the image designation information from the smartphone 10 (step SS31), the server 70 transmits the designated image to the smart glasses 50 (step SS32).

[0115] When the CPU 61 of the smart glasses 50 receives an image from the server 70, it executes an image acquisition process to acquire the image as an image to be displayed on the display unit 64 (step SG31). Next, the CPU 61 adjusts the resolution of the acquired image to match the resolution of the display unit 64 of the smart glasses 50. In the initial state, the image is sized so that the entire image fits entirely on the screen of the display unit 64. The CPU 61 adjusts the image to the initial image size and displays it on the display unit 64 (step SG32).

[0116] Next, in step SP33, the CPU 21 of the smartphone 10 displays the display screen IV shown in Figure 12 on the touch panel 24, and enters a state in which it can accept instructions from the user U to change the display magnification and scroll the image displayed on the smart glasses 50 (step SP33).

[0117] When a first operation instruction for changing the display magnification of the image or a second operation instruction for scrolling the image IM is input (determined Yes in step SP34), the CPU 21 transmits these operation instructions to the smart glasses 50 (step SP35).

[0118] The CPU 61 of the smart glasses 50 receives an operation instruction directly from the smartphone 10, and executes an operation instruction acquisition process to acquire the operation instruction as an instruction to change the display mode (step SG33). Next, the CPU 61 executes a display control process to change the display magnification or scroll the image IM in response to the operation instruction, and displays the image on the display unit 64 with the display mode changed by the change in display magnification or scrolling (step SP34).

[0119] In the smartphone 10, the above-described process of changing the display magnification of the image and scrolling is repeated until an instruction to end the image display is input from the user U (steps SP34 to SP36). In response to the process of the smartphone 10, the smart glasses 50 and the server 70 also repeat the process shown in FIG. 20 .

[0120] When the user U has finished checking the image, he / she inputs an instruction to end the image display to the smartphone 10. The instruction to end the image display is input, for example, by using a physical button (not shown) provided on the smartphone 10 or a back button (not shown) on the touch panel 24. When the instruction to end the image display is input (determined Yes in step SP36), the CPU 21 of the smartphone 10 transmits an instruction to end the display to the smart glasses 50 (step SP37) and ends the process.

[0121] When the CPU 61 of the smart glasses 50 receives the display end instruction directly from the smartphone 10, it ends the display of the image on the display unit 64 (step SG35).

[0122] [Effects] In the present embodiment, the smart glasses 50, which are an example of a wearable monitor in the technology of the present disclosure, also function as a display control device in the technology of the present disclosure. In addition, the CPU 61 of the smart glasses 50 acquires an image from an external server 70 in an image acquisition process, and acquires a first operation instruction and a second operation instruction directly from the smartphone 10 in an operation instruction acquisition process.

[0123] By adopting such an embodiment, images other than those captured by the smartphone 10 can also be acquired from the server 70 and displayed on the smart glasses 50.

[0124] Furthermore, the smart glasses 50 acquire the first operation instruction and the second operation instruction for performing display control processes such as changing the display magnification of an image and scrolling directly from the smartphone 10 without going through the server 70. Therefore, compared to when the first operation instruction and the second operation instruction are acquired through the server 70, the time lag (time lag due to communication delay) between when the user U inputs an operation instruction to the smartphone 10 and when the display mode is changed can be reduced.

[0125] [Fourth embodiment] Next, a fourth embodiment of the present disclosure will be described. The image display system according to the fourth embodiment differs from the first embodiment in that the initial position C0 of the center position when changing the display magnification of an image is changed in the display control process in the CPU 21 of the smartphone 10. In this embodiment, the hardware configurations of the smartphone 10 and the smart glasses 50 are the same as those in the first embodiment, and therefore description thereof will be omitted.

[0126] 21, the image file F1 of the image IM handled in this embodiment includes image data D1 as well as additional information data D2. The additional information data D2 includes information such as the date and time the image was taken and the location where the image was taken, as well as tilt angle information D2a, which is the angle of inclination of the smartphone 10 with respect to the horizontal direction when the image was taken.

[0127] 22 , in the technology disclosed herein, the tilt angle θtilt is specifically the angle between the optical axis direction Z (synonymous with the shooting direction) of the imaging lens 26a of the smartphone 10 and the horizontal direction H. The tilt angle θtilt is 0° when the optical axis direction Z faces the horizontal direction H, a positive angle when the optical axis direction Z faces above the horizontal direction H, and a negative angle when the optical axis direction Z faces below the horizontal direction H.

[0128] 23 is a diagram illustrating the initial position C0 of the center position when changing the display magnification of image IM. In FIG. 23, the coordinate in the X direction (i.e., the horizontal direction in FIG. 23) of image IM is designated as X, and the coordinate in the Y direction (i.e., the vertical direction in FIG. 23) of image IM is designated as Y. With regard to the two-dimensional coordinates (X, Y) of image IM, the lower left coordinate is designated as the origin (0, 0), and the upper right coordinate is designated as the vertex (100, 100). Furthermore, both the X and Y coordinates indicate values ​​normalized so that the maximum number of pixels in image IM is 100.

[0129] In the first embodiment, the coordinates of the initial position C0 of the center position when changing the display magnification of an image were set to (50, 50), which is the coordinate position of the center position of the image IM. In contrast, in the present embodiment, the CPU 21 of the smartphone 10 changes the coordinate position of the initial position C0 of the center position when changing the display magnification of an image from the default coordinate position (50, 50) based on the tilt angle information D2a included in the image file F1.

[0130] 24, when the tilt angle θtilt is 0, the coordinate position (50, 50) of the initial center position C0 is not changed from the default position. When the tilt angle θtilt is positive, the amount of change in the center position on the Y axis is changed linearly from 0 to 50 from greater than 0° up to 45°, and when the tilt angle exceeds 45°, the amount of change in the center position on the Y axis is set to 50.

[0131] Furthermore, when the tilt angle θtilt is negative, the amount of change in the center position on the Y axis is changed linearly from 0 to -50 from less than 0° up to -45°, and when the angle falls below -45°, the amount of change in the center position on the Y axis is set to -50. That is, when the tilt angle is positive, the upper part of the image IM becomes the initial position C0 of the center position when changing the display magnification of the image IM, and when the tilt angle is negative, the lower part of the image IM becomes the initial position C0 of the center position when changing the display magnification of the image IM. As an example, when the tilt angle θtilt is 22.5°, the coordinates of the initial position of the changed center position C0a are changed to (50, 75), as shown in FIG.

[0132] In this embodiment, the image file F1 of the image IM includes, in addition to the image data D1, information D2a on the tilt angle, which is the tilt angle of the smartphone 10 relative to the horizontal direction when the image IM was captured, as auxiliary information data D2. In this case, the CPU 21 of the smartphone 10 changes the initial position C0 of the center position when changing the display magnification of the image IM based on the tilt angle information D2a in the display control process.

[0133] For example, when taking a photograph of a distant landscape, if the photographing lens 26a of the smartphone 10 is pointed upward, an area closer to the sky will be captured, and if the zoom magnification is changed in this state, the zoom magnification will be changed with the sky as the center.

[0134] By adopting this embodiment, even if the image IM has already been captured, if the tilt angle at the time of capture was positive, that is, if the image IM was captured with the optical axis direction Z of the image capture device facing upward, the initial position C0 of the center position when changing the display magnification of the image IM is changed to the upper side of the image IM. Therefore, when the image is displayed, the user U can feel the sensation of a pseudo-zoom change at the time of capture.

[0135] [Fifth Embodiment] Next, a fifth embodiment of the present disclosure will be described. The image display system according to the fifth embodiment differs from the first embodiment in that the image IM is corrected in the horizontal direction when the image IM is displayed on the smart glasses 50. In this embodiment, the hardware configurations of the smartphone 10 and the smart glasses 50 are the same as those in the first embodiment, and therefore description thereof will be omitted.

[0136] 25, the image file F2 of the image IM handled in this embodiment has, in addition to the image data D1, additional information data D2. The additional information data D2 includes information such as the date and time the image was taken and the location where the image was taken, as well as roll angle information D2b, which is the angle of rotation around the optical axis direction Z (synonymous with the shooting direction) of the smartphone 10 relative to the horizontal direction when the image was taken.

[0137] 26 , in the technology of the present disclosure, the roll angle θroll is specifically the angle between the horizontal direction HP and the horizontal direction H of the smartphone 10 during image capture. When capturing an image with the smartphone 10, it is possible to switch between capturing a portrait image and capturing a landscape image by changing the angle of the housing 11 of the smartphone 10 with respect to the horizontal direction H. In either case, the direction of the smartphone 10 corresponding to the horizontal direction (longitudinal direction) of the image IM is defined as the horizontal direction HP.

[0138] When the shooting direction of the smartphone 10 is toward the back of the page in Fig. 26, the roll angle θroll is a positive angle when the right end of the smartphone 10 in Fig. 26 is higher than the horizontal direction H about the optical axis direction Z, i.e., when the smartphone 10 is rotating counterclockwise. Also, when the right end of the smartphone 10 in Fig. 26 is lower than the horizontal direction H about the optical axis direction Z ( Fig. 26 shows this state), i.e., when the smartphone 10 is rotating clockwise, it is a negative angle.

[0139] Fig. 26 shows a state in which a landscape image is captured with the smartphone 10. In this state, the longitudinal direction of the housing 11 of the smartphone 10 corresponds to the landscape direction HP of the smartphone 10. Fig. 26 also shows a state in which the roll angle θroll of the smartphone 10 when capturing an image is negative. When an image is captured in this state, as shown in Fig. 27, the horizontal direction HI of the subject in the image IM is rotated counterclockwise around the center CI of the image IM.

[0140] The CPU 21 of the smartphone 10 performs horizontal correction of the image IM when displaying the image on the smart glasses 50. For example, the horizontal correction can be performed by rotating the entire image IM by the same angle as the roll angle θroll around the center CI of the image IM as the axis. In this case, the direction in which the image IM is rotated counterclockwise is considered to be a positive angle. Furthermore, the direction in which the image IM is rotated clockwise is considered to be a negative angle.

[0141] As an example, in the case of an image IM captured with a roll angle θroll of −20°, the horizontal direction HI of the subject in the image IM is rotated 20° counterclockwise around the center CI of the image IM as an axis, as shown in Fig. 27. Therefore, by rotating the entire image IM by −20°, the same angle as the roll angle θroll, around the center CI of the image IM as an axis, as shown in Fig. 28, the horizontal direction HI of the subject in the image IM can be made to coincide with the left-right direction of the image IM.

[0142] By adopting this configuration, even if the image is taken in an inclined position, the horizontal direction of the subject can be aligned with the left-right direction of the image IM, so that an image that does not cause the user U to feel uncomfortable can be displayed on the smart glasses 50.

[0143] [Sixth embodiment] Next, a sixth embodiment of the present disclosure will be described. The image display system according to the sixth embodiment is configured to match the amount of change in magnification in response to the amount of magnification change operation during shooting with the amount of magnification change operation during image display, particularly when the shooting device used to shoot an image is different from the shooting device used to input operation instructions such as image magnification change and scrolling.

[0144] The present embodiment is different from the first embodiment in the content of the display control process in the CPU 21 of the smartphone 10. In the present embodiment, the hardware configurations of the smartphone 10 and the smart glasses 50 are the same as those in the first embodiment, and therefore, description thereof will be omitted.

[0145] 29, the image file F3 of the image handled in this embodiment includes image data D1 as well as incidental information data D2. The incidental information data D2 includes information such as the date and time the image was taken and the location where the image was taken, as well as magnification change operation related information D2c relating to the amount of change in magnification in response to the amount of magnification change operation performed when taking the image using the camera that took the image. In this embodiment, as an example of this magnification change operation related information D2c, a case will be described in which information on the maximum zoom magnification of the smartphone that took the image is recorded.

[0146] The images stored in the storage 23 of the smartphone 10 of the image display system are not necessarily images captured by the smartphone 10. If the maximum zoom magnification of the smartphone that captured the image differs from the maximum zoom magnification of the smartphone 10, the operational feel when changing the zoom magnification will differ between the smartphone that captured the image and the smartphone 10.

[0147] In such a case, the CPU 21 of the smartphone 10 acquires information on the maximum zoom magnification of the smartphone that captured the image as magnification change operation related information D2c from the supplementary information data D2 of the image file F3 of the image to be displayed.

[0148] 30 is a graph showing the relationship between the swipe amount (i.e., the operation amount) and the magnification in the zoom magnification change operation. Here, as an example, a case will be described in which the maximum zoom magnification of the smartphone 10 is 5x and the maximum zoom magnification of the smartphone that captured the image is 10x.

[0149] The state in which the lower limit value of the magnification specification line 30b of the zoom magnification input unit 30 in the smartphone 10 is superimposed on the magnification display unit 30a is set as the reference position (i.e., swipe amount 0), and graph R1 shows the relationship between the amount of change in magnification and the amount of swipe from the reference position. Also, the state in which the lower limit value of the magnification specification line of the zoom magnification input unit in the smartphone that captured the image is superimposed on the magnification display unit is set as the reference position (i.e., swipe amount 0), and graph R2 shows the relationship between the amount of change in magnification and the amount of swipe from the reference position.

[0150] As described above, the maximum zoom magnification of the smartphone that captured the image is 10x, and the maximum zoom magnification of smartphone 10 is 5x, so even if the same amount of operation is performed to change the zoom magnification, the amount of change in zoom magnification will not match.

[0151] Therefore, based on information about the maximum zoom magnification of the smartphone that captured the image, the CPU 21 of the smartphone 10 performs processing to match the amount of change in zoom magnification in response to the amount of operation of the zoom magnification change operation on the smartphone that captured the image with the amount of change in zoom magnification in response to the operation of the zoom magnification change operation on the smartphone 10. Specifically, as shown in graph R1a in Fig. 30 , the display magnification at one end of the magnification designation line 30b is set to 1x, the display magnification at the other end of the magnification designation line 30b is set to 10x, the same as the maximum zoom magnification of the smartphone that captured the image, and the display magnification in the middle portion is set to a magnification between 1x and 10x depending on the distance from the one end.

[0152] Also, as shown in FIG. 31, in the zoom magnification input unit 30, the display magnification at one end of the magnification specification line 30b is set to 1x, the display magnification at the other end of the magnification specification line 30b is set to 10x, and the display magnification at the intermediate portion is set to a magnification between 1x and 10x depending on the distance from one end.

[0153] By adopting this configuration, when the maximum zoom magnification of the smartphone that captured the image is different from the maximum zoom magnification of smartphone 10, the operating feel when changing the zoom magnification can be made the same between the smartphone that captured the image and smartphone 10.

[0154] In addition, if the size of the touch panel of the smartphone used to take the image is different from the size of the touch panel 24 of the smartphone 10, the operating feel when changing the zoom magnification will not be exactly the same between the smartphone used to take the image and the smartphone 10.

[0155] Furthermore, if the GUI used to change the zoom magnification of the smartphone used to take the image is different from the GUI used to change the zoom magnification of smartphone 10, the operating feel when changing the zoom magnification will not be exactly the same between the smartphone used to take the image and smartphone 10.

[0156] However, even in these cases, by adopting the configuration of this embodiment, the operational feel when changing the zoom magnification on the smartphone 10 can be made closer to the operational feel of the smartphone that captured the image.

[0157] [Seventh embodiment] Next, a seventh embodiment of the present disclosure will be described. The image display system according to the seventh embodiment is configured to match the amount of change in magnification in response to the amount of magnification change operation during shooting with the amount of magnification change operation during image display, particularly when the shooting device used to shoot an image is different from the shooting device used to input operation instructions such as image magnification change and scrolling.

[0158] This embodiment is different from the sixth embodiment in that the interface for changing the zoom magnification on the smartphone 10 is different. Also, the content of the display control process in the CPU 21 of the smartphone 10 is different. In this embodiment, the hardware configurations of the smartphone 10 main body and the smart glasses 50 are the same as those in the first embodiment, and therefore, description thereof will be omitted.

[0159] As shown in FIGS. 32 and 33, in this embodiment, the smartphone 10 is combined with a smartphone cover 15.

[0160] The smartphone cover 15 includes a cover unit 16 and a ring-shaped controller 18. The cover unit 16 engages with the smartphone 10. The cover unit 16 has an opening 17 formed therein for exposing the photographing lens 26a of the smartphone 10 when the smartphone 10 is engaged. In addition, the ring-shaped controller 18 is attached to the back surface 16b of the cover unit 16 so as to be rotatable relative to the cover unit 16.

[0161] The ring controller 18 is rotatable relative to the cover unit 16 in the same manner as the zoom ring of a typical digital camera, and functions as an interface for changing the zoom magnification and the display magnification of the smartphone 10. The ring controller 18 has an internal detection unit (not shown) that detects the rotation direction and rotation angle of the ring controller 18. Signals indicating the rotation direction and rotation angle of the ring controller 18 detected by the detection unit are transmitted to the smartphone 10 by a communication unit (not shown). A standard such as Bluetooth (registered trademark) is used for this communication unit.

[0162] 34, the image file F4 of the image handled in this embodiment has image data D1 as well as incidental information data D2. The incidental information data D2 includes information such as the date and time the image was taken and the location where the image was taken, as well as magnification change operation related information D2d relating to the amount of change in magnification relative to the amount of magnification change operation performed when taking the image using the photographing device. In this embodiment, as an example of this magnification change operation related information D2d, a case will be described in which information on the amount of magnification change relative to the amount of change in rotation angle of the zoom ring of the digital camera used to take the image and information on the maximum zoom magnification of the digital camera are recorded.

[0163] The images stored in the storage 23 of the smartphone 10 of the image display system are not necessarily images captured by the smartphone 10. If the image capturing device used to capture the images is a digital camera, the operational feel when changing the zoom magnification will be different between the digital camera and the smartphone 10.

[0164] Generally, in a digital camera, the zoom magnification is changed by rotating a zoom ring, but in the smartphone 10, the zoom magnification is changed by swiping on the touch panel 24.

[0165] The smartphone cover 15 is a cover that allows the zoom magnification to be changed on the smartphone 10 with the same feel as when using a digital camera.

[0166] When the CPU 21 of the smartphone 10 detects that the smartphone cover 15 is attached to the smartphone 10 , it causes the ring-shaped controller 18 of the smartphone cover 15 to function as an interface for changing the zoom magnification of the smartphone 10 .

[0167] The CPU 21 of the smartphone 10 may detect that the smartphone cover 15 is attached to the smartphone 10, for example, by detecting that communication has been established between the communication unit 25 of the smartphone 10 and the communication unit of the ring-type controller 18.

[0168] In addition, the CPU 21 of the smartphone 10 acquires, from the supplementary information data D2 of the image file F4 of the image to be displayed, information on the amount of magnification change relative to the amount of change in the rotation angle of the zoom ring of the digital camera that captured the image, and information on the maximum zoom magnification of the digital camera, as magnification change operation related information D2d.

[0169] 35 is a graph showing the relationship between the amount of rotation (i.e., the amount of operation) of the zoom ring or ring-type controller 18 and the magnification during a zoom magnification change operation. Here, as an example, a case will be described in which, for the smartphone 10, the zoom magnification increases by 1 for every 90° clockwise rotation of the ring-type controller 18, with the maximum zoom magnification being 5x. Also, for the digital camera that captured the image, the zoom magnification increases by 1 for every 40° clockwise rotation of the zoom ring, with the maximum zoom magnification being 10x.

[0170] Graph R10 shows the relationship between the amount of change in zoom magnification and the amount of rotation (i.e., the amount of operation) from a reference position (i.e., the amount of rotation 0) of the ring controller 18 on the smartphone 10. Also, the minimum magnification position of the zoom ring on the digital camera that captured the image is set as the reference position (i.e., the amount of rotation 0), and graph R20 shows the relationship between the amount of change in zoom magnification and the amount of rotation (i.e., the amount of operation) from the reference position.

[0171] The ring-type controller 18 has no limit to its rotation and can be rotated freely in the same direction any number of times. Therefore, the reference position of the ring-type controller 18 may be, for example, the position of the ring-type controller 18 at the time when an image in the initial display state is displayed on the display unit 64.

[0172] As described above, the amount of change in magnification relative to the amount of change in the rotation angle of the zoom ring of the digital camera that captured the image is different from that of the ring-type controller 18 attached to the smartphone 10. Furthermore, the maximum zoom magnification of the digital camera that captured the image is also different from that of the smartphone 10. Therefore, even if the same amount of operation is performed in the zoom magnification change operation, the amount of change in zoom magnification will not match.

[0173] Therefore, based on information about the amount of magnification change relative to the amount of change in the rotation angle of the zoom ring of the digital camera that captured the image and information about the maximum zoom magnification of the digital camera, the CPU 21 of the smartphone 10 performs processing to match the amount of zoom magnification change relative to the amount of zoom magnification change operation on the digital camera that captured the image with the amount of zoom magnification change relative to the operation of the zoom magnification change operation on the smartphone 10. Specifically, as shown in graph R10a in Fig. 30 , the operation input of the ring controller 18 attached to the smartphone 10 is set to increase the zoom magnification by 1 for every 40° clockwise rotation, and the maximum zoom magnification is set to 10.

[0174] By adopting such an embodiment, the operational feel when changing the zoom magnification on the smartphone 10 can be made closer to the operational feel of the digital camera that captured the image.

[0175] Eighth Embodiment Next, an eighth embodiment of the present disclosure will be described. The image display system according to the eighth embodiment differs from the first embodiment in that the display control process in the CPU 21 of the smartphone 10 corrects the scroll amount when scrolling an image in response to a second operation instruction. In this embodiment, the hardware configurations of the smartphone 10 and the smart glasses 50 are the same as those in the first embodiment, and therefore description thereof will be omitted.

[0176] 36, the image file F5 of the image IM handled in this embodiment has, in addition to image data D1, additional information data D2. The additional information data D2 includes information such as the date and time the image was taken and the location where the image was taken, as well as field-of-view information D2e relating to the field-of-view of the smartphone 10 when the image was taken.

[0177] The angle of view information D2e may be any information that can identify the angle of view of the smartphone 10 at the time of capturing an image, and may be information on the angle of view itself or information on the zoom magnification corresponding to the angle of view. When acquiring information on the zoom magnification, information on the angle of view corresponding to the zoom magnification may be acquired separately. Here, as an example, the angle of view information D2e is the angle of view of the captured area.

[0178] 37, the angle of view θavw at the wide-angle end (i.e., zoom magnification is 1x) of the smartphone 10 when capturing an image is five times larger than the angle of view θavt at the telephoto end (i.e., zoom magnification is 5x). Furthermore, the larger the angle of view when capturing an image, the larger the tilt angle of the smartphone 10 required to change the orientation of the photographing lens 26a from one end to the other of the capturing range. As in the above example, when the angle of view is five times larger, the tilt angle of the smartphone 10 required to change the orientation of the photographing lens 26a from one end to the other of the capturing range is five times larger.

[0179] When the CPU 21 of the smartphone 10 executes a display control process for scrolling an image displayed on the display unit 64 of the smart glasses 50 in response to a second operation instruction, the CPU 21 corrects the amount of scrolling of the image based on the angle of view information D2e at the time of shooting.

[0180] Specifically, the amount of scrolling of the image is corrected so that the tilt angle of the smartphone 10 required to scroll the image from one end to the other when displaying the image matches the tilt angle of the smartphone 10 required to change the orientation of the photographing lens 26a from one end to the other of the photographing range when photographing the image.

[0181] By adopting such an embodiment, it is possible to reproduce, when scrolling through images, a sensation similar to the change in the photographic area that occurs when the smartphone 10 is tilted when photographing an image.

[0182] [Modifications] The present disclosure has been described above based on the preferred embodiments, but embodiments to which the present disclosure can be applied are not limited to the above-described embodiments.

[0183] For example, in the above embodiment, a case has been described in which the user U does not wear the smart glasses 50 when capturing an image, but instead captures the image using only the smartphone 10. However, the user U may capture the image while wearing the smart glasses 50. In this case, as shown in FIG. 38 , setting information of the smartphone 10 at the time of capturing the image may be displayed as an auxiliary display in the image IX displayed in a part of the field of view of the right eye. Note that, regarding the content to be displayed in the image IX, in addition to the setting information at the time of capturing the image, a live view of the smartphone 10 may be displayed, or the live view of the smartphone 10 and the setting information may be superimposed on each other.

[0184] Furthermore, the smart glasses 50 are not limited to those that display images in part of the field of view of the right eye, but may also display images in part of the field of view of the left eye, part of the field of view of both eyes, or the entire field of view of both eyes.

[0185] Furthermore, the wearable monitor in the technology of the present disclosure is not limited to smart glasses, and may be other display devices such as an HMD.

[0186] Furthermore, the photographing device in the technology of the present disclosure is not limited to a smartphone, but may be a digital camera having the same functions as the smartphone 10 described above.

[0187] Furthermore, the processes executed by the CPU 21 and CPU 61 after reading the software (programs) in each of the above embodiments may be executed by various processors other than CPUs. Examples of such processors include programmable logic devices (PLDs) whose circuit configuration can be changed after manufacture, such as field-programmable gate arrays (FPGAs), and dedicated electrical circuits, such as application-specific integrated circuits (ASICs), which are processors having a circuit configuration specifically designed to execute specific processes. Each process may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor devices.

[0188] In addition, in each of the above embodiments, the display control program is described as being pre-stored (installed) in storage 23 or storage 63, but this is not limiting. The program may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network.

[0189] ROM is a broad concept that includes rewritable ROM such as flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (Electrically Erasable Programmable Read Only Memory).

[0190] In the first to fifth embodiments, the CPU is provided integrally within the wearable device or the smartphone, but the CPU may be provided on a network connected to the wearable device or the smartphone, in an operation unit connected to the network, or in a cloud computing unit connected to the network.

[0191] When the CPU is located on a network or on a device connected to the network, the network is managed by a cloud system, and it is desirable that the CPU connected to the network operate virtually as a CPU built into a wearable device or smartphone, while being connected via the network using cloud computing technology.

[0192] The above embodiment describes a case where the need to change the display magnification or scroll arises due to a difference in resolution between the captured image and the displayed image. In addition, the need to change the display magnification or scroll arises due to a difference in aspect ratio, such as in panoramic photography, or due to a difference between the resolution of the displayed image and the resolution of an image generated by interpolation or rendering, rather than the resolution of the captured image itself. In these cases, the same effect as in the above embodiment can be achieved.

[0193] In the above embodiment, the wearable monitor, which is an example of a wearable device equipped with a monitor, does not have a camera, but it may also have a camera and be capable of taking pictures on its own without being combined with an imaging device. In this case, it is preferable that when the wearable monitor is combined with an imaging device, it can also take pictures with the camera on the imaging device side.

[0194] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0195] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0196] DESCRIPTION OF SYMBOLS 1, 2 Image display system 10 Smartphone 10 The smartphone 11 Housing 11a Surface 11b Back 15 Smartphone cover 16 Cover section 16b Back 17 Opening 18 Ring-shaped controller 21 CPU 22 Memory 23 Storage 24 Touch panel 24a Zoom magnification input area 24b Display magnification input area 25 Communication section 26 Shooting section 26a Shooting lens 27 Orientation detection section 28 Bus 30 Zoom magnification input section 30a Magnification display section 30b Magnification specification line 31 Shutter button 32 Live view display section 50 Smart glasses 51 Frame 52 Lens 61 CPU 62 Memory 63 Storage 64 Display section 65 Communication section 66 Bus 70 Server 71 Network C0, C0a Center position CI Center D1 Image data D2 Supplementary information data D2a to D2e information DA Area F1 to F5 Image file H, H1 Horizontal direction HP Lateral direction IM Image IP Shooting screen IV Display screen IX Image R Radius U User Z Optical axis direction

Claims

1. A display control device for a wearable device having a monitor, comprising at least one processor and a memory built into or connected to the processor, wherein the processor executes: an image acquisition process for acquiring an image to be displayed on the monitor; a first operation instruction acquisition process for acquiring a first operation instruction input through an operation on an operation unit of an image capture device separate from the wearable device in order to change the display magnification of the image, the first operation instruction being input through a magnification change operation on the image capture device; and a display control process for changing the display magnification of the image in accordance with the first operation instruction.

2. The display control device according to claim 1, wherein the magnification change operation is a zoom magnification change operation performed when taking a picture using the image capture device.

3. A display control device according to claim 1 or 2, wherein the processor controls the amount of change in the display magnification of the image in accordance with the amount of change in magnification relative to the amount of operation of the magnification change operation in the display control process.

4. A display control device as claimed in any one of claims 1 to 3, wherein the processor further executes a second operation instruction acquisition process to acquire a change in the attitude of the photographing device detected by an attitude detection unit that detects the attitude of the photographing device as a second operation instruction, and in the display control process, scrolls the image in accordance with the second operation instruction.

5. A display control device according to any one of claims 1 to 4, wherein when the photographing device is equipped with a touch panel that functions as the operation unit, and the magnification change operation when photographing with the photographing device is a swipe operation within a first area on the touch panel, the processor detects a swipe operation within a second area on the touch panel that includes the first area and is larger than the first area as an operation to change the display magnification of the image.

6. The display control device according to claim 5, wherein the processor detects an arc-shaped swipe operation on the touch panel with a radius of 3 cm to 8 cm as the magnification change operation.

7. A display control device according to any one of claims 1 to 6, wherein the image is an image captured by the imaging device.

8. A display control device as described in any one of claims 1 to 7, wherein the processor is mounted on the wearable device, and the processor: in the image acquisition process, acquires the image from an external server; and in the first operation instruction acquisition process, acquires the first operation instruction input through an operation on an operation unit of the imaging device via the server.

9. A display control device as described in any one of claims 1 to 7, wherein the processor is mounted on the wearable device, and in the image acquisition process, the processor acquires the image from an external server, and in the first operation instruction acquisition process, the processor acquires the first operation instruction input through an operation on an operation unit of the imaging device directly from the imaging device.

10. A display control device according to any one of claims 1 to 9, wherein the image includes information on a tilt angle, which is the angle of inclination of the image capture device relative to the horizontal direction when the image was captured, and the processor, in the display control process, changes the initial position of the center position when changing the display magnification of the image based on the tilt angle.

11. A display control device according to any one of claims 1 to 10, wherein the image includes information on a roll angle, which is a rotation angle relative to the horizontal direction around the optical axis of the imaging device when the image was captured, and the processor, in the display control process, corrects the horizontal direction of the image based on the roll angle.

12. A display control device as described in any one of claims 1 to 11, wherein the image includes magnification change operation related information regarding the amount of change in magnification relative to the amount of operation of the magnification change operation when photographing the image using a photographing device that photographed the image, and the processor, in the display control processing, matches the amount of change in magnification relative to the amount of operation when photographing the image with the amount of change in magnification relative to the amount of operation when changing the display magnification of the image based on the magnification change operation related information.

13. A display control device as described in claim 4, wherein the image includes angle of view information regarding the angle of view of the imaging device when the image was captured, and the processor, in the display control process, corrects the amount of scrolling when scrolling the image in response to the second operation instruction based on the angle of view information.

14. A display control method for a wearable device having a monitor, comprising: an image acquisition processing step for acquiring an image to be displayed on the monitor; a first operation instruction acquisition processing step for acquiring a first operation instruction input through an operation on an operation unit of an image capturing device separate from the wearable device in order to change the display magnification of the image, the first operation instruction being input through a magnification change operation on the image capturing device; and a display control processing step for changing the display magnification of the image in accordance with the first operation instruction.

15. A display control program for a wearable device having a monitor, the display control program causing a computer to execute the following steps: an image acquisition processing step for acquiring an image to be displayed on the monitor; a first operation instruction acquisition processing step for acquiring a first operation instruction input through an operation on an operation unit of an image capture device separate from the wearable device in order to change the display magnification of the image, the first operation instruction being input through a magnification change operation on the image capture device; and a display control processing step for changing the display magnification of the image in accordance with the first operation instruction.

Images