Display control device, display control method, and display control program

Abstract

The present application provides a display control device, a display control method and a display control program that improve convenience. A processor of the display control device performs: an image acquisition process that acquires an image displayed on a monitor; a first operation instruction acquisition process that acquires a first operation instruction for changing a display magnification of the image, the first operation instruction being input by an operation on an operation section of a photographing device that is separate from a wearable device and being input by a magnification change operation in the photographing device; and a display control process that changes the display magnification of the image in accordance with the first operation instruction.

Description

Technical Field

[0001] The present invention relates to a display control device, a display control method, and a display control program. Background Technology

[0002] In recent years, wearable monitors such as HMDs (Head-Mounted Displays) have been developed as monitors worn on the user's head. Furthermore, unlike conventional HMDs, products called smart glasses have been released that use small LCD (liquid crystal display) projectors to display information in front of the eyes, resulting in a smaller and lighter appearance compared to conventional HMDs, and are about the same size or slightly larger than eyeglasses. These smart glasses can display the monitor's image while simultaneously transmitting light, or display it within a portion of the field of vision. It is believed that such products will become the mainstream of wearable monitors in the future. Such a wearable monitor is an example of a wearable device with a monitor. A display control device has been proposed that allows image scrolling when displaying a photographed image on the monitor of a wearable device using a photographic device (for example, see Patent Document 1). Patent Document 1 describes scrolling of the image displayed on the monitor of a wearable device based on the tilt of the smartphone.

[0003] Previous technical documents

[0004] Patent documents

[0005] Patent Document 1: International Publication No. 2014 / 185146 Summary of the Invention

[0006] The present invention provides a display control device, display control method, and display control program that improve convenience.

[0007] means for solving technical problems

[0008] The display control device of the present invention is a display control device for a wearable device having a monitor, comprising: at least one processor and memory built into or connected to the processor, the processor performing the following processes: image acquisition processing to acquire an image displayed on the monitor; first operation instruction acquisition processing to acquire a first operation instruction for changing the display magnification of the image, the first operation instruction acquisition processing being input by an operation of an operation unit of a camera device separate from the wearable device and by a magnification change operation in the camera device; and display control processing to change the display magnification of the image according to the first operation instruction.

[0009] The zoom level change operation can be performed when shooting with a camera device to change the zoom level.

[0010] Furthermore, the processor can perform the following processing: in the display control processing, control the change in the display magnification of the image based on the change in the magnification relative to the operation amount of the magnification change operation.

[0011] Furthermore, the processor can also perform the following processing: a second operation instruction acquisition process that acquires the change in the posture of the camera device detected by the posture detection unit of the camera device as a second operation instruction; and in the display control processing, the image can be scrolled according to the second operation instruction.

[0012] Furthermore, when the camera device has a touch panel that functions as an operation unit, if the magnification change operation during shooting is a swipe operation in the first area of ​​the touch panel, the processor can detect the swipe operation in the second area of ​​the touch panel that includes the first area and is larger than the first area as an operation to change the display magnification of the image.

[0013] Furthermore, the processor can detect circular sliding operations with a radius of 3cm or more but less than 8cm on the touch panel as scaling operations.

[0014] Furthermore, the image can be one captured by a photographic device.

[0015] Furthermore, the processor can be mounted on a wearable device. In image acquisition processing, the processor can acquire images from an external server, and in the first operation instruction acquisition processing, it can acquire the first operation instruction input by operating the operation unit of the camera device via the server.

[0016] Furthermore, the processor can be mounted on a wearable device. In image acquisition processing, the processor can acquire images from an external server, and in the first operation instruction acquisition processing, it can directly acquire the first operation instruction input by the operation of the operation unit of the camera device.

[0017] Furthermore, the image can contain information about the tilt angle, i.e., the pitch angle, of the photographic device relative to the horizontal direction when the image is captured. The processor can determine the initial position of the center position when changing the display magnification of the image based on the pitch angle during the display control process.

[0018] Furthermore, the image can contain information about the rotation angle of the photographic device around the optical axis relative to the horizontal direction when the image is captured, i.e., the roll angle. The processor can correct the horizontal direction of the image based on the roll angle during the display control process.

[0019] Furthermore, the image may contain magnification change operation related information, which relates to the change in the magnification of the operation amount relative to the magnification change operation at the time of shooting in the photographic device that took the image. The processor may, in the display control processing, make the change in the magnification of the operation amount relative to the time of shooting consistent with the change in the magnification of the operation amount relative to the display magnification of the image, based on the magnification change operation related information.

[0020] Furthermore, the image may contain perspective information related to the viewing angle of the photographic device when the image is captured, and the processor may correct the amount of scrolling when scrolling the image according to the second operation instruction based on the perspective information during the display control processing.

[0021] The display control method of the present invention is a display control method for a wearable device having a monitor, comprising the following steps: an image acquisition processing step for acquiring an image displayed on the monitor; a first operation instruction acquisition processing step for acquiring a first operation instruction in order to change the display magnification of the image, the first operation instruction being input by an operation of an operation unit of a camera device separate from the wearable device and by a magnification change operation in the camera device; and a display control processing step for changing the display magnification of the image according to the first operation instruction.

[0022] The display control program of the present invention is a display control program for a wearable device with a monitor, which causes a computer to perform the following steps: an image acquisition processing step, acquiring an image displayed on the monitor; a first operation instruction acquisition processing step, acquiring a first operation instruction in order to change the display magnification of the image, the first operation instruction being input by operation of the operation unit of the camera device separate from the wearable device and by magnification change operation in the camera device; and a display control processing step, changing the display magnification of the image according to the first operation instruction. Attached Figure Description

[0023] Figure 1 This is a schematic structural diagram of an image display system including a display control device based on the first embodiment.

[0024] Figure 2 It is a block diagram representing the hardware structure of a smartphone.

[0025] Figure 3 This is a functional block diagram of a smartphone.

[0026] Figure 4 This is an image of the smart glasses.

[0027] Figure 5 This is a diagram used to illustrate the user's field of vision as seen through smart glasses.

[0028] Figure 6This is a block diagram representing the hardware structure of smart glasses.

[0029] Figure 7 It is a graph representing the user's state at the time the image was captured.

[0030] Figure 8 This is a flowchart illustrating the image capture process in the first embodiment.

[0031] Figure 9 This is an example of a photographic image displayed on a touch panel.

[0032] Figure 10 It is a graph representing the user's state when the image is displayed.

[0033] Figure 11 This is a flowchart illustrating the processing of image display in the first embodiment.

[0034] Figure 12 This is an example of a display screen shown on a touch panel.

[0035] Figure 13 It is a graph showing the relationship between the amount of sliding and the magnification when performing zoom and display magnification changes.

[0036] Figure 14 It is a diagram used to illustrate the sliding operation in the display screen.

[0037] Figure 15 This is a diagram representing the magnification of an image displayed on smart glasses from its initial display state.

[0038] Figure 16 This indicates that the image displayed on the smart glasses is from Figure 15 The diagram shows the state as it scrolls to the right.

[0039] Figure 17 This indicates that the image displayed on the smart glasses is from Figure 15 The diagram shows the state of scrolling upwards.

[0040] Figure 18 This is a schematic structural diagram of an image display system including a display control device based on the second embodiment.

[0041] Figure 19 This is a flowchart illustrating the image display process in the second embodiment.

[0042] Figure 20 This is a flowchart illustrating the image display process in the third embodiment.

[0043] Figure 21 This is a diagram illustrating the structure of the image file in the fourth embodiment.

[0044] Figure 22 This is a diagram illustrating the pitch angle in the technology of this invention.

[0045] Figure 23 This is a diagram used to illustrate the initial position of the center when changing the display magnification.

[0046] Figure 24 It is a chart showing the relationship between the pitch angle and the change in the position of the Y-axis center.

[0047] Figure 25 This is a diagram illustrating the structure of the image file in the fifth embodiment.

[0048] Figure 26 This is a diagram illustrating the roll angle in the technology of this invention.

[0049] Figure 27 This is an image showing a smartphone taken at an angle.

[0050] Figure 28 This is a diagram showing the state of an image taken when the smartphone is tilted.

[0051] Figure 29 This is a diagram illustrating the structure of the image file in the sixth embodiment.

[0052] Figure 30 It is a graph showing the relationship between the amount of sliding and the magnification in the touch panel.

[0053] Figure 31 This is an example of a display screen shown on a touch panel.

[0054] Figure 32 This is a frontal view of a smartphone with a smartphone case.

[0055] Figure 33 This is a view of a smartphone with a smartphone case on, viewed from the back.

[0056] Figure 34 This is a diagram illustrating the structure of the image file in the seventh embodiment.

[0057] Figure 35 It is a graph showing the relationship between the rotation of the zoom ring or ring controller and the magnification.

[0058] Figure 36 This is a diagram illustrating the structure of the image file in the eighth embodiment.

[0059] Figure 37 It is a diagram illustrating the relationship between viewing angle and the tilt angle of a smartphone.

[0060] Figure 38 This is an example of an image displayed on smart glasses during the shooting process. Detailed Implementation

[0061] [First Implementation]

[0062] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Figure 1 This is a schematic structural diagram of an image display system including a display control device based on the first embodiment of the present invention. Figure 1 The image display system 1 shown includes a smartphone 10 and smart glasses 50. By using the smart glasses 50 as an external display for the smartphone 10, images that can be displayed by the smartphone 10 can be displayed on the smart glasses 50. In the image display system 1, as shown below, the display magnification and scrolling of the images displayed on the smart glasses 50 can be changed through operation of the smartphone 10.

[0063] like Figure 1 As shown, in the image display system 1, as an example, the smartphone 10 and smart glasses 50 are wirelessly connected. The smartphone 10 is an example of a photographic device in the technology of the present invention. The smart glasses 50 is a glasses-type wearable computer, a so-called wearable monitor. A wearable monitor is an example of a wearable device that is worn by a user U and has a monitor that displays images within the user U's field of vision.

[0064] As is well known, the smartphone 10 is a mobile terminal that functions as a mobile computer and a mobile phone. The smartphone 10 has a tablet-shaped frame 11. A touch panel 24 is disposed on one side of the frame 11, and a photographic lens 26a is disposed on the side 11b opposite to the side where the touch panel 24 is disposed. Furthermore, in the following description, for convenience, the side of the smartphone 10 frame 11 where the touch panel 24 is disposed will be referred to as the front side 11a (see reference). Figure 8 The side with the photographic lens 26a will be described as the back side 11b.

[0065] like Figure 2 As shown, the smartphone 10 includes a CPU (Central Processing Unit) 21, memory 22, storage 23, a touch panel 24, a communication unit 25, a camera unit 26, and a gesture detection unit 27. These components are interconnected via a bus 28.

[0066] CPU 21 oversees and controls all parts of smartphone 10 by executing control programs. Memory 22 is operating memory, for example, composed of RAM (Random Access Memory). CPU 21 reads the control program from memory 23 into memory 22 and executes the control program using memory 22 as an operating area. CPU 21 performs control and various processing on the aforementioned structures according to the control program.

[0067] CPU 21 is an example of a processor in the technology of the present invention. Memory 22 is an example of memory in the technology of the present invention. The smartphone 10 equipped with CPU 21 and memory 22 also functions as a display control device in the technology of the present invention.

[0068] The memory 23 stores various programs, including operating system control programs and application programs, as well as various data, including image data. The memory 23 is, for example, composed of non-volatile memory such as flash memory. In this embodiment, the memory 23 stores a display control program PG, which is one of the various programs.

[0069] The touch panel 24 has the functions of displaying various images and receiving touch input operations.

[0070] The communication unit 25 is an interface for communication between the smartphone 10 and the smart glasses 50 and other devices, such as using standards like Wi-Fi (registered trademark) and Routertooth (registered trademark).

[0071] The photography unit 26 includes a photographic lens 26a and an imaging element (not shown). The photography unit 26 acquires image data of the subject by taking pictures of the subject.

[0072] The posture detection unit 27 is a component used to detect the posture of the smartphone 10, for example, using a gyroscope sensor.

[0073] like Figure 3 As shown, the CPU 21 functions as an image acquisition processing unit 21a, an operation instruction acquisition processing unit 21b, and a display control processing unit 21c by executing the display control program PG stored in the memory 23.

[0074] The image acquisition and processing unit 21a performs image acquisition processing to acquire the image displayed on the smart glasses 50.

[0075] The operation instruction acquisition processing unit 21b performs a first operation instruction acquisition process to acquire a first operation instruction for changing the display magnification of an image. This first operation instruction is input through an operation on the operation unit of the smartphone 10, which is separate from the smart glasses 50, and through a magnification change operation in the smartphone 10. As an example, the first operation instruction is the same operation as the zoom magnification change operation when taking a picture using the smartphone 10, and is a swipe operation on the touch panel 24.

[0076] Furthermore, the operation instruction acquisition processing unit 21b performs a second operation instruction acquisition process, which acquires the change in the posture of the smartphone 10 detected by the posture detection unit 27, namely the change in the orientation of the camera lens 26a of the smartphone 10, as the second operation instruction.

[0077] The display control processing unit 21c performs processing to change the display magnification of the image displayed on the smart glasses 50 according to the first operation instruction. Furthermore, the display control processing unit 21c performs processing to scroll the image displayed on the smart glasses 50 according to the second operation instruction.

[0078] like Figure 4 As shown, the smart glasses 50 are wearable monitors in the form of eyeglasses, comprising a frame 51, lenses 52 positioned in front of the eyes and covering both eyes, and a display unit 64 disposed within the lenses 52 at a position corresponding to the right eye. As an example, the display unit 64 projects the display light of the image directly onto the retina. Figure 5 The image shown is displayed on a retinal projection type display unit that places the image IM in a portion of the right eye's visual field. Figure 5 This diagram schematically illustrates the state of the image IM projected onto the field of vision of user U wearing smart glasses 50. To user U, the image IM appears as if it were projected onto lens 52.

[0079] like Figure 6 As shown, the smart glasses 50 includes a CPU 61, a memory 62, a storage unit 63, a display unit 64, and a communication unit 65. These components are interconnected via a bus 66.

[0080] CPU 61 controls various parts of smart glasses 50 by executing control programs. Memory 62 is operating memory, such as RAM. Storage 63 stores various programs and data. Storage 63 is, for example, flash memory. CPU 61 reads the control program from storage 63 into memory 62 and executes the control program using memory 62 as the operating area. CPU 61 performs control and various processing on the aforementioned structures according to the control program.

[0081] Display unit 64 is a component that projects display light onto the pupil of user U to display an image within a portion of user U's field of vision. It includes an image display element (not shown) and a projection lens, among other things. The resolution of the image display element is, for example, 640 dots horizontally and 360 dots vertically.

[0082] The communication unit 65 is an interface for communication between the smart glasses 50 and the smartphone 10 and other devices, such as using standards like Wi-Fi (registered trademark) and Bluetooth (registered trademark).

[0083] [Processing flow]

[0084] First, the processing flow when user U takes an image using smartphone 10 will be explained. For example... Figure 7 As shown, when taking an image, for example, user U does not wear smart glasses 50 but only uses smartphone 10 to take the picture.

[0085] Figure 8 This is a flowchart illustrating the processing when taking images using a smartphone 10.

[0086] If the user inputs a camera activation instruction or other instruction to set the image capture mode, the CPU 21 of the smartphone 10 switches to the image capture mode (step SP1).

[0087] If switching to image capture mode, CPU21 will use the image capture IP (refer to...) Figure 9 The display is shown on touch panel 24 (step SP2). For example... Figure 9 As shown, the photographic display IP has a GUI (Graphical User Interface) for adjusting the zoom level, namely a zoom level input unit 30, a shutter button 31, and a live preview display unit 32. The live preview display unit 32 displays a real-time preview of the captured image IM. Figure 9 In order to avoid complicating the accompanying drawings, the image IM is shown outside the column of the real-time preview display unit 32. However, the image IM is actually displayed within the real-time preview display unit 32, and the image IM is displayed overlapping with the zoom magnification input unit 30.

[0088] The zoom ratio input unit 30 includes, for example, a zoom ratio display unit 30a that displays the current zoom ratio and a zoom ratio specification line 30b for specifying the zoom ratio from the wide-angle end to the telephoto end. The zoom ratio is displayed based on the focal length at the wide-angle end. That is, the zoom ratio at the wide-angle end is the reference, and therefore becomes 1x. As an example, the zoom ratio at the telephoto end is 5x. The zoom ratio specification line 30b is an arc-shaped curve. One end of the zoom ratio specification line 30b (at...) Figure 9 The middle (bottom) represents the zoom ratio at the wide-angle end, which is set to 1x in this example. The other end of the zoom ratio specification line 30b (in...) Figure 9 The zoom ratio at the top (middle section) is 5x at the telephoto end, set to 5x in this example. On the zoom ratio specification line 30b, the zoom ratio in the middle section is set to 1x to 5x depending on the distance from one end. In this example, the zoom ratio display section 30a is circular and has dimensions sufficient to enclose numbers indicating zoom ratios such as 1x and 5x.

[0089] The circular magnification display unit 30a has a fixed display position. On the other hand, the magnification specification line 30b moves along an arc-shaped trajectory relative to the magnification display unit 30a via a sliding operation by the user U. The zoom magnification numbers, such as 1x (x1.0) and 5x (x5.0), displayed at both ends of the magnification specification line 30b, move freely along with the magnification specification line 30b. The magnification specification line 30b moves via the user U's sliding operation, thereby changing its intersection position relative to the magnification display unit 30a. The zoom magnification corresponding to the intersection position on the magnification specification line 30b is displayed within the circular magnification display unit 30a. Figure 9 The image shows a zoom level of 2.5x (x2.5) displayed within the zoom display unit 30a. Thus, the zoom level is input corresponding to the intersection position of the zoom specification line 30b relative to the zoom display unit 30a.

[0090] The zoom magnification input area 24a is the area including the magnification display unit 30a and the magnification specification line 30b, and is set to be an arc shape along the arc-shaped curve of the magnification specification line 30b. The zoom magnification input area 24a represents the range of movement of the user U's finger operating the zoom magnification input unit 30. The zoom magnification input area 24a is the first area in the technology of the present invention.

[0091] The CPU 21 detects a sliding operation of the zoom ratio input area 24a within the zoom ratio specification line 30b in the touch panel 24 as a zoom ratio change operation performed when taking a picture using the smartphone 10. If a zoom ratio change operation is detected, the CPU 21 changes the zoom ratio of the photographic lens 26a or the electronic zoom ratio. As the zoom ratio changes, the displayed image IM is zoomed in the real-time preview display unit 32. That is, the closer the zoom ratio is to the telephoto end, the narrower the angle of view in which the subject in the image IM is magnified; the closer the zoom ratio is to the wide-angle end, the wider the angle of view in which the subject in the image IM is reduced.

[0092] Furthermore, in the smartphone 10, the optical axis direction Z of the photographic lens 26a located on the back 11b of the frame 11 becomes the image capture direction. The user U moves the smartphone 10 to adjust the orientation of the photographic lens 26a, thereby adjusting the image capture direction.

[0093] User U adjusts the shooting direction and zoom level. If shooting is ready, the user touches the shutter button 31. If the touch of shutter button 31 is detected (step SP3), CPU 21 controls the camera unit 26 to take a picture (step SP4) and acquire image data (step SP5). CPU 21 saves the image data acquired by the camera unit 26 in memory 23 and ends the processing.

[0094] Next, the process of user U confirming an image in image display system 1 will be explained. For example... Figure 10 As shown, when user U uses smart glasses 50 to confirm image IM, user U is wearing smart glasses 50.

[0095] Figure 11 This is a flowchart illustrating the processing involved in displaying an image in image display system 1.

[0096] If the user inputs an instruction to set to image display mode, the CPU 21 of the smartphone 10 switches to image display mode (step SP11).

[0097] If the system switches to image display mode, the CPU 21 acquires information about the resolution of the display unit 64 of the smart glasses 50 connected to the smartphone 10 (step SP12). This resolution information is stored in the memory 23, for example, during the settings for wirelessly connecting the smartphone 10 to the smart glasses 50, along with the model information of the smart glasses 50. The CPU 21 acquires this information by reading it from the memory 23 when the system switches to image display mode.

[0098] If user U specifies the image to be displayed from an image captured by smartphone 10, CPU 21 executes image acquisition processing (step SP13) to retrieve the image from memory 23. Here, as an example, regarding the display... Figure 9 The image IM showing a landscape with people will be explained below. As an example, the resolution of the image IM taken by the smartphone 10 is 4096 pixels horizontally and 3072 pixels vertically.

[0099] In this example, the resolution of the display unit 64 of the smart glasses 50 is lower than the resolution of the image IM. Therefore, the CPU 21 adjusts the resolution of the image IM to match the resolution of the display unit 64 of the smart glasses 50. Adjusting the resolution of the image IM is equivalent to adjusting the image size. In the initial state, the image IM is the size that the entire image IM is contained within the screen of the display unit 64. The CPU 21 adjusts the image IM to the initial image size and displays it on the smart glasses 50 (step SP14).

[0100] Next, as Figure 12As shown, the CPU 21 of the smartphone 10 is set to receive a first operation instruction from the user U for changing the display magnification of the image IM displayed on the smart glasses 50 and a second operation instruction for scrolling the image IM (step SP15). The display screen IV is the operation screen displayed on the touch panel 24 when the image IM is displayed on the smart glasses 50.

[0101] In this example, the image IM is displayed only on the smart glasses 50, not on the display screen IV of the smartphone 10. Alternatively, the image IM can also be displayed on the display screen IV when it is displayed on the smart glasses 50.

[0102] If a first operation instruction for changing the display magnification of the image IM or a second operation instruction for scrolling the image IM is input via the display screen IV (the determination in step SP16 is "Yes"), then the CPU 21 acquires the input operation instruction. Step SP16 is an example of operation instruction acquisition processing for acquiring the first or second operation instruction. Next, the CPU 21 executes display control processing to change the display magnification and scroll the image IM according to the acquired operation instruction, generates an image IM with a changed display form, and displays it on the smart glasses 50 (step SP18).

[0103] The above-mentioned image IM display magnification change and scrolling process (steps SP16-18) is repeated until the user U inputs an image display end indication.

[0104] If the confirmation of image IM ends, the user U inputs an image display end instruction to the smartphone 10. The input of the image display end instruction can be made, for example, using a physical button (not shown) on the smartphone 10 or a back button (not shown) on the touch panel 24. If the image display end instruction is input (the determination in step SP19 is "yes"), the CPU 21 stops displaying image [M] and ends the process.

[0105] Furthermore, after displaying the display screen IV (step SP1 5), if an input indicating the end of image display is entered (the determination in step SP19 is "yes") without any input of any operation instructions for changing the display magnification or scrolling of the image IM (the determination in step SP1 6 is "no"), then the CPU 21 maintains the initial state and stops displaying the image IM, and ends the process.

[0106] Here, the handling of changes in the display magnification of the image IM and the handling during scrolling are explained in detail individually.

[0107] First, the change in the display magnification of the image IM will be explained. For example... Figure 12As shown, the zoom ratio input unit 30 described above is displayed on the display screen IV. In image display mode, the zoom ratio input unit 30 is displayed as an operation unit for changing the display ratio of the image IM. The zoom ratio input unit 30 is displayed, for example, at a position approximately the same as the photographic screen IP. Therefore, the zoom ratio change operation can be performed using the same operation as the zoom ratio change operation performed during shooting.

[0108] The magnification input area 24b is an area for receiving input for magnification change operations to change the display magnification. The magnification input area 24b is set to be a rectangle larger than the zoom magnification input area 24a. The magnification input area 24b is the second area in the technology of this invention.

[0109] The CPU 21 detects a sliding operation that moves the magnification specification line 30b within the magnification input area 24b of the touch panel 24 as a magnification change operation for changing the display magnification. At this time, when a finger touches a position away from the magnification specification line 30b within the magnification input area 24b, the CPU 21 considers the finger to have touched the closest position on the magnification specification line 30b from the finger's contact position and detects the sliding operation targeting the magnification specification line 30b as a magnification change operation.

[0110] Figure 13 This is a graph showing the relationship between the sliding amount (i.e., the operation amount) and the magnification during magnification change operations. The reference position (i.e., sliding amount 0) is set as the state where the lower limit of the magnification on the magnification specification line 30b shown on the zoom magnification input unit 30 overlaps with the magnification display unit 30a. The relationship between the change in zoom magnification and the sliding amount from the reference position is shown in graph R1. In the zoom magnification input unit 30, as described above, the reference position represents the zoom magnification at the wide-angle end (1x) in image capture mode. On the other hand, in image display mode, the reference position represents the display magnification of the image IM at the same magnification (1x). Figure 5 As shown, equal-scale (1x) display refers to the state in which the image IM in its initial state, which matches the resolution of the touch panel 24, is displayed on the smart glasses 50.

[0111] In graph R1, the magnification changes proportionally to the amount of sliding. In image capture mode, when a sliding operation is performed, CPU 21 determines the zoom magnification corresponding to the amount of sliding based on the relationship shown in graph R1. The greater the sliding amount, the greater the zoom magnification. Then, in image display mode, when a sliding operation is performed, CPU 21 determines the display magnification corresponding to the amount of sliding based on the relationship shown in graph R1. The greater the sliding amount, the greater the display magnification. The greater the display magnification, the more the image IM is magnified. The relationship between the amount of operation and the magnification shown in graph R1 applies to both the change in zoom magnification in image capture mode and the change in display magnification in image display mode. Therefore, in image display mode, when a sliding operation with the same amount of sliding as in image capture mode is performed, CPU 21 changes the display magnification of image IM by the same amount as the change in zoom magnification in image capture mode.

[0112] like Figure 14 As shown, the sliding operation for changing magnification in image display mode is an arc-shaped sliding operation, the same as the zoom magnification change operation in image capture mode. The CPU 21 detects this arc-shaped sliding operation as a magnification change operation. The arc-shaped sliding operation in the touch panel 24 is, for example, an arc-shaped sliding operation with a radius of 3cm or more and 8cm or less. This is a shape and range that allows the user U to operate the smartphone 10 without difficulty when sliding with their thumb.

[0113] When a sliding operation is entered within the magnification input area 24b (see reference) Figure 11 In step SP16), CPU21 obtains the sliding operation as the first operation instruction.

[0114] CPU21 executes display control processing to change the display magnification of the image IM displayed on the smart glasses 50 according to the zoom magnification change operation as the first operation instruction (see reference). Figure 11 (Step SP17). At this time, CPU21 controls the change in the display magnification of image IM based on the change in the magnification relative to the operation amount of the zoom magnification change operation.

[0115] Furthermore, as an example, the center position when changing the display magnification of the image IM is the center position of the image IM currently displayed on the smart glasses 50. For example... Figure 15 As shown, in the initial state, the center position of the entire image IM becomes the initial position CO of the center position when the display magnification is changed. That is, when the image IM is magnified according to the display magnification, the image IM is magnified with the initial position CO as the center. Region DA represents the area displayed on the smart glasses 50 when the image IM is magnified with the initial position CO as the center.

[0116] Furthermore, the center position can also be changed from the initial position CO. For example, the center position can be specified by the user U touching any position in the image IM. Accordingly, the image IM can be zoomed in with the area of ​​interest of the user U in the image IM as the center.

[0117] Next, the scrolling of the image IM will be explained. Scrolling of the image IM can be performed even when the display magnification of the image IM has been changed and the entire image IM is not displayed on the smart glasses 50. For example, as... Figure 15 As shown, when a region DA is displayed as part of an image IM, the displayed region DA can be changed within the image IM by scrolling the image IM.

[0118] Regarding scrolling, the user U moves the smartphone 10 in the same way as when taking a picture to adjust the orientation of the smartphone 10's camera lens 26a, thereby specifying the scrolling direction. For example, to scroll the image IM to the right, the orientation of the smartphone 10's camera lens 26a is tilted to the right from the reference direction. And, to scroll the image IM upwards, the orientation of the smartphone 10's camera lens 26a is tilted upwards from the reference direction.

[0119] Regarding the reference direction, the reference direction in the left and right directions is set, for example, as follows: Figure 11 The orientation of the camera lens 26a of the smartphone 10 at the point when it switches to image display mode in step SP11 is shown. Furthermore, the reference direction in the vertical direction is set, for example, so that the orientation of the camera lens 26a of the smartphone 10 becomes the horizontal direction.

[0120] CPU 21 continuously monitors the posture of smartphone 10 detected by posture detection unit 27, and obtains changes in the posture of smartphone 10 as a second operation instruction (see reference). Figure 11 Step SP16).

[0121] CPU21 executes display control processing (step SP17) for scrolling the image IM displayed on the smart glasses 50 according to the second operation instruction.

[0122] For example, when it is detected that the orientation of the camera lens 26a of the smartphone 10 is tilted to the right from the reference direction, such as Figure 16 As shown, CPU21 transfers image IM from Figure 15 The state shown scrolls to the right. Therefore, the area DA displayed on the smart glasses 50 in the image IM is changed to the right side of the image IM. Furthermore, when the orientation of the camera lens 26a of the smartphone 10 is tilted upwards from the reference direction, as... Figure 17 As shown, CPU21 transfers image IM from Figure 15The displayed state scrolls upwards. As a result, the area DA displayed in the image IM on the smart glasses 50 is changed to the upper part of the image IM.

[0123] [Effects]

[0124] In this embodiment, the CPU 21 (equivalent to a processor) of the smartphone 10 (equivalent to a camera device and a display control device) performs the following processes: image acquisition processing to acquire an image displayed on the smart glasses 50 (equivalent to a wearable monitor); first operation instruction acquisition processing to acquire a first operation instruction for changing the display magnification of the image, the first operation instruction being input by operation on the operation unit of the smartphone 10 which is separate from the smart glasses 50 and by a magnification change operation in the smartphone 10; and display control processing to change the display magnification of the image according to the first operation instruction.

[0125] Therefore, the user U can change the display magnification of the image displayed on the smart glasses 50 through the magnification change operation in the smartphone 10, thus making it a highly convenient display control device.

[0126] Generally speaking, the display resolution of wearable monitors, such as smart glasses 50, is lower than the resolution of images captured by photographic devices, such as smartphones 10. Therefore, the operation of magnifying the image by changing the display magnification is frequent in order to check details such as noise in the image on the wearable monitor.

[0127] According to the technology of the present invention, the display magnification change operation for such frequently performed images can be performed with the same ease of operation as the zoom magnification change operation performed during shooting, as in this example. Therefore, even when wearing a wearable monitor such as smart glasses 50 and part of the user's field of vision is obstructed, the user U can easily perform the magnification change operation. This is believed to be because the zoom magnification change during shooting and the display magnification change during image display are common in terms of enlarging and reducing the displayed image, and the user U is familiar with the zoom magnification change operation in a photographic device such as a smartphone 10. Thus, a display control device with good operability and high convenience for the user can be provided.

[0128] Furthermore, in this example, an example of changing the display magnification of an image on a wearable monitor by means of a zoom magnification change operation performed by a smartphone 10 (an example of a camera device) during shooting, which is described as a zoom magnification change operation in a camera device, is given, but this is not a limitation. Zoom magnification change is one example; the magnification change operation of the present invention can also be applied to changes in the magnification of a beam expander lens in a TV lens, etc. The magnification change operation in a camera device can also be a display magnification change operation that changes the display magnification of an image when displaying an image using a smartphone 10 (an example of a camera device). That is, the display magnification of an image on a wearable monitor can be changed by means of the same operation as the image display magnification change operation in a camera device.

[0129] Furthermore, the CPU 21 controls the change in the display magnification of the image IM based on the change in the magnification ratio relative to the operation amount of the magnification change operation during the display control processing. This configuration allows for changing the display magnification of the image displayed on the smart glasses 50 with the same operational feel as the magnification change operation in the smartphone 10, which is therefore preferable.

[0130] Furthermore, the CPU 21 also performs a second operation instruction acquisition process, which acquires the change in the posture of the smartphone 10 detected by the posture detection unit 27 as a second operation instruction, and scrolls the image according to the second operation instruction in the display control process. By configuring it in this way, the scrolling operation can be performed with the same feeling as when tilting the smartphone 10 toward the desired subject during shooting.

[0131] Furthermore, the smartphone 10 has a touch panel 24. When the zoom ratio change operation during smartphone 10 shooting is a sliding operation within the zoom ratio input area 24a on the touch panel, the CPU 21 detects the sliding operation within the magnification input area 24b on the touch panel 24, which includes the zoom ratio input area 24a and is larger than the zoom ratio input area 24a, as an operation to change the display magnification of the image. By configuring it in this way, even if the user U does not look at the touch panel 24 and the position of the finger performing the sliding operation is far away from the magnification designation line 35b, the sliding operation can still be detected. Therefore, a display control device with higher convenience can be realized.

[0132] Furthermore, the CPU 21 detects arc-shaped swipe operations with a radius R of 3 cm or more and 8 cm or less on the touch panel 24 as display magnification change operations. An arc-shaped swipe operation with a radius R of 3 cm or more and 8 cm or less is a shape that allows the user U to easily perform a swipe operation when holding the smartphone 10 and using their thumb. By setting it in this form, it becomes a display control device with greater convenience. On the other hand, if the radius R of the arc-shaped swipe operation is less than 3 cm, the thumb must be extremely bent to operate, thus making the swipe operation difficult. Furthermore, if the radius R of the arc-shaped swipe operation exceeds 8 cm, it becomes impossible to reach even with the thumb fully extended, thus making the swipe operation difficult.

[0133] Furthermore, the image IM displayed on the smart glasses 50 is an image captured by the smartphone 10. That is, the same smartphone 10 can be used to capture the image IM and change the display magnification when displaying the image IM. Therefore, a highly convenient display control device can be realized.

[0134] [Second Implementation]

[0135] Next, a second embodiment of the present invention will be described. The image display system 2 of the second embodiment differs from the first embodiment in that the device functioning as a display control device is changed to smart glasses 50 instead of the smartphone 10. That is, as described later, the smart glasses 50 performs image acquisition processing, first operation instruction acquisition processing, and display control processing. Furthermore, in the second embodiment, unlike the first embodiment, the image displayed by the smart glasses 50 is obtained from an external server 70 instead of from the smartphone 10.

[0136] The hardware structure of the smartphone 10 and the smart glasses 50 is the same as that in the first embodiment, so the description of the content that is repeated in the first embodiment is omitted. Figure 18 This is a schematic structural diagram of an image display system including a display control device based on the second embodiment.

[0137] like Figure 18 As shown, the image display system 2 includes a smartphone 10 and smart glasses 50. The CPU 61 of the smart glasses 50 is an example of a processor in the technology of this invention. Furthermore, the memory 62 is an example of memory in the technology of this invention. The smart glasses 50, equipped with CPU 61 and memory 62, also functions as a display control device in the technology of this invention. The CPU 61 of the smart glasses 50 obtains images displayed by the smart glasses 50 from an external server 70.

[0138] Smartphone 10 and smart glasses 50 are connected directly wirelessly. Smartphone 10 and server 70 are connected wirelessly via network 71. Similarly, smart glasses 50 and server 70 are connected wirelessly via network 71.

[0139] [Processing flow]

[0140] Next, the processing performed in this embodiment will be explained. Figure 19 This is a flowchart illustrating the processing involved in displaying images in image display system 2.

[0141] If the user inputs an instruction to set to image display mode, the CPU 21 of the smartphone 10 switches to image display mode (step SP21).

[0142] When user U specifies the image to be displayed from the images stored in server 70, CPU 21 sends image specification information (step SP22) to server 70, indicating the specified image.

[0143] Upon receiving image specification information from smartphone 10 (step SS21), server 70 sends the specified image to smart glasses 50 (step SS22).

[0144] The CPU 61 of the smart glasses 50 performs image acquisition processing (step SG21) by receiving an image from the server 70 and displaying it on the display unit 64. Next, the CPU 61 adjusts the resolution of the acquired image to match the resolution of the display unit 64 of the smart glasses 50. Initially, the image is sized to fit entirely within the screen of the display unit 64. The CPU 61 adjusts the size of the image IM to the initial image size and displays it on the display unit 64 (step SG22).

[0145] Next, in step SP23, the CPU 21 of the smartphone 10 will... Figure 12 The display screen IV shown is displayed on the touch panel 24, and is in a state where the display magnification of the image displayed on the smart glasses 50 is changed and scrolled, received from the user U.

[0146] If the input is a first operation instruction for changing the display magnification of an image or a second operation instruction for scrolling an image IM (the determination in step SP24 is "yes"), then the CPU21 sends these operation instructions to the server 70 (step SP25).

[0147] If either the first operation instruction or the second operation instruction is received from the smartphone 10 (step SS23), the server 70 transmits the operation instruction to the smart glasses 50 (step SS24).

[0148] The CPU 61 of the smart glasses 50 executes an operation instruction acquisition process (step SG23) to obtain the operation instruction as an instruction to change the display mode by receiving the operation instruction from the server 70. Next, the CPU 61 executes a display control process to change or scroll the display magnification of the image IM according to these operation instructions, and displays the image IM whose display mode has been changed by changing the display magnification or scrolling on the display unit 64 (step SG24).

[0149] In the smartphone 10, the above-mentioned image display magnification change and scrolling process is repeated until the user inputs an image display end indication (steps SP24-SP26). Following the processing of the smartphone 10, the smart glasses 50 and the server 70 also repeat the process. Figure 19 The processing is shown.

[0150] If the image confirmation is complete, the user U inputs an image display end instruction to the smartphone 10. The input of the image display end instruction can be made, for example, using a physical button (not shown) on the smartphone 10 or a back button (not shown) on the touch panel 24. When the image display end instruction is input (the determination in step SP26 is "yes"), the CPU 21 of the smartphone 10 sends the end instruction to the server 70 (step SP27) and terminates the process.

[0151] If a display end instruction is received from the smartphone 10 (step SS25), the server 70 transmits the display end instruction to the smart glasses 50 (step SS26).

[0152] If a display end instruction is received from the server 70, the CPU 61 of the smart glasses 50 ends the display of the image in the display unit 64 (step SG25).

[0153] [Effects]

[0154] In this embodiment, the smart glasses 50, which is an example of a wearable monitor in the technology of the present invention, also functions as a display control device in the technology of the present invention. Furthermore, the CPU 61 of the smart glasses 50 acquires an image IM from an external server 70 during image acquisition processing, and acquires a first operation instruction and a second operation instruction via the server 70 during operation instruction acquisition processing.

[0155] By configuring it in this way, images other than those captured by the smartphone 10 can also be obtained from the server 70 and displayed on the smart glasses 50.

[0156] Furthermore, when displaying the image IM on the smart glasses 50, all data acquired by the smart glasses 50 is retrieved from the server 70. Therefore, when displaying the image, the smart glasses 50 does not need to communicate with the smartphone 10, but only with the server 70, thus reducing the communication processing load between the smart glasses 50 and the smartphone 10.

[0157] [Third Implementation]

[0158] Next, the third embodiment of the present invention will be described. The difference between the image display system in the third embodiment and the second embodiment is that the CPU 61 of the smart glasses 50 directly obtains the first operation instruction, the second operation instruction, and the display end instruction (refer to steps SP35, SP37, and steps SG33, SG35) from the smartphone 10 without going through the server 70. The hardware structure of the smartphone 10 and the smart glasses 50 is the same as in the second embodiment, therefore, its description is omitted.

[0159] [Processing flow]

[0160] Next, the processing performed in this embodiment will be explained. Figure 20 This is a flowchart illustrating the image display process in the image display system of this embodiment.

[0161] If the user inputs an instruction to set to image display mode, the CPU 21 of the smartphone 10 switches to image display mode (step SP31).

[0162] When user U specifies the image to be displayed from the images stored in server 70, CPU 21 sends image specification information (step SP32) to server 70 indicating the specified image.

[0163] Upon receiving image specification information from smartphone 10 (step SS31), server 70 sends the specified image to smart glasses 50 (step SS32).

[0164] If an image is received from the server 70, the CPU 61 of the smart glasses 50 performs image acquisition processing to acquire it as an image to be displayed on the display unit 64 (step SG31). Next, the CPU 61 adjusts the resolution of the acquired image to match the resolution of the display unit 64 of the smart glasses 50. The image is initially sized to fit entirely within the screen of the display unit 64. The CPU 61 adjusts the image to its initial size and displays it on the display unit 64 (step SG32).

[0165] Next, in step SP33, the CPU 21 of the smartphone 10 will... Figure 12The display shown is shown on the touch panel 24 and is set to receive the display magnification change and scrolling indication of the image displayed on the smart glasses 50 from the user U (step SP33).

[0166] If a first operation instruction for changing the display magnification of an image or a second operation instruction for scrolling an image IM is input (the determination in step SP34 is "yes"), then the CPU21 sends these operation instructions to the smart glasses 50 (step SP35).

[0167] The CPU 61 of the smart glasses 50 executes an operation instruction acquisition process (step SG33) to obtain the operation instruction as an instruction to change the display mode by directly receiving the operation instruction from the smartphone 10. Next, the CPU 61 executes a display control process to change or scroll the display magnification of the image IM according to these operation instructions, and displays the image whose display mode has been changed by changing the display magnification or scrolling on the display unit 64 (step SP34).

[0168] In the smartphone 10, the above-mentioned image display magnification change and scrolling process is repeated until the user inputs an image display end instruction (steps SP34-36). Following the processing of the smartphone 10, the smart glasses 50 and the server 70 also repeat the process. Figure 20 The processing is shown.

[0169] If the image display is completed, the user U inputs an image display completion instruction to the smartphone 10. This instruction can be input, for example, using a physical button (not shown) on the smartphone 10 or a back button (not shown) on the touch panel 24. When the image display completion instruction is input (the determination in step SP36 is "yes"), the CPU 21 of the smartphone 10 sends the completion instruction to the smart glasses 50 (step SP37) and terminates the process.

[0170] If a display end instruction is received directly from the smartphone 10, the CPU 61 of the smart glasses 50 ends the display of the image in the display unit 64 (step SG35).

[0171] [Effects]

[0172] In this embodiment, the smart glasses 50, which is an example of a wearable monitor in the technology of the present invention, also functions as a display control device in the technology of the present invention. Furthermore, the CPU 61 of the smart glasses 50 acquires images from an external server 70 in image acquisition processing, and directly acquires a first operation instruction and a second operation instruction from the smartphone 10 in operation instruction acquisition processing.

[0173] By configuring it in this way, images other than those captured by the smartphone 10 can also be obtained from the server 70 and displayed on the smart glasses 50.

[0174] Furthermore, the smart glasses 50 directly obtains the first and second operation instructions for display control processing, such as changing the display magnification and scrolling, from the smartphone 10 without going through the server 70. Therefore, compared to obtaining the first and second operation instructions through the server 70, the time lag (time lag caused by communication delay) from the time the user U inputs the operation instructions to the smartphone 10 until the display mode is changed can be reduced.

[0175] [Fourth Implementation]

[0176] Next, the fourth embodiment of the present invention will be described. The difference between the image display system in the fourth embodiment and the first embodiment lies in the initial position CO of the center position when changing the display magnification of the image during the display control processing in the CPU 21 of the smartphone 10. In this embodiment, the hardware structure of the smartphone 10 and the smart glasses 50 is the same as in the first embodiment, and therefore, its description is omitted.

[0177] like Figure 21 As shown, the image file F1 of the image IM processed in this embodiment has image data D1 and additional information data D2. As additional information data D2, in addition to information such as the date and time of the image being taken and the location of the image being taken, it also includes information D2a about the tilt angle, i.e., the pitch angle, of the smartphone 10 relative to the horizontal direction when the image was taken.

[0178] like Figure 22 As shown, in the technology of this invention, the pitch angle θtilt is specifically the angle between the optical axis direction Z (meaning the same as the shooting direction) of the photographic lens 26a of the smartphone 10 and the horizontal direction H. Regarding the pitch angle θtilt, it is set to 0° when the optical axis direction Z is oriented towards the horizontal direction H, a positive angle when the optical axis direction Z is more upward than the horizontal direction H, and a negative angle when the optical axis direction Z is more downward than the horizontal direction H.

[0179] Figure 23 This is a diagram illustrating the initial position CO of the center when changing the display magnification in an image IM. Figure 23 In the image IM, the X direction (i.e., Figure 23 The horizontal coordinate of the image IM is set as X, and the Y-axis of the image IM (i.e., the horizontal coordinate) is set as X. Figure 23The vertical coordinate of the image IM is set to Y. Regarding the two-dimensional coordinates (X, Y) of the image IM, the lower left coordinate is set as the origin (0, 0), and the upper right coordinate is set as the vertex (100, 100). Furthermore, both the X and Y coordinates represent values ​​that normalize the maximum number of pixels in the image IM to 100.

[0180] In the first embodiment, the coordinates of the initial position CO of the center position when changing the display magnification of the image are set to the coordinates of the center position of the image IM, i.e., (50, 50). In contrast, in this embodiment, the CPU 21 of the smartphone 10 changes the coordinates of the initial position CO of the center position when changing the display magnification of the image from the default coordinates (50, 50) based on the pitch angle information D2a contained in the image file F1.

[0181] Regarding the change of the initial position CO of the center position when changing the display magnification of an image, as an example, Figure 24 As shown, when the pitch angle θtilt is 0, the initial position CO coordinates (50, 50) of the center position are not changed from the default position. Furthermore, when the pitch angle θtilt is positive, the Y-axis center position change is linearly varied from 0 to 50 from greater than 0° to 45°. Beyond 45°, the Y-axis center position change is set to 50.

[0182] Furthermore, when the pitch angle θtilt is negative, the change in the center position of the Y-axis linearly varies from 0 to -50 from less than 0° to -45°. When the pitch angle is less than -45°, the change in the center position of the Y-axis is set to -50. That is, when the pitch angle is positive, the upper part of the image IM becomes the initial position CO of the center position when changing the display magnification of the image IM; when the pitch angle is negative, the lower part of the image IM becomes the initial position CO of the center position when changing the display magnification of the image IM. As an example, when the tilt angle θtilt is 22.5°, such as... Figure 23 As shown, the initial coordinates of the changed center position COa are changed to (50, 75).

[0183] In this embodiment, the image file F1 of the image IM includes not only image data D1, but also information D2a, which is the tilt angle (i.e., pitch angle) of the smartphone 10 relative to the horizontal direction when the image IM was captured, as supplementary information data D2. In this case, the CPU 21 of the smartphone 10 changes the initial position CO of the center position when altering the display magnification of the image IM based on the pitch angle information D2a during the display control processing.

[0184] For example, when taking a landscape photo of a distant view, if the camera lens 26a of the smartphone 10 is pointed upwards, the area closer to the sky will be captured. If the zoom ratio is changed in this state, the zoom ratio will be changed with the sky as the center.

[0185] By adopting the configuration of this embodiment, even when the image IM has already been captured, if the pitch angle during capture is positive (i.e., when the image IM is captured with the optical axis Z of the camera device pointing upwards), the initial position CO of the center position when changing the display magnification of the image IM is shifted upwards towards the image IM. Therefore, when displaying the image, the user U can simulate the feeling of zoom change during capture.

[0186] [Fifth Implementation]

[0187] Next, the fifth embodiment of the present invention will be described. The difference between the image display system of the fifth embodiment and the first embodiment is that, when displaying the image IM on the smart glasses 50, horizontal correction of the image IM is performed. In this embodiment, the hardware structure of the smartphone 10 and the smart glasses 50 is the same as in the first embodiment, therefore, the description is omitted.

[0188] like Figure 25 As shown, the image file F2 of the image IM processed in this embodiment has image data D1 and additional information data D2. As additional information data D2, in addition to information such as the date and time of the image being taken and the location of the image being taken, it also includes information D2b about the rotation angle of the smartphone 10 around the optical axis direction Z (meaning the same as the shooting direction) relative to the horizontal direction when the image was taken, i.e., the roll angle.

[0189] like Figure 26 As shown, in the technology of this invention, the roll angle θroll is specifically the angle between the horizontal direction HP and the horizontal direction H of the smartphone 10 during shooting. During shooting with the smartphone 10, by changing the angle of the frame 11 of the smartphone 10 relative to the horizontal direction H, it is possible to switch between shooting vertically elongated images and shooting horizontally elongated images. In any case, the direction of the smartphone 10 corresponding to the horizontal (long side) direction of the image IM is set to the horizontal direction HP.

[0190] Regarding the roll angle θroll, when the camera direction of the smartphone 10 is set to... Figure 26 In the case of the inside of the paper, it will be Figure 26 When the right end of the smartphone 10 rises higher than the horizontal direction H with the optical axis Z as the center, that is, when the smartphone 10 rotates counterclockwise, the angle is set as positive. Furthermore, in Figure 26When the right end of the smartphone 10 is lower than the horizontal direction H with the optical axis Z as the center ( Figure 26 (This state is shown), that is, the smartphone 10 is rotated in a clockwise direction and the angle is set to negative.

[0191] Figure 26 This illustrates a state where a horizontal image is captured using the smartphone 10. In this state, the long side of the frame 11 of the smartphone 10 forms the horizontal (HP) aspect ratio of the smartphone 10. Furthermore, in... Figure 26 The image shows the smartphone 10 in a state where the roll angle θroll is negative when taking a picture. If a picture is taken in this state, then... Figure 27 As shown, in image IM, the horizontal direction HI of the subject is rotated counterclockwise around the center CI of image IM.

[0192] When an image is displayed on the smart glasses 50, the CPU 21 of the smartphone 10 performs horizontal correction on the image IM. For example, horizontal correction can be achieved by rotating the entire image IM around its center CI as an axis by an angle equal to the roll angle θroll. At this time, the direction of counter-clockwise rotation of the image IM is set as a positive angle, and the direction of clockwise rotation of the image IM is set as a negative angle.

[0193] As an example, such as Figure 27 As shown, in the case of image IM taken with a roll angle θroll of -20°, the horizontal direction HI of the subject in image IM is rotated 20° counterclockwise about the center CI of image IM. Therefore, as Figure 28 As shown, by rotating the entire image IM around its center CI as an axis by -20°, the same angle as the roll angle θroll, the horizontal direction HI of the subject in the image IM can be made consistent with the left and right direction of the image IM.

[0194] By setting it in this way, even if the image is taken at an angle, the horizontal direction of the subject can be aligned with the left and right directions of the image, so that the image can be displayed on the smart glasses 50 without making the user feel uncomfortable.

[0195] [Sixth Implementation]

[0196] Next, the sixth embodiment of the present invention will be described. The image display system based on the sixth embodiment is particularly a structure for making the change in the magnitude of the operation amount relative to the magnification of the magnification operation during shooting consistent with the change in the magnitude of the operation amount relative to the display magnification of the image when the photographing device that has taken the picture is different from the photographing device that provides operation instructions such as magnification change operation and scrolling for inputting the image.

[0197] Compared to the first embodiment, the display control processing in the CPU 21 of the smartphone 10 in this embodiment differs. In this embodiment, the hardware structure of the smartphone 10 and the smart glasses 50 is the same as in the first embodiment, therefore, a description is omitted.

[0198] like Figure 29 As shown, the image file F3 of the image processed in this embodiment has image data D1 and additional information data D2. As additional information data D2, in addition to information such as the date and time of image capture and the location of image capture, it also includes magnification change operation related information D2c, which relates to the amount of change in magnification relative to the amount of magnification change operation performed on the photographic device that captured the image. In this embodiment, as an example, the case where information about the maximum zoom magnification of the smartphone that captured the image is recorded will be explained regarding this magnification change operation related information D2c.

[0199] The images stored in the memory 23 of the smartphone 10, which is an image display system, are not necessarily limited to images captured by the smartphone 10. When the maximum zoom ratio of the smartphone that took the image is different from that of the smartphone 10, the operation feel of changing the zoom ratio is different in the smartphone that took the image and the smartphone 10.

[0200] In this case, the CPU 21 of the smartphone 10 obtains the information of the maximum zoom ratio of the smartphone that took the picture from the accompanying information data D2 of the image file F3 of the displayed image as the zoom ratio change operation related information D2c.

[0201] Figure 30 This is a graph showing the relationship between the amount of sliding (i.e., the amount of operation) and the zoom ratio during zoom change operations. Here, as an example, we will explain the case where the maximum zoom ratio of smartphone 10 is 5x and the maximum zoom ratio of the smartphone that took the picture is 10x.

[0202] The reference position (i.e., a sliding amount of 0) is set as the state where the lower limit of the magnification specification line 30b of the zoom magnification input unit 30 in the smartphone 10 overlaps with the magnification display unit 30a. The relationship between the change in magnification and the sliding amount from the reference position is shown in Graph R1. Furthermore, the reference position (i.e., a sliding amount of 0) is set as the state where the lower limit of the magnification specification line of the zoom magnification input unit in the smartphone that has taken an image overlaps with the magnification display unit. The relationship between the change in magnification and the sliding amount from the reference position is shown in Graph R2.

[0203] As mentioned above, the maximum zoom ratio of the smartphone that took the picture is 10x, and the maximum zoom ratio of the smartphone 10 is 5x. Therefore, even if the same amount of operation is performed in the zoom ratio change operation, the amount of change in zoom ratio will not be consistent.

[0204] Therefore, the CPU 21 of the smartphone 10 performs processing based on information about the maximum zoom ratio of the smartphone that has taken the image, ensuring that the change in zoom ratio relative to the zoom ratio change operation in the smartphone that has taken the image is consistent with the change in zoom ratio relative to the zoom ratio change operation in the smartphone 10. Specifically, as... Figure 30 As shown in the diagram R1a, the display magnification at one end of the magnification specification line 30b is set to 1x, the display magnification at the other end of the magnification specification line 30b is set to 10x, which is the same as the maximum zoom magnification of the smartphone that has taken the image, and the display magnification of the middle part is set to a magnification of 1x to 10x depending on the distance from one end.

[0205] And, as Figure 31 As shown, in the zoom magnification input section 30, the display magnification at one end of the magnification specification line 30b is set to 1x, the display magnification at the other end of the magnification specification line 30b is set to 10x, and the display magnification of the middle part is set to a magnification of 1x to 10x depending on the distance from one end.

[0206] By configuring it in this way, when the maximum zoom ratio of the smartphone that took the picture is different from the maximum zoom ratio of the smartphone 10, the operation feel of changing the zoom ratio can be the same in both the smartphone that took the picture and the smartphone 10.

[0207] Furthermore, when the size of the touch panel of the smartphone that took the picture is different from the size of the touch panel 24 of the smartphone 10, the operation of changing the zoom ratio in the smartphone that took the picture and the smartphone 10 will not be exactly the same.

[0208] Furthermore, since the GUI for changing the zoom level on the smartphone that has taken an image is different from that on the smartphone 10, the user experience for changing the zoom level on the smartphone that has taken an image will not be exactly the same on the two smartphones.

[0209] However, even in these cases, by adopting the configuration of this embodiment, the user experience of zooming in the smartphone 10 can be made similar to that of a smartphone that has taken a picture.

[0210] [Seventh Implementation]

[0211] Next, the seventh embodiment of the present invention will be described. The image display system based on the seventh embodiment is particularly a structure for making the change in the magnitude of the operation amount relative to the magnification of the magnification change operation during shooting consistent with the change in the magnification of the operation amount relative to the display magnification of the image when the photographing device that has taken the picture is different from the photographing device that provides operation instructions such as magnification change operation and scrolling for inputting the image.

[0212] Compared to the sixth embodiment, the interface for changing the zoom ratio in the smartphone 10 of this embodiment is different. Furthermore, the content of the display control processing in the CPU 21 of the smartphone 10 is different. In this embodiment, the hardware structure of the main body of the smartphone 10 and the smart glasses 50 is the same as in the first embodiment, therefore, descriptions are omitted.

[0213] like Figure 32 and Figure 33 As shown, in this embodiment, the smartphone 10 and the smartphone casing 15 are combined.

[0214] The smartphone housing 15 includes a housing portion 16 and a ring-shaped controller 18. The housing portion 16 engages with the smartphone 10. An opening 17 is formed in the housing portion 16 to expose the photographic lens 26a of the smartphone 10 when the smartphone 10 is engaged. Furthermore, the ring-shaped controller 18 is rotatably mounted on the back side 16b of the housing portion 16 relative to the housing portion 16.

[0215] The loop controller 18, like the zoom ring of a typical digital camera, can rotate relative to the housing 16, serving as an interface for zoom and display magnification changes in the smartphone 10. The loop controller 18 internally includes a detection unit (not shown) that detects the rotation direction and angle of the loop controller 18. Signals indicating the rotation direction and angle detected by the detection unit are transmitted to the smartphone 10 via a communication unit (not shown). This communication unit uses a standard such as Bluetooth (registered trademark).

[0216] like Figure 34As shown, the image file F4 of the image processed in this embodiment has image data D1 and additional information data D2. As additional information data D2, in addition to information such as the date and time of image capture and the location of image capture, it also includes magnification change operation related information D2d, which relates to the amount of magnification change relative to the amount of magnification change operation performed during image capture in the photographic device. In this embodiment, as an example, regarding this magnification change operation related information D2d, we will explain the case where information on the amount of magnification change relative to the change in the rotation angle of the zoom ring of the digital camera that performed image capture and information on the maximum zoom magnification of the digital camera is recorded.

[0217] The images stored in the memory 23 of the smartphone 10's image display system are not necessarily limited to images captured by the smartphone 10. When the photographic device used to capture the image is a digital camera, the feel of operating the zoom ratio change is different in digital cameras and smartphones 10.

[0218] Generally, in digital cameras, the zoom level is changed by rotating the zoom ring, but in smartphones, the zoom level is changed by sliding the touch panel 24.

[0219] The smartphone casing 15 is a casing designed to enable zoom magnification changes in the smartphone 10 with the same ease of operation as a digital camera.

[0220] When it is detected that the smartphone housing 15 is installed on the smartphone 10, the CPU 21 of the smartphone 10 enables the ring controller 18 of the smartphone housing 15 to function as an interface for zoom ratio change operations in the smartphone 10.

[0221] In the CPU 21 of the smartphone 10, as a method to detect that the smartphone casing 15 is installed on the smartphone 10, for example, it can be performed by detecting that the communication unit 25 of the smartphone 10 has established communication with the communication unit of the ring controller 18.

[0222] Furthermore, the CPU 21 of the smartphone 10 obtains information about the change in magnification relative to the rotation angle of the zoom ring of the digital camera that took the image, as well as information about the maximum zoom magnification of the digital camera, from the accompanying information data D2 of the image file F4 of the displayed image, as magnification change operation related information D2d.

[0223] Figure 35This is a graph showing the relationship between the rotation amount (i.e., the operation amount) of the zoom ring or ring controller 18 and the zoom ratio during zoom ratio change operations. Here, as an example, regarding the smartphone 10, the zoom ratio increases by 1x each time the ring controller 18 is rotated 90° clockwise, with a maximum zoom ratio of 5x. Furthermore, regarding the digital camera that has captured an image, the zoom ratio increases by 1x each time the zoom ring is rotated 40° clockwise, with a maximum zoom ratio of 10x.

[0224] The relationship between the change in zoom magnification and the rotation amount (i.e., operation amount) of the ring controller 18 in the smartphone 10 from the reference position (i.e., rotation amount 0) is shown in Figure R10. Furthermore, the lower limit position of the zoom ring in the digital camera that has taken an image is set as the reference position (i.e., rotation amount 0), and the relationship between the change in zoom magnification and the rotation amount (i.e., operation amount) from the reference position is shown in Figure R20.

[0225] Furthermore, the ring controller 18 does not have a rotating end and can rotate freely any number of times in the same direction. Therefore, the reference position of the ring controller 18 can be set, for example, to the position of the ring controller 18 at the point when the image of the initial display state is displayed on the display unit 64.

[0226] As described above, the magnitude of the change in the zoom ring rotation angle of the digital camera that has taken an image differs from that of the ring controller 18 installed on the smartphone 10. Furthermore, the maximum zoom magnification of the digital camera that has taken an image also differs from that of the smartphone 10. Therefore, even if the same amount of operation is performed during zoom magnification change operations, the amount of change in zoom magnification will not be consistent.

[0227] Therefore, the CPU 21 of the smartphone 10 performs processing based on information about the change in zoom ratio relative to the rotation angle change of the zoom ring of the digital camera that has taken the image, and information about the maximum zoom ratio of the digital camera, to ensure that the change in zoom ratio relative to the zoom ratio change operation in the digital camera that has taken the image is consistent with the change in zoom ratio relative to the zoom ratio change operation in the smartphone 10. Specifically, as follows... Figure 30 As shown in diagram R10a, the operation input for the ring controller 18 installed on the smartphone 10 is set to increase the zoom magnification by 1x each time it rotates 40° clockwise, and the maximum zoom magnification is set to 10x.

[0228] By setting it in this way, the operation of changing the zoom ratio in the smartphone 10 can be made to feel similar to that of a digital camera that has taken pictures.

[0229] [Eighth Implementation]

[0230] Next, the eighth embodiment of the present invention will be described. The difference between the image display system of the eighth embodiment and the first embodiment lies in that, in the display control processing of the CPU 21 of the smartphone 10, the scrolling amount when scrolling the image is corrected according to the second operation instruction. In this embodiment, the hardware structure of the smartphone 10 and the smart glasses 50 is the same as that of the first embodiment, and therefore, the description is omitted.

[0231] like Figure 36 As shown, the image file F5 of the image IM processed in this embodiment has image data D1 and additional information data D2. As additional information data D2, in addition to information such as the date and time of the image being taken and the location of the image being taken, it also includes perspective information D2e related to the perspective of the smartphone 10 when the image was taken.

[0232] The viewing angle information D2e can be any information that determines the viewing angle of the smartphone 10 at the time of shooting. This can be the viewing angle itself, or it can be the zoom ratio information corresponding to the viewing angle. Alternatively, when obtaining the zoom ratio information, the viewing angle information corresponding to the zoom ratio can be obtained separately. Here, as an example, the viewing angle information D2e is set to the viewing angle of the shooting area.

[0233] like Figure 37 As shown, compared to the field of view θavt at the telephoto end (i.e., a zoom ratio of 5x), the field of view θavw at the wide-angle end (i.e., a zoom ratio of 1x) of the smartphone 10 when capturing an image is 5 times larger. Furthermore, the larger the field of view when capturing an image, the larger the tilt angle of the smartphone 10 required to change the orientation of the photographic lens 26a from one end of the shooting range to the other. As shown in the example above, if the field of view increases by 5 times, the tilt angle of the smartphone 10 required to change the orientation of the photographic lens 26a from one end of the shooting range to the other becomes 5 times.

[0234] When the CPU 21 of the smartphone 10 performs display control processing for an image that is scrolled on the display unit 64 of the smart glasses 50 according to the second operation instruction, it corrects the scrolling amount of the image based on the viewing angle information D2e at the time of shooting.

[0235] Specifically, the amount of image scrolling is corrected in such a way that the tilt angle of the smartphone 10 required to scroll from one end of the image to the other when displaying the image is consistent with the tilt angle of the smartphone 10 required to change the orientation of the photographic lens 26a from one end of the photographic range to the other when taking an image.

[0236] By setting it to this configuration, it is possible to reproduce the feeling of the changing shooting area when tilting the smartphone 10 to approach the captured image while scrolling through images.

[0237] [Variation Example]

[0238] The present invention has been described above according to its preferred embodiments, but the embodiments applicable to the present invention are not limited to the above embodiments.

[0239] For example, in the above embodiment, the case where the user U does not wear the smart glasses 50 and only uses the smartphone 10 to take pictures was described. However, it is also possible for the user U to take pictures while wearing the smart glasses 50. In this case, such as Figure 38 As shown, the settings information of the smartphone 10 during shooting can be displayed in a portion of the image IX displayed in the right eye's field of vision. In addition to displaying the settings information during shooting, the content displayed in image IX can also display a live preview of the smartphone 10, and the live preview of the smartphone 10 can be overlaid with the settings information.

[0240] Furthermore, the smart glasses 50 are not limited to displaying images in a portion of the right eye's field of vision; they can also display images in a portion of the left eye's field of vision, a portion of both eyes' field of vision, or the entirety of both eyes' field of vision.

[0241] Furthermore, the wearable monitor in the technology of this invention is not limited to smart glasses, but can also be other display devices such as HMD.

[0242] Furthermore, the photographic device in the present invention is not limited to a smartphone, but can also be a digital camera with the same functions as the smartphone 10 described above.

[0243] Furthermore, in the above embodiments, the processing executed by CPU21 and CPU61 by various processors other than the CPU can also be performed. Examples of such processors include FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices) whose circuit structure can be modified after manufacturing, and ASICs (Application Specific Integrated Circuits)—processors with circuit structures specifically designed for performing specific processes, i.e., dedicated circuits. Each process can be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs or a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is a circuit composed of circuit elements such as semiconductor components.

[0244] Furthermore, while the above embodiments describe the display control program being pre-stored (installed) in memory 23 or memory 63, this is not a limitation. The program may also be provided on recording media such as CD-ROM (Compact Disk Read Only Memory), DVD-ROM (Digital Versatile Disk Read Only Memory), and USB (Universal Serial Bus) memory. Additionally, the program may be configured to be downloaded from an external device via a network.

[0245] Furthermore, ROM is a broad concept that includes rewritable ROM, such as flash memory, EPROM (Erasable Programmable Read-Only Memory), and EEPROM (Electrically Erasable Programmable Read-Only Memory).

[0246] Furthermore, in embodiments 1 to 5, the CPU is integrally disposed within the wearable device or integrally disposed within the smartphone. However, the CPU may also be disposed on a network connected to the wearable device or smartphone, or on an operating unit connected to the network, or on a cloud computing unit connected to the network.

[0247] When a CPU is located on a network or a device connected to a network, it is desirable that the network be managed by a cloud system, and the CPU connected to the network can work as a simulated CPU built into a wearable device or smartphone while using cloud computing technology through the network.

[0248] In the above embodiments, the situation where it is necessary to change the display magnification or scroll due to the difference between the resolution of the photographic image and the resolution of the displayed image has been described. In addition, there are also situations where it is necessary to change the display magnification or scroll due to the difference in aspect ratio, such as in panoramic photography, or due to the difference between the resolution of the image generated through interpolation processing or rendering (rather than the resolution of the photographic image itself) and the resolution of the displayed image. In these cases, the same effect as in the above embodiments can be obtained.

[0249] In the above embodiments, the wearable monitor, as an example of a wearable device equipped with a monitor, does not have a camera. However, it can also be a wearable monitor that further includes a camera and can take pictures independently without being combined with a photography device. In this case, when combined with a photography device, the wearable monitor is preferably a monitor that can also take pictures with the camera on the photography device side.

[0250] The descriptions and illustrations above are detailed explanations of the parts related to the technology of this invention, and are merely one example of the technology of this invention. For example, the descriptions related to the above structure, function, effect, and effect are examples of the structure, function, effect, and effect of the parts related to the technology of this invention. Therefore, without departing from the spirit of this invention, unnecessary parts may be deleted from the descriptions and illustrations above, or new elements may be added or replaced. Furthermore, to avoid complications and to facilitate understanding of the parts related to the technology of this invention, descriptions related to common technical knowledge that are not particularly necessary to explain in terms of enabling the implementation of this invention have been omitted from the descriptions and illustrations above.

[0251] All documents, patent applications and technical standards described in this specification, and the specific and separately described documents, patent applications and technical standards incorporated herein by reference, are incorporated herein by reference to the same extent.

[0252] Symbol Explanation

[0253] 1, 2 - Image display system; 10 - Smartphone; 11 - Housing; 11a - Front; 11b - Back; 15 - Smartphone casing; 16 - Casing section; 16b - Back; 17 - Opening; 18 - Ring controller; 21 - CPU; 22 - Memory; 23 - Storage; 24 - Touch panel; 24a - Zoom magnification input area; 24b - Display magnification input area; 25 - Communication unit; 26 - Photography unit; 26a - Photography lens; 27 - Attitude detection unit; 28 - Bus; 30 - Zoom magnification input unit; 30a - Magnification display unit; 30b - Magnification specification line; 31 - Shutter button. 32-Instant preview display unit, 50-Smart glasses, 51-Frame, 52-Lens, 61-CPU, 62-Memory, 63-Memory, 64-Display unit, 65-Communication unit, 66-Bus, 70-Server, 71-Network, CO, COa-Center position, CI-Center, D1-Image data, D2-Supplementary information data, D2a~D2e-Information, DA-Area, F1~F5-Image file, H, H1-Horizontal direction, HP-Horizontal direction, IM-Image, IP-Photographic image, IV-Display image, IX-Image, R-Radius, U-User, Z-Optical axis direction.

Claims

1. A display control device, which is a display control device for a wearable device having a monitor, comprising: At least one processor and memory built into or connected to said processor, The processor performs the following processing: Image acquisition processing, to acquire an image displayed on the monitor; The first operation instruction acquisition process acquires a first operation instruction in order to change the display magnification of the image. This first operation instruction is input via an operation of the operation unit of the camera device, which is separate from the wearable device, and via a magnification change operation within the camera device. The display control process changes the display magnification of the image according to the first operation instruction. The image contains magnification change operation related information, which relates to the amount of change in the magnification change operation relative to the amount of the operation performed during the capture in the photographic device that took the image. In the display control processing, the processor makes the change in the magnification of the operation amount relative to the shooting time consistent with the change in the operation amount relative to the display magnification of the image, based on the relevant information of the magnification change operation.

2. The display control device according to claim 1, wherein, The zoom ratio change operation is based on the zoom ratio change operation performed when shooting with the camera device.

3. The display control device according to claim 1 or 2, wherein, In the display control process, the processor controls the change in the display magnification of the image based on the change in the magnification relative to the operation amount of the magnification change operation.

4. The display control device according to claim 1 or 2, wherein, The processor also performs the following processing: The second operation instruction acquisition process acquires the change in the posture of the camera device detected by the posture detection unit, which detects the posture of the camera device, as the second operation instruction. In the display control process, the image is scrolled according to the second operation instruction.

5. The display control device according to claim 1 or 2, wherein, When the photographic device is equipped with a touch panel that functions as the operation unit, When the magnification change operation during shooting by the photographic device is a sliding operation within the first area of ​​the touch panel... The processor detects a swipe operation in the touch panel within a second region that includes the first region and is larger than the first region as an operation that changes the display magnification of the image.

6. The display control device according to claim 5, wherein, The processor detects arc-shaped sliding operations with a radius of 3cm or more but less than 8cm on the touch panel as the scaling operation.

7. The display control device according to claim 1 or 2, wherein, The image is an image captured by the photographic device.

8. The display control device according to claim 1 or 2, wherein, The processor is mounted on the wearable device. The processor acquires the image from an external server during the image acquisition process. In the first operation instruction acquisition process, the server acquires the first operation instruction input by operating the operation unit of the camera device.

9. The display control device according to claim 1 or 2, wherein, The processor is mounted on the wearable device. The processor acquires the image from an external server during the image acquisition process. In the first operation instruction acquisition process, the first operation instruction input by operation of the operation unit of the camera device is directly acquired from the camera device.

10. The display control device according to claim 1 or 2, wherein, The image contains information about the tilt angle, or pitch angle, of the photographic device relative to the horizontal direction when the image was captured. The processor determines the initial position of the center position when changing the display magnification of the image according to the tilt angle during the display control process.

11. The display control device according to claim 1 or 2, wherein, The image contains information about the roll angle, or rotation angle, of the photographic device about the optical axis relative to the horizontal direction when the image was captured. The processor corrects the horizontal direction of the image based on the roll angle during the display control process.

12. The display control device according to claim 4, wherein, The image contains perspective information related to the perspective of the photographic device when the image was captured. The processor, in the display control processing, corrects the amount of scrolling when the image is scrolled according to the second operation instruction based on the viewing angle information.

13. A display control method for a wearable device with a monitor, comprising the following steps: Image acquisition and processing steps: Acquire an image displayed on the monitor; The first operation instruction acquisition and processing step acquires a first operation instruction in order to change the display magnification of the image. The first operation instruction is input via an operation of the operation unit of the camera device, which is separate from the wearable device, and via a magnification change operation within the camera device. The display control processing step involves changing the display magnification of the image according to the first operation instruction. The image contains magnification change operation related information, which relates to the amount of change in the magnification change operation relative to the amount of the operation performed during the capture in the photographic device that took the image. In the display control processing step, the change in the magnification of the magnification change operation relative to the operation amount during shooting is made consistent with the change in the operation amount relative to the operation amount when changing the display magnification of the image, based on the relevant information of the magnification change operation.

14. A program product comprising a display control program, said display control program being a display control program for a wearable device having a monitor, said display control program causing a computer to perform the following steps: Image acquisition and processing steps: Acquire an image displayed on the monitor; The first operation instruction acquisition and processing step acquires a first operation instruction in order to change the display magnification of the image. The first operation instruction is input via an operation of the operation unit of the camera device, which is separate from the wearable device, and via a magnification change operation within the camera device. The display control processing step involves changing the display magnification of the image according to the first operation instruction. The image contains magnification change operation related information, which relates to the amount of change in the magnification change operation relative to the amount of the operation performed during the capture in the photographic device that took the image. In the display control processing step, the change in the magnification of the magnification change operation relative to the operation amount during shooting is made consistent with the change in the operation amount relative to the operation amount when changing the display magnification of the image, based on the relevant information of the magnification change operation.

Images