Imaging system capable of appropriately setting orientation information, imaging device, control method of imaging device, and program

Abstract

[PROBLEM] Provided is an imaging system whereby an appropriate orientation value for displaying an image in the same direction as in the real world can be included in an image file generated by image capture using a light beam bending device for bending an external light beam and sending the light beam to an imaging device. [SOLUTION] An imaging system (10) comprises: a light beam bending device (100) that bends an external light beam; and an imaging device (200) that generates image data on the basis of the light beam bent by the light beam bending device (100). The imaging device (200) generates an image file including an orientation value corresponding to the installation direction of the imaging device (200) with respect to the light beam bending device (100) and still image data generated by the imaging sequence of S327.

Description

Image capture system, image capture device, image capture device control method, and program capable of appropriately setting orientation information

[0001] The present disclosure relates to an imaging system, an imaging device, a control method for an imaging device, and a program that are capable of appropriately setting orientation information.

[0002] Conventionally, in photographing with an imaging device, control is performed to track a desired subject by performing pan and tilt at high speed. In this control, a technique is known in which, instead of moving the heavy imaging device itself, a movable mirror built into the imaging device is used to bend the optical axis of the imaging device to improve the pan and tilt operation speed for tracking the desired subject (see Patent Document 1).

[0003] An image file containing image data captured by an imaging device includes an Orientation value, which is used to control the display of the image data and indicates the image orientation when the image based on the image data is displayed. By using the Orientation value, the image can be displayed in the same orientation as in the real world. The Orientation value is determined based on the orientation of the imaging device, such as horizontal or vertical.

[0004] Japanese Patent Application Laid-Open No. 2001-28706

[0005] Meanwhile, development of a system in which a ray bending means, such as a movable mirror, that bends external light rays is provided externally to the imaging device rather than internally is being considered. In such a system, the imaging device photographs the subject through the ray bending means rather than directly, and image data is obtained that shows the actual subject in a left-right inverted state based on the external light rays bent by the ray bending means. In order to display an image based on such image data in the same orientation as in the real world, the imaging device must determine an Orientation value taking into account the ray bending means provided externally. In other words, the conventional Orientation value determined based on the attitude of the imaging device cannot display an image in the same orientation as in the real world.

[0006] The present disclosure provides a technology that enables an image file generated by photographing using a light bending device that bends external light rays and sends them to an imaging device to include an appropriate Orientation value that displays the image in the same orientation as in the real world.

[0007] The imaging system of the present disclosure comprises a ray bending device that bends external light rays, an imaging device that generates image data based on the light rays bent by the ray bending device, and means for detecting the installation direction of the imaging device relative to the ray bending device, and is characterized in that the imaging device generates an image file including the image data and a value corresponding to the detected installation direction of the imaging device, the value being used to control the display of the image data.

[0008] According to the present disclosure, an image file generated by capturing an image using a light bending device that bends external light rays and sends them to an imaging device can include an appropriate Orientation value that displays the image in the same orientation as in the real world.

[0009] FIG. 1 is an external view showing a physical layout of an imaging system according to an embodiment of the present disclosure and a state of incident light. FIG. 2 is a block diagram schematically showing the configuration of the beam bending device and imaging device of FIG. 1. FIG. 3 is a flowchart showing the procedure of an imaging control process executed by the imaging system of FIG. 1. FIG. 4 is a flowchart showing the procedure of an imaging control process executed by the imaging system of FIG. 1. FIG. 5 is a diagram for explaining Orientation values ​​used in the present embodiment. FIG. 6 is a diagram for explaining Orientation values ​​used in the present embodiment. FIG. 7 is a diagram for explaining Orientation values ​​used in the present embodiment. FIG. 8 is a diagram for explaining Orientation values ​​used in the present embodiment. FIG. 9 is a diagram for explaining Orientation values ​​used in the present embodiment. FIG. 10 is a diagram for explaining Orientation values ​​used in the present embodiment. FIG. 11 is a diagram for explaining Orientation values ​​used in the present embodiment. FIG. 12 is a diagram for explaining the state of the imaging device when imaging is handheld without using the beam bending device of FIG. 2. FIG. 13 is a diagram for explaining the state of the imaging device when imaging is handheld without using the beam bending device of FIG. 2. FIG. 3 is a diagram showing a state of the imaging device when the imaging device is handheld and not using the beam bending device of FIG. 2. FIG. 4 is a diagram showing a state of the imaging device when the imaging device is handheld and not using the beam bending device of FIG. 2. FIG. 5 is a diagram showing a state in which the imaging device is installed in the beam bending device of FIG. 1. FIG. 6 is a diagram showing a state in which the imaging device is installed in the beam bending device of FIG. 1. FIG. 7 is a diagram showing a state in which the imaging device is installed in the beam bending device of FIG. 1. FIG. 8 is a diagram showing a state in which the imaging device is installed in the beam bending device of FIG. 1. FIG. 9 is a diagram showing an example of still image data generated by imaging in the state of FIG. 5A. FIG. 10 is an example of still image data generated by imaging in the state of FIG. 5C. FIG. 11 is an example of still image data generated by imaging in the state of FIG. 6C. FIG. 11 is an example of still image data generated by imaging in the state of FIG. 6D. FIG. 12 is an example of still image data generated by imaging in the state of FIG. 5D. FIG. 13 is an example of still image data generated by imaging in the state of FIG. 6B. FIG. 14 is an example of still image data generated by imaging in the state of FIG. 5B.3B. 3C is a table summarizing the correspondence between Orientation values, display arrangement of pixel information when displayed on a display means of an information processing device, memory arrangement of pixel information when capturing images, and physical arrangement of the imaging device. 3D is a diagram for explaining determination of the elevation angle of the imaging device in S331 of FIG. 3B. 3E is a diagram for explaining determination of the elevation angle of the imaging device in S331 of FIG. 3B. 3F is a diagram for explaining calculation of the orientation of the optical axis after bending in S332 of FIG. 3B. 3G is a diagram for explaining calculation of the orientation of the optical axis after bending in S332 of FIG. 3B. 3H is a diagram for explaining calculation of the orientation of the optical axis after bending in S332 of FIG. 3B.

[0010] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0011] 1 is an external view showing the physical layout of an imaging system 10 according to an embodiment of the present disclosure and the state of incident light. Note that in FIG. 1, the internal configuration is shown transparently for ease of understanding.

[0012] In FIG. 1, an imaging system 10 includes a beam bending device 100 and an imaging device 200 .

[0013] An imaging device 200 is fixed and installed inside the beam bending device 100 by installation direction detection means 114 shown in Fig. 2, which will be described later. The imaging device 200 is installed with its lens facing the upper side of the beam bending device 100.

[0014] A wide-angle optical system 101 and an image sensor 102 are provided on the upper part of the light bending device 100. The wide-angle optical system 101 acquires light rays with a wide angle of view indicated by lines 153a and 153b, and the image sensor 102 acquires image data with the wide angle of view.

[0015] A tilt mirror 113 is also provided on the upper part of the beam bending device 100. The tilt mirror 113 is disposed at a reference position where the angle between the optical axis of the imaging device 200 and the vertical axis of the tilt mirror 113 is 45 degrees, and is rotatable by a tilt mirror motor 111 shown in FIG. 2 (described later). For example, when the tilt mirror 113 is in the reference position, the angle of view from which the imaging device 200 can acquire light rays is the region indicated by lines 151a and 151c. At this time, among the light rays acquired by the imaging device 200, the light rays that are bent by the tilt mirror 113 and positioned on the optical axis of the imaging device 200 are light rays 151b. Furthermore, when the tilt mirror 113 is rotated to the position indicated by 113' in FIG. 1, the angle of view from which the imaging device 200 can acquire light rays is the region indicated by lines 152a and 152c. At this time, among the light rays obtained by the imaging device 200, the light rays that are bent by the tilt mirror 113 and positioned on the optical axis of the imaging device 200 become light rays 152b.

[0016] A pan axis rotation motor 108 is provided on the bottom surface of the beam bending device 100 for rotating the beam bending device 100 in which the imaging device 200 is installed.

[0017] FIG. 2 is a block diagram showing a schematic configuration of the beam bending device 100 and the imaging device 200 shown in FIG.

[0018] First, a description will be given of the configuration of the beam bending device 100. In addition to the wide-angle optical system 101, image sensor 102, pan axis rotation motor 108, and tilt mirror 113 described above, the beam bending device 100 also includes image processing HW 103, memory 104, recognition HW 105, CPU 106, and driver circuit 107, as shown in Fig. 2. The beam bending device 100 further includes a rotation position sensor 109, a driver circuit 110, a tilt mirror motor 111, a rotation position sensor 112, installation direction detection means 114, a GPS 115, and a communication unit 116.

[0019] As described above, the wide-angle optical system 101 acquires light rays with a wide angle of view. The image sensor 102 converts the light rays acquired by the wide-angle optical system 101 into electrical signals. The image processing HW 103 performs image processing such as conversion to an appropriate image size, noise reduction, and other image processing on image data based on the electrical signals converted by the image sensor 102, and stores the processed image data in the memory 104. The recognition HW 105 performs object recognition processing based on the image data stored in the memory 104. In the object recognition processing, an object area is detected from an image based on this image data, and coordinate information indicating the position of this object area within the image and movement information of this object area are detected. The information detected by the recognition HW 105 is output to the CPU 106.

[0020] The CPU 106 includes a memory (not shown) and executes programs stored in the memory or the like to perform various controls. For example, the CPU 106 selects a main subject area from among multiple subject areas detected by the recognition HW 105 and acquires the position of the main subject area within the current angle of view. The CPU 106 controls the pan axis rotation motor 108 and the tilt mirror motor 111 so that the main subject area is positioned at the center of the angle of view. The control of these two motors will now be described in detail.

[0021] The control of the pan axis rotation motor 108 will now be described. The rotation position of the pan axis rotation motor 108 is detected by the rotation position sensor 109, and the CPU 106 acquires this rotation position. The CPU 106 calculates a target rotation position to be reached by the pan axis rotation motor 108 based on this rotation position and the lateral position of the main subject area within the angle of view, and calculates a voltage control amount corresponding to this target rotation position. The CPU 106 controls the drive voltage of the pan axis rotation motor 108 based on the calculated voltage control amount. Here, feedback processing such as PID control based on the difference between the current rotation position and the target rotation position is performed, for example. Note that in FIG. 2 , the pan axis rotation motor 108 and the wide-angle optical system 101 are connected by a line, which indicates that the direction of the wide-angle optical system 101 changes in response to the rotation of the pan axis rotation motor 108.

[0022] Next, the control of the tilt mirror motor 111 will be described. The rotational position of the tilt mirror motor 111 is detected by the rotational position sensor 112, and the CPU 106 acquires this rotational position. Based on this rotational position and the vertical position of the main subject area within the angle of view, the CPU 106 calculates a target rotational position to be reached by the tilt mirror motor 111, and calculates a voltage control amount corresponding to this target rotational position. The CPU 106 controls the drive voltage of the tilt mirror motor 111 based on the calculated voltage control amount. Here, as with the pan axis rotation motor 108, feedback processing such as PID control based on the difference between the current rotational position and the target rotational position is performed. Through this series of processing, the tilt mirror 113 is controlled to the desired tilt angle. Note that in FIG. 2, the pan axis rotation motor 108 and the tilt mirror motor 111 are connected by a line. This indicates that in response to the rotation of the pan axis rotation motor 108, the tilt mirror motor 111, the rotation position sensor 112 connected to the tilt mirror motor 111, and the tilt mirror 113 also rotate.

[0023] Tilt mirror 113 serves to send light rays for generating image data to image capture device 200. Specifically, tilt mirror 113 refracts external light rays and sends the light rays to telephoto optical system 201 of image capture device 200. The reflective surface of tilt mirror 113 corresponds to the "refracting optical surface" in the claims. The reflective surface of tilt mirror 113 is located at a position where it intersects with the optical axis of image capture device 200.

[0024] The installation direction detection means 114 detects the installation direction of the imaging device 200 installed in the beam bending device 100. In this embodiment, the imaging device 200 installed in the beam bending device 100 has installation freedom in 90-degree increments around the optical axis, as will be described in detail later. The installation direction detection means 114 is composed of a mechanical detection lever provided at an appropriate location on the beam bending device 100 on which the imaging device 200 is installed, a photointerrupter that detects the tilting state of the lever, and the like. The GPS 115 acquires the geographical position and orientation from a GPS system. The communication unit 116 communicates various information with the imaging device 200. The GPS 115 corresponds to the "orientation detection means" in the claims.

[0025] Next, a description will be given of the configuration of the imaging device 200. As shown in Fig. 2, the imaging device 200 includes a telephoto optical system 201, an image sensor 202, image processing HW 203, a memory 204, recognition HW 205, a CPU 206, an SD slot 207, an SD card 208, a GPS 209, an acceleration sensor 210, and a communication unit 211.

[0026] The telephoto optical system 201 acquires light rays with a telephoto angle of view via the tilt mirror 113. The image sensor 202 converts the light rays acquired by the telephoto optical system 201 into an electrical signal. The image processing HW 203 performs image processing, such as conversion to an appropriate image size, noise reduction, and other image processing, on image data based on the electrical signal converted by the image sensor 202, and stores the processed image data in the memory 204. The recognition HW 205 performs object recognition processing based on the image data stored in the memory 204. In this object recognition processing, as in the object recognition processing by the recognition HW 105 described above, an object area is detected from an image based on the image data. Coordinate information indicating the position of this object area within the image and movement information of this object area are also detected. The information detected by the recognition HW 205 is output to the CPU 206.

[0027] The CPU 206 includes a memory (not shown) and executes programs stored in the memory to perform various control operations. The SD card 208 stores image files via the SD slot 207. These image files include, for example, processed image data obtained by appropriately processing image data recorded in the memory 204, and various information determined and calculated according to the flowchart in FIG. 3B (described later). The GPS 209 acquires geographical position and orientation from a GPS system. The GPS 209 corresponds to the "orientation detection means" in the claims. The acceleration sensor 210 has the function of detecting gravitational acceleration in three axes and can three-dimensionally detect the tilt of the imaging device 200. The communication unit 211 communicates various information with the light bending device 100.

[0028] Figures 3A and 3B are flowcharts showing the procedure of the imaging control process executed by the imaging system 10 of Figure 1. The flow of this imaging control process will be described below using the flowcharts of Figures 3A and 3B and Figures 4A to 10C. This imaging control process is performed by the light bending device 100 and the imaging device 200. Figure 3A is a flowchart showing the procedure of the control process by the light bending device 100. Figure 3B is a flowchart showing the procedure of the control process by the imaging device 200. The imaging control process is realized by communicating between the light bending device 100 and the imaging device 200 to exchange various information.

[0029] 3A will be described. The control process by the beam bending device 100 is realized by the CPU 106 executing a program stored in the memory of the CPU 106 or the like.

[0030] 3A, in S301, the CPU 106 detects a user's instruction to power on the beam bending device 100. Note that in this embodiment, the power on instruction is assumed to be a power on action by the user using the power button, etc. Next, this processing proceeds to S302. At this time, the user issues an instruction to the imaging device 200 to perform automatic shooting. As a result, the control processing by the imaging device 200 in FIG. 3B proceeds to S322.

[0031] 3A, in S302, the CPU 106 acquires state information of the installation direction detection means 114. The processing of S302 will now be described in detail with reference to FIG.

[0032] 6A to 6D are diagrams showing a state in which the imaging device 200 is installed in the beam bending device 100 of Fig. 1. In Fig. 6A to 6D, the optical axis of the imaging device 200 is the z-axis, the rotation axis of the tilt mirror 113 is the x-axis, and the axis perpendicular to the x-axis and z-axis is the y-axis.

[0033] Comparing the two diagrams, FIG. 6A and FIG. 6B, the viewfinder 601 of the imaging device 200 is located in the + direction of the y-axis in FIG. 6A, and the viewfinder 601 of the imaging device 200 is located in the + direction of the x-axis in FIG. 6B. Generally, when photographing with the imaging device 200 held by hand, horizontal photography is performed by holding the imaging device 200 horizontally (horizontal position), and vertical photography is performed by rotating the imaging device 200 90 degrees and holding it vertically (vertical position). In contrast, the configuration of FIG. 6A corresponds to photographing at a horizontal angle via the tilt mirror 113, and the configuration of FIG. 6B corresponds to photographing at a vertical angle via the tilt mirror 113. Note that in this embodiment, the imaging device 200 can also be installed as shown in FIG. 6C to photograph at a horizontal angle via the tilt mirror 113. It is also possible to install the imaging device 200 as shown in FIG. 6D to photograph at a vertical angle via the tilt mirror 113. In this embodiment, it is possible to identify in which of the configurations shown in Figures 6A, 6B, 6C, and 6D the imaging device 200 is installed, based on the status information of the installation orientation detection means 114. The CPU 106 controls the communication unit 116 to transmit the acquired status information of the installation orientation detection means 114 to the imaging device 200.

[0034] Next, in S303, the CPU 106 performs subject recognition processing by controlling the recognition HW 105. In this subject recognition processing, the recognition HW 105 uses image data obtained from light rays captured by the wide-angle optical system 101 via the image sensor 102, the image processing HW 103, and the memory 104 to detect a subject area from an image based on this image data.

[0035] Next, in S304, the CPU 106 determines a main subject region to be captured from among the one or more detected subject regions. It is assumed that the method for determining the main subject region in S304 is predetermined. For example, there are various methods, such as determining a specific person region or a specific object region as the main subject region. Here, it is assumed that one main subject region is determined using any of these methods. At this time, information such as the position and size of the main subject region within the angle of view is calculated, and the type of the main subject region, such as a person or a dog, is identified. This information is stored in the memory of the CPU 106 and transmitted to the image capture device 200. This information is used in the process of determining the main subject region in the image capture device 200 (see S325, described later).

[0036] Next, in S305, CPU 106 generates a pan rotation position command for controlling pan axis rotation motor 108 and a tilt rotation position command for controlling tilt mirror motor 111. Specifically, based on the calculated position of the main subject area in the image and information from rotation position sensor 109 and rotation position sensor 112, it is calculated to what positions pan axis rotation motor 108 and tilt mirror motor 111 should be rotated to, respectively, so that the main subject area is located at the center of the angle of view. CPU 106 generates a pan rotation position command and a tilt rotation position command based on the calculation results.

[0037] Next, in S306, the CPU 106 performs motor position feedback processing. In the motor position feedback processing, the CPU 106 controls the drive voltage of each motor based on the difference between the current rotational position of each motor and the rotational position corresponding to each rotational position command generated in S305. In this way, the pan axis rotation motor 108 and the tilt mirror motor 111 are controlled so as to move to their respective desired rotational positions.

[0038] Next, in S307, CPU 106 determines whether or not an inquiry about angular position information of tilt mirror motor 111 has been received from image capture device 200. Note that the process of image capture device 200 inquiring about angular position information of tilt mirror motor 111 corresponds to the process of S329 in FIG. 3B , which will be described later. If it is determined that an inquiry has been received, this process proceeds to S308, and if it is determined that an inquiry has not been received, this process proceeds to S309.

[0039] In S308, the CPU 106 transmits information indicating the rotational position of the tilt mirror motor 111 detected by the rotational position sensor 112 as angular position information of the tilt mirror motor 111 to the imaging device 200 via the communication unit 116.

[0040] Next, in S309, the CPU 106 determines whether or not it has detected a user's instruction to power off the beam bending device 100. Note that the power off instruction in this embodiment is assumed to be a power off action by the user using a power button, or the like.

[0041] If it is determined that the instruction to turn off the power of the beam bending device 100 has not been detected, the process returns to S303. If it is determined that the instruction to turn off the power of the beam bending device 100 has been detected, the process proceeds to S310.

[0042] In S310, the CPU 106 performs a termination process for the beam bending device 100. In this termination process, information that is used in the processes being performed in the beam bending device 100 and that will be needed later is saved or stored, and various drive mechanisms are reset to their initial positions. After that, this process ends.

[0043] Next, a control process by the imaging device 200 in Fig. 3B will be described. The control process by the imaging device 200 is realized by the CPU 206 executing a program stored in the memory or the like of the CPU 206. Note that the control process by the imaging device 200 in Fig. 3B is performed assuming that the imaging device 200 has already been started.

[0044] In FIG. 3B, in S321, the CPU 206 detects an automatic shooting instruction from the user.

[0045] Next, in S322, the CPU 206 determines whether or not it has received the status information of the installation direction detection unit 114 that was acquired and transmitted in S302 by the light bending device 100. The CPU 206 waits until it receives the status information of the installation direction detection unit 114, and when it receives the status information of the installation direction detection unit 114 (YES in S322), the process proceeds to S323.

[0046] In S323, the CPU 206 stores the received state information of the installation direction detection unit 114 in the memory of the CPU 206 or the like.

[0047] Next, in S324, the CPU 206 controls the recognition HW 205 to perform object recognition processing on an image based on the image data recorded in the memory 204. In this embodiment, as described above in the description of FIG. 2, light rays acquired by the telephoto optical system 201 are converted into electrical signals by the image sensor 202. The image processing HW 203 then performs image processing, such as conversion to an appropriate image size, noise reduction, and other image processing, on the image data based on these electrical signals, and the processed image data is recorded in the memory 204. In the object processing in S324, accompanying information, such as the type of object area recognized in the image based on the image data recorded in the memory 204 and coordinates indicating its position in the image, is calculated.

[0048] Next, in S325, the CPU 206 controls the recognition HW 205 to determine a main subject region from among the one or more subject regions detected in S324. In S325, the main subject region is determined based on information transmitted from the light bending device 100 in S304, for example, information indicating the type of the main subject region determined in S304. In this way, the same subject region is determined as the main subject region in the process of S304 by the light bending device 100 and the process of S325 by the image capturing device 200. At this time, accompanying information such as the position and size of the main subject region within the angle of view is also calculated, and this accompanying information is stored in the memory of the CPU 206, etc.

[0049] Next, in S326, CPU 206 determines whether the accompanying information of the main subject region determined in S325 satisfies the shooting conditions. This shooting condition is, for example, a condition such as whether the main subject region is captured at a certain size or greater within a certain range within the angle of view. If it is determined that the accompanying information of the main subject region determined in S325 does not satisfy the shooting conditions, the process returns to S324. Thus, in this embodiment, the still image shooting sequence is not performed until the accompanying information of the main subject region satisfies the shooting conditions.

[0050] If it is determined in S325 that the accompanying information of the main subject region determined satisfies the shooting conditions, the process proceeds to S327. In S327, the CPU 206 executes a still image capture sequence. In this capture sequence, the series of processes from the time when image data obtained based on light rays acquired by the telephoto optical system 201 to the time when the image data is recorded in the memory 204 is the same as the processes described above. However, this capture sequence differs from the processes described above in that it also acquires high-resolution still image data that is intended to be saved in the SD card slot 207. As described above, in this embodiment, once the user issues an automatic capture instruction, the still image capture sequence is executed in accordance with the fact that the accompanying information of the main subject region satisfies the shooting conditions, even without subsequently issuing a shooting instruction such as pressing the shutter button.

[0051] Next, in S328, the CPU 206 determines a value to be set as the Orientation value based on the status information of the installation orientation detection means 114. The Orientation value is a value of the EXIF ​​information included in the image file together with the still image data generated in S327. The Orientation value is a value used for display control of this still image data, and more specifically, a value representing the image orientation when a still image based on this still image data is displayed. The determination method of S328 will be described in detail using FIGS. 4A to 8.

[0052] 4A to 4H are diagrams illustrating the Orientation values ​​used in this embodiment. Here, an example is described in which an image file is stored in an information processing device, such as a PC (not shown), and a still image based on the still image data contained in the image file is displayed on the display means of the information processing device. In this case, FIGS. 4A to 4H illustrate how a group of pixel information is aligned and displayed as a still image based on the Orientation value of the EXIF ​​information contained in the image file. Note that the memory of the information processing device holds a group of pixel information for C columns by R rows for displaying a still image based on the still image data, and the stored coordinates of each pixel in memory are represented by (column, row).

[0053] 4A shows a display layout when the Orientation value is "1." In this case, the display means of the information processing device displays a still image in an orientation such that, in the pixel information group arranged in the memory of the information processing device, the pixel information at stored coordinates (0, 0) is arranged in the upper left, the pixel information at stored coordinates (C, 0) is arranged in the upper right, the pixel information at stored coordinates (0, R) is arranged in the lower left, and the pixel information at stored coordinates (C, R) is arranged in the lower right.

[0054] Figure 4B shows a display layout when the Orientation value is "2," which is rotated 90 degrees from Figure 4A. In this case, the display means of the information processing device displays a still image in an orientation such that, in the pixel information group arranged in the memory of the information processing device, the pixel information at stored coordinates (C, 0) is arranged in the upper left, the pixel information at stored coordinates (0, 0) is arranged in the upper right, the pixel information at stored coordinates (C, R) is arranged in the lower left, and the pixel information at stored coordinates (0, R) is arranged in the lower right. Figures 4C to 4H show that the Orientation value is changed one by one, and the still image is displayed with the pixel arrangement corresponding to each change.

[0055] 5A to 5D are diagrams illustrating the state of the imaging device 200 when the imaging device 200 is handheld and photographed without using the beam bending device 100 of FIG. 2. FIG. 5A is a diagram illustrating the state when the imaging device 200 is positioned horizontally to photograph a subject. FIG. 5B is a diagram illustrating the state when the imaging device 200 is rotated 90 degrees counterclockwise around its optical axis from the state of FIG. 5A to photograph a subject in a vertical position to photograph a subject. FIG. 5C is a diagram illustrating the state when the imaging device 200 is rotated 90 degrees counterclockwise around its optical axis from the state of FIG. 5B. FIG. 5D is a diagram illustrating the state when the imaging device 200 is rotated 90 degrees counterclockwise around its optical axis from the state of FIG. 5C.

[0056] 6A to 6D are diagrams illustrating a state in which the imaging device 200 is installed in the beam bending device 100 of FIG. 2. FIG. 6A is a diagram illustrating a state in which the imaging device 200 is installed in the beam bending device 100 so as to take images in a horizontal position. FIG. 6B is a diagram illustrating a state in which the imaging device 200 is installed in the beam bending device 100 so as to take images in a vertical position. FIG. 6C is a diagram illustrating a state in which the imaging device 200 is installed after being rotated 90 degrees counterclockwise around its optical axis from the state of FIG. 6B. FIG. 6D is a diagram illustrating a state in which the imaging device 200 is installed after being rotated 90 degrees counterclockwise around its optical axis from the state of FIG. 6C. Note that, in this embodiment, as described above, it is possible to identify in which of the configurations of FIG. 6A, FIG. 6B, FIG. 6C, and FIG. 6D the imaging device 200 is installed, based on the state information of the installation direction detection means 114.

[0057] Figures 7A to 7H are examples of still image data generated by capturing images in any of the states shown in Figures 5A to 5D and 6A to 6D. Note that Figures 7A to 7H each show still image data obtained by capturing an image of a skier skiing from the upper left to the lower right. Note that Figures 7A to 7H also show the storage coordinates of each pixel of the upper left, upper right, lower left, and lower right of the still image data in the memory of the information processing device.

[0058] FIG. 7A shows still image data obtained by photographing in the state of FIG. 5A , i.e., horizontal photography with the handheld imaging device 200 without using the beam bending device 100. FIG. 7B shows still image data obtained by photographing in the state of FIG. 6A , i.e., horizontal photography with the imaging device 200 installed in the beam bending device 100. FIG. 7C shows still image data obtained by photographing in the state of FIG. 5C . FIG. 7D shows still image data obtained by photographing in the state of FIG. 6C . FIG. 7E shows still image data obtained by photographing in the state of FIG. 6D . FIG. 7F shows still image data obtained by photographing in the state of FIG. 5D . FIG. 7G shows still image data obtained by photographing in the state of FIG. 6B , i.e., vertical photography with the imaging device 200 installed in the beam bending device 100. FIG. 7H shows still image data obtained by photographing in the state of FIG. 5B , i.e., vertical photography with the handheld imaging device 200 without using the beam bending device 100.

[0059] When a still image based on still image data obtained by handheld shooting with the imaging device 200 is displayed on the display means of the information processing device, if the Orientation value is determined based on the posture (horizontal position, vertical position) of the imaging device 200, the still image will be displayed in the same orientation as in the real world. As an example, a case will be described in which a still image based on the still image data of FIG. 7A is displayed on the display means of the information processing device. As described above, the still image data of FIG. 7A is still image data obtained by shooting in the state of FIG. 5A , that is, by horizontal shooting with the handheld imaging device 200 without using the light bending device 100. When a still image is displayed based on such still image data, if the display layout shown in FIG. 4A is set to a display layout with an Orientation value of "1," a still image of a skier gliding from the upper left to the lower right, as in the real world, is displayed on the display means of the information processing device.

[0060] On the other hand, when a still image based on still image data obtained by imaging using the beam bending device 100 is displayed on the display means of an information processing device, if the Orientation value is determined based on the attitude of the imaging device 200, the still image will not be displayed in the same orientation as in the real world. As an example, a case will be described in which a still image based on the still image data of FIG. 7B is displayed on the display means of an information processing device. As described above, the still image data of FIG. 7B is still image data obtained by imaging in the state of FIG. 6A , i.e., by swift imaging using the imaging device 200 installed on the beam bending device 100. The still image data of FIG. 7B is still image data obtained by bending light rays from a subject with the tilt mirror 113 and allowing them to enter the telephoto optical system 201 of the imaging device 200. At this time, because the light rays are reflected once by the tilt mirror 113, the light entering the telephoto optical system 201 is reversed left and right. 7B obtained by imaging using the beam bending device 100 is still image data obtained by left-right inverting the still image data of Fig. 7A obtained by the same intercepted imaging and imaging using the handheld imaging device 200. When displaying a still image based on such still image data of Fig. 7B, if the display layout is set to have an Orientation value of "1" as shown in Fig. 4A, a still image that is left-right inverted from the real world will be displayed on the display means of the information processing device.

[0061] Therefore, in this embodiment, the value to be set for the Orientation value is determined based on whether the installation orientation of image capture device 200 with respect to tilt mirror 113 is one of FIGS. 6A, 6B, 6C, and 6D. As described above, the installation orientation of image capture device 200 with respect to tilt mirror 113 can be identified based on the status information of installation orientation detection means 114. For example, if the installation orientation of image capture device 200 with respect to tilt mirror 113 is that of FIG. 6A, the value to be set for the Orientation value is determined to be "2." As a result, when a still image is displayed based on the still image data of FIG. 7B, a display layout with an Orientation value of "2" as shown in FIG. 4B is used, and a still image of a skier gliding from the upper left to the lower right, as in the real world, is displayed on the display means of the information processing device. As described above, in this embodiment, an appropriate Orientation value is determined based on the installation direction of image capture device 200 relative to tilt mirror 113 so that a still image is displayed in the same orientation as in the real world for image data obtained by imaging using beam bending device 100. (See, for example, FIG. 8.) FIG. 8 is a table summarizing the correspondence between Orientation values, the display arrangement of pixel information when displayed on the display means of the information processing device shown in FIGS. 4A to 4H, the memory arrangement of pixel information when imaging shown in FIGS. 7A to 7H, and the physical arrangement of image capture device 200 shown in FIGS. 5A to 5D and 6A to 6D.

[0062] 3B , in S329, the CPU 206 controls the communication unit 211 to inquire of the beam bending device 100 about the angular position information of the tilt mirror motor 111. Upon receiving this inquiry, the beam bending device 100 transmits, in the above-mentioned S308, information indicating the rotational position of the tilt mirror motor 111 detected by the rotational position sensor 112 to the imaging device 200 as angular position information of the tilt mirror motor 111.

[0063] In S330, CPU 206 determines whether or not it has received the angular position information of tilt mirror motor 111 inquired about in S329. CPU 206 waits until it receives the angular position information of tilt mirror motor 111, and when it receives the angular position information of tilt mirror motor 111 (YES in S330), the process proceeds to S331.

[0064] In S331, the CPU 206 determines a value to be set for the elevation angle (Camera Elevation Angle value) of the image capture device 200 defined in the EXIF ​​standard, based on the angular position information of the tilt mirror motor 111. Note that the Camera Elevation Angle value is also a value of the EXIF ​​information included in the image file together with the still image data generated in S327. The processing of S331 will be described in detail using Figures 9A and 9B.

[0065] 9A is a diagram showing a state in which tilt mirror 113 is placed at a reference position. In FIG. 9A, the horizontal direction is the X axis, and the vertical direction is the Y axis, and tilt mirror 113 is shown tilted at 45 degrees with respect to the X axis and Y axis. At this time, optical axis 901 of imaging device 200 is reflected by tilt mirror 113. The optical axis reflected by tilt mirror 113 is referred to as reflected optical axis 903.

[0066] In addition, in Figure 9A, optical axis 901 and reflected optical axis 903 each form an angle of 45 degrees with respect to normal 902 of tilt mirror 113. The state of tilt mirror 113 in Figure 9A is defined as a state in which the tilt mirror tilt angle is 0 degrees. When tilt mirror motor 111 is rotated in this state, tilt mirror 113 tilts and the direction of reflected optical axis 903 changes. Here, the direction in which tilt mirror motor 111 rotates clockwise is defined as the positive angle direction. Figure 9B shows a state in which the tilt mirror tilt angle is a positive value of α degrees.

[0067] In FIG. 9B , optical axis 901 is reflected by tilt mirror 113 to become reflected optical axis 905, which is oriented angularly θ upward from reflected optical axis 903. When tilt mirror motor 111 tilts by α, normal 904 of tilt mirror 113 also tilts by α from normal 902 of tilt mirror 113. Due to the constraints of mirror reflection, the angle between normal 904 of tilt mirror 113 and optical axis 901 is the same as the angle between normal 904 of tilt mirror 113 and reflected optical axis 905, and this angle is defined as β. This leads to the relationship β = 45 degrees + α. Furthermore, the angle between optical axis 901 of imaging device 200 and reflected optical axis 905 can be expressed as 2β, which can also be expressed as 2β = 90 degrees + 2α.

[0068] Here, when the angle formed by the optical axis 901 and the reflected optical axis 905 of the imaging device 200 is expressed using θ, it becomes 90 degrees + θ. Therefore, the relationship θ = 2α can be derived. In other words, when the tilt mirror 113 is tilted by α degrees from the reference position, the reflected optical axis changes in tilt by an angle of 2α from the horizontal direction. Note that the angle α in the above explanation corresponds to "information indicating the rotational position of the motor for rotating the bending optical surface" in the claims, and the angle θ corresponds to "elevation angle information" in the claims.

[0069] That is, in S331, the CPU 206 acquires a value corresponding to α in Figures 9A and 9B from the beam bending device 100 as angular position information of the tilt mirror motor 111, and determines the value obtained by doubling that value as the value to be set for the elevation angle (CameraElevationAngle).

[0070] Next, in S332, the CPU 206 calculates the orientation of the optical axis after bending based on the orientation information detected by the GPS 209 included in the imaging device 200 and the angular position information of the tilt mirror motor 111 received in S330. The processing of S332 will be described in detail with reference to Figures 10A to 10C.

[0071] 10A , like FIG. 5A described above, is a diagram showing a state in which landscape photography is performed by holding the image capture device 200 in a landscape position (normal position). The GPS 209 is provided inside the image capture device 200 and can capture the north, south, east, and west directions in three dimensions: x, y, and z. For example, in FIG. 10A , if the z direction is north, the GPS 209 perceives the front side of the image capture device 200 (the side having the telephoto optical system 201) as north. In this case, if the image capture device 200 is rotated around the x axis to change the orientation of the telephoto optical system 201 upward, the GPS 209 will perceive the bottom side of the image capture device 200 (the side opposite the top side having the viewfinder unit 1001) as north.

[0072] 10B and 10C are diagrams showing the state in which the imaging device 200 is installed in the beam bending device 100. The installation orientation of the imaging device 200 relative to the tilt mirror 113 differs between FIGS. 10B and 10C. In these two installation states, the installation orientation of the imaging device 200 relative to the tilt mirror 113 can be detected by the installation orientation detection unit 114. For example, in the case of FIG. 10B , the optical axis after bending is optical axis 1002, and the orientation of optical axis 1002 coincides with the orientation of the top surface (the surface having the viewfinder unit 1001) of the imaging device 200 among the orientations detectable by the GPS 209. Therefore, in this case, the orientation of the top surface of the imaging device 200 is selected from the three-dimensional orientation information detected by the GPS 209, and this orientation is calculated as the orientation of the optical axis after bending. 10C , the optical axis after bending is optical axis 1003, and the orientation of optical axis 1003 coincides with the orientation of the right side of image capture device 200 among the orientations detectable by GPS 209. Therefore, in this case, the orientation of the right side of image capture device 200 is selected from the three-dimensional orientation information detected by GPS 209, and this orientation is calculated as the orientation of the optical axis after bending.

[0073] Next, in S333, the CPU 206 determines the orientation of the optical axis after bending calculated in S332 as the value to be set as the GPSImgDirection value defined in the EXIF ​​standard. The GPSImgDirection value is also a value of the EXIF ​​information included in the image file together with the still image data generated in S327.

[0074] Next, in S334, the CPU 206 generates an image file. This image file includes the still image data generated in the imaging sequence of S327 and the values ​​determined in S328, S331, and S333. The generated image file is written to the SD card 208 via the SD slot 207.

[0075] Next, in S335, the CPU 206 determines whether or not a user instruction to end automatic shooting has been detected. If it is determined that a user instruction to end automatic shooting has not been detected, the process returns to S324. If it is determined that a user instruction to end automatic shooting has been detected, the process ends.

[0076] According to the above-described embodiment, the installation direction of the image capture device 200 relative to the light bending device 100 is detected, and an image file is generated that includes an Orientation value corresponding to the detected installation direction and still image data generated in the imaging sequence of S327. That is, the Orientation value is determined taking into consideration not only the posture of the image capture device 200 but also the light bending device 100 provided outside the image capture device 200. This makes it possible to include an appropriate Orientation value that displays an image in the same orientation as in the real world in an image file generated by imaging using the light bending device 100.

[0077] Furthermore, in the above-described embodiment, the installation direction of the image capturing device 200 relative to the light bending device 100 can be changed around the optical axis of the image capturing device 200. As a result, even if the installation direction of the image capturing device 200 is changed around the optical axis of the image capturing device 200, an appropriate Orientation value that displays an image in the same orientation as in the real world can be included in an image file generated by capturing an image using the light bending device 100.

[0078] Furthermore, in the above-described embodiment, the installation orientation of the imaging device 200 relative to the light bending device 100 can be changed to a vertical position or a horizontal position around the optical axis of the imaging device 200. As a result, even if the installation orientation of the imaging device 200 relative to the light bending device 100 is changed to either a horizontal position or a vertical position, an appropriate Orientation value can be included in an image file generated by imaging using the light bending device 100.

[0079] In the above-described embodiment, the image file also includes the Camera Elevation Angle value determined in S331. This allows the image file generated by capturing an image using the beam bending device 100 to include an appropriate Camera Elevation Angle value that takes into account the bending of the optical axis by the beam bending device 100.

[0080] Furthermore, in the above-described embodiment, the image file also includes the GPSimgDirection value determined in S333. This allows an appropriate GPSimgDirection value that takes into account the bending of the optical axis by the beam bending device 100 to be included in the image file generated by imaging using the beam bending device 100. Furthermore, by generating an image file that includes such a Camera ElevationAngle value and GPSimgDirection value, it becomes possible to identify the direction from which the image was captured when, for example, a still image based on the image file is displayed in association with a map such as Google Maps.

[0081] Furthermore, in the above-described embodiment, the light bending device 100 includes the installation direction detection means 114 and the communication unit 116 that transmits status information of the installation direction detection means 114 to the image capturing device 200. This allows the image capturing device 200 to easily identify the installation direction of the image capturing device 200 relative to the light bending device 100. As a result, an image file generated by image capturing using the light bending device 100 can include an appropriate Orientation value that displays the image in the same orientation as in the real world.

[0082] In the above-described embodiment, the elevation angle is determined only from the angle of the tilt mirror 113, but this is not limiting. For example, if the image capture device 200 itself has a certain degree of tilt with respect to the XY plane, the elevation angle may be determined by taking into account the tilt of the image capture device 200 detected by the acceleration sensor 210 in addition to the angle of the tilt mirror 113. This makes it possible to include an appropriate Camera Elevation Angle value that takes into account the tilt of the image capture device 200 in an image file generated by capturing an image using the light bending device 100.

[0083] Furthermore, in the above-described embodiment, a configuration has been described in which the value to be set in the GPSmgDirection value is determined using the orientation detected by the GPS 209 included in the image capturing device 200, but the present invention is not limited to this configuration. For example, the value to be set in the GPSmgDirection value may be determined using the orientation detected by the GPS 115 included in the light bending device 100. In this case, at least before S333 is performed, a process is performed in which the image capturing device 200 inquires of the light bending device 100 about and acquires orientation information detected by the GPS 115. Even with such a configuration, it is possible to achieve the same effects as those of the above-described embodiment.

[0084] The present disclosure can also be realized by a process in which a program that realizes one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present disclosure can also be realized by a circuit (e.g., an ASIC) that realizes one or more functions.

[0085] This application claims priority based on Japanese Patent Application No. 2024-144291, filed on August 26, 2024, the entire contents of which are incorporated herein by reference.

[0086] 10 Imaging system 100 Light bending device 113 Tilt mirror 114 Installation direction detection means 115 GPS 200 Imaging device 209 GPS

Claims

1. An imaging system comprising: a light beam bending device that bends external light beams; an imaging device that generates image data based on the light beams bent by the light beam bending device; and means for detecting the installation direction of the imaging device relative to the light beam bending device, wherein the imaging device generates an image file containing the image data and a value corresponding to the detected installation direction of the imaging device, the value being used to control the display of the image data.

2. The imaging system described in claim 1, characterized in that the light bending device has a rotatable bending optical surface that bends the external light beam, the bending optical surface is positioned at a position where it intersects with the optical axis of the imaging device, and the installation direction of the imaging device can be changed around the optical axis of the imaging device.

3. The imaging system according to claim 2, wherein the installation direction of said imaging device can be changed to a vertical position or a horizontal position around the optical axis of said imaging device.

4. The imaging system described in claim 2, characterized in that the light bending device further comprises means for transmitting information indicating the rotational position of a motor for rotating the bending optical surface to the imaging device, and the imaging device generates elevation angle information of the optical axis bent by the bending optical surface based on the information indicating the rotational position, and generates an image file including a value corresponding to the detected installation direction of the imaging device and used for display control of the image data, the image data, and the elevation angle information.

5. The imaging system according to claim 4, characterized in that the imaging device further comprises means for detecting the tilt of the imaging device, and the imaging device generates elevation angle information of the optical axis bent by the bending optical surface based on the information indicating the rotational position and the detected tilt of the imaging device.

6. The imaging system according to claim 1, wherein said beam bending device comprises said detecting means and means for transmitting said detected installation direction of said imaging device to said imaging device.

7. The imaging system according to claim 1, further comprising a means for detecting an orientation, wherein the imaging device calculates the orientation of the optical axis after bending by the light bending device based on the detected orientation and the detected installation direction of the imaging device, and the imaging device generates an image file including a value corresponding to the installation direction of the imaging device received from the imaging device and used for display control of the image data, the image data, and the calculated orientation.

8. The imaging system according to claim 7, wherein said beam bending device includes means for detecting said orientation.

9. The imaging system according to claim 7, wherein said imaging device comprises means for detecting said orientation.

10. An imaging device installed in a beam bending device, comprising: means for generating image data based on a beam bent by the beam bending device; means for acquiring the installation direction of the imaging device relative to the beam bending device; and means for generating an image file including the image data and a value corresponding to the acquired installation direction of the imaging device, the value being used for display control of the image data.

11. A method for controlling an imaging device installed in a beam bending device, comprising the steps of: generating image data based on light rays bent by the beam bending device; acquiring the installation direction of the imaging device relative to the beam bending device; and generating an image file containing the image data and a value corresponding to the acquired installation direction of the imaging device, the value being used to control the display of the image data.

12. A program that causes a computer to execute a control method for an imaging device installed in a beam bending device, the control method for the imaging device comprising the steps of: generating image data based on light rays bent by the beam bending device; acquiring the installation direction of the imaging device relative to the beam bending device; and generating an image file that includes the image data and a value that corresponds to the acquired installation direction of the imaging device and is used to control the display of the image data.

Images