Image capture system, image capture device, image capture device control method, and program

Abstract

To provide an imaging system capable of including an appropriate Orientation value for displaying an image in the same orientation as in the real world in an image file generated by imaging using a light bending device that bends external light rays and sends them to an imaging device. [Solution] The imaging system 10 is composed of a beam bending device 100 that bends external light rays, and an imaging device 200 that generates image data based on the light rays bent by the beam bending device 100. The imaging device 200 generates an image file that includes an Orientation value corresponding to the installation direction of the imaging device 200 relative to the beam bending device 100 and still image data generated in the imaging sequence of S327.

Description

[Technical Field]

[0001] The present invention relates to an imaging system, an imaging device, a control method for an imaging device, and a program. [Background technology]

[0002] Conventionally, in photographing with an imaging device, control is performed to track a desired subject by performing pan and tilt at high speed. In this control, instead of moving the heavy imaging device itself, a technique is known in which a movable mirror built into the imaging device is used to bend the optical axis of the imaging device to improve the speed of pan and tilt operation to track the desired subject (see Patent Document 1).

[0003] An image file containing image data captured by an imaging device includes an Orientation value, which is used to control the display of the image data and indicates the image orientation when the image based on the image data is displayed. By using the Orientation value, the image can be displayed in the same orientation as in the real world. The Orientation value is determined based on the orientation of the imaging device, such as a horizontal position. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2001-28706 Summary of the Invention [Problem to be solved by the invention]

[0005] Meanwhile, development of a system in which a ray bending means, such as a movable mirror, that bends external light rays is provided externally to the imaging device rather than internally is being considered. In such a system, the imaging device photographs the subject through the ray bending means rather than directly, and image data is obtained that resembles a photograph of the actual subject, flipped left and right, based on the external light rays bent by the ray bending means. In order to display an image based on such image data in the same orientation as in the real world, the imaging device must determine an Orientation value taking into account the ray bending means provided externally. In other words, the conventional Orientation value determined based on the posture of the imaging device cannot display an image in the same orientation as in the real world.

[0006] The present invention aims to provide a mechanism that enables an image file generated by photography using a light bending device that bends external light rays and sends them to an imaging device to include an appropriate Orientation value that displays the image in the same orientation as in the real world. [Means for solving the problem]

[0007] In order to achieve the above object, the imaging system of the present invention comprises a ray bending device that bends external light rays, an imaging device that generates image data based on the light rays bent by the ray bending device, and means for detecting the installation direction of the imaging device relative to the ray bending device, and is characterized in that the imaging device generates an image file including the image data and a value corresponding to the detected installation direction of the imaging device, the value being used to control the display of the image data. [Effects of the Invention]

[0008] According to the present invention, an image file generated by photographing using a light bending device that bends external light rays and sends them to an imaging device can include an appropriate Orientation value that displays the image in the same orientation as in the real world. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is an external view showing a physical layout of an imaging system according to an embodiment of the present invention and a state of incident light; [Figure 2] 2 is a block diagram showing a schematic configuration of the light bending device and the imaging device of FIG. 1. FIG. [Figure 3] 2 is a flowchart showing the procedure of an imaging control process executed by the imaging system of FIG. [Figure 4] FIG. 10 is a diagram for explaining Orientation values ​​used in the present embodiment. [Figure 5] 3 is a diagram showing a state of the imaging device when the imaging device is handheld and photographs are taken without using the light bending device of FIG. 2. FIG. [Figure 6] 2 is a diagram showing a state in which an imaging device is installed in the light bending device of FIG. 1. FIG. [Figure 7] 7 is an example of still image data generated by capturing an image in one of the states shown in FIGS. 5 and 6. [Figure 8] 10 is a table summarizing the correspondence between the Orientation value, the display arrangement of pixel information when displayed on the display means of the information processing device, the memory arrangement of pixel information when capturing an image, and the physical arrangement of the image capturing device. [Figure 9] FIG. 4 is a diagram for explaining the determination of the elevation angle of the imaging device in S331 of FIG. [Figure 10] FIG. 4 is a diagram for explaining calculation of the orientation of the optical axis after bending in S332 of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0010] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0011] 1 is an external view showing the physical layout of an imaging system 10 according to an embodiment of the present invention and the state of incident light. Note that in FIG. 1, the internal configuration is shown transparently for ease of understanding.

[0012] In FIG. 1, an imaging system 10 includes a beam bending device 100 and an imaging device 200.

[0013] Inside the beam bending device 100, the imaging device 200 is fixed and installed by installation direction detection means 114 shown in Fig. 2, which will be described later. The imaging device 200 is installed with its lens facing the upper side of the beam bending device 100.

[0014] A wide-angle optical system 101 and an image sensor 102 are provided on the upper part of the light bending device 100. The wide-angle optical system 101 acquires light rays with a wide angle of view indicated by lines 153a and 153b, and the image sensor 102 acquires image data with the wide angle of view.

[0015] Further, a tilt mirror 113 is provided on the upper part of the beam bending device 100. The tilt mirror 113 is disposed at a reference position where the angle between the optical axis of the imaging device 200 and the vertical axis of the tilt mirror 113 is 45 degrees, and can be rotated by a tilt mirror motor 111 shown in FIG. 2 (described later). For example, when the tilt mirror 113 is at the reference position, the angle of view from which the imaging device 200 can acquire light rays is the region indicated by lines 151a and 151c. At this time, among the light rays acquired by the imaging device 200, the light rays that are bent by the tilt mirror 113 and positioned on the optical axis of the imaging device 200 are light rays 151b. Furthermore, when the tilt mirror 113 is rotated to the position indicated by 113' in FIG. 1, the angle of view from which the imaging device 200 can acquire light rays is the region indicated by lines 152a and 152c. At this time, among the light rays obtained by the imaging device 200, the light ray that is bent by the tilt mirror 113 and positioned on the optical axis of the imaging device 200 becomes the light ray 152b.

[0016] A pan axis rotation motor 108 is provided on the bottom surface of the beam bending device 100 for rotating the beam bending device 100 in which the imaging device 200 is installed.

[0017] FIG. 2 is a block diagram showing a schematic configuration of the beam bending device 100 and the imaging device 200 shown in FIG.

[0018] First, the configuration of the beam bending device 100 will be described. In addition to the wide-angle optical system 101, image sensor 102, pan axis rotation motor 108, and tilt mirror 113 described above, the beam bending device 100 also includes image processing hardware 103, memory 104, recognition hardware 105, CPU 106, and driver circuit 107, as shown in Fig. 2. The beam bending device 100 further includes a rotational position sensor 109, a driver circuit 110, a tilt mirror motor 111, a rotational position sensor 112, installation direction detection means 114, a GPS 115, and a communication unit 116.

[0019] As described above, the wide-angle optical system 101 captures light rays with a wide angle of view. The image sensor 102 converts the light rays captured by the wide-angle optical system 101 into electrical signals. The image processing HW 103 performs image processing, such as conversion to a necessary and sufficient image size, noise reduction, and other image processing, on image data based on the electrical signals converted by the image sensor 102, and stores the processed image data in memory 104. The recognition HW 105 performs object recognition processing based on the image data stored in memory 104. In the object recognition processing, an object area is detected from an image based on this image data, and coordinate information indicating the position of this object area within the image and movement information of this object area are detected. The information detected by the recognition HW 105 is output to the CPU 106.

[0020] CPU 106 includes a memory (not shown) and executes programs stored in the memory or the like to perform various controls. For example, CPU 106 selects a main subject area from among multiple subject areas detected by recognition HW 105 and acquires the position of the main subject area within the current angle of view. CPU 106 controls pan axis rotation motor 108 and tilt mirror motor 111 so that the main subject area is positioned at the center of the angle of view. The control of these two motors will now be described in detail.

[0021] The control of the pan axis motor 108 will now be described. The rotational position of the pan axis motor 108 is detected by a rotational position sensor 109, and the CPU 106 acquires this rotational position. Based on this rotational position and the lateral position of the main subject area within the angle of view, the CPU 106 calculates a target rotational position to be reached by the pan axis motor 108, and calculates a voltage control amount corresponding to this target rotational position. The CPU 106 controls the drive voltage of the pan axis motor 108 based on the calculated voltage control amount. Here, feedback processing such as PID control based on the difference between the current rotational position and the target rotational position is performed, for example. Note that in FIG. 2, the pan axis motor 108 and the wide-angle optical system 101 are connected by a line, which indicates that the direction of the wide-angle optical system 101 changes in response to the rotation of the pan axis motor 108.

[0022] Next, the control of tilt mirror motor 111 will be described. The rotational position of tilt mirror motor 111 is detected by rotational position sensor 112, and CPU 106 acquires this rotational position. Based on this rotational position and the vertical position of the main subject area within the angle of view, CPU 106 calculates a target rotational position to be reached by tilt mirror motor 111 and calculates a voltage control amount corresponding to this target rotational position. CPU 106 controls the drive voltage of tilt mirror motor 111 based on the calculated voltage control amount. Here, as with pan axis rotation motor 108, feedback processing such as PID control based on the difference between the current rotational position and the target rotational position is performed. Through this series of processing, tilt mirror 113 is controlled to the desired tilt angle. Note that in FIG. 2, pan axis rotation motor 108 and tilt mirror motor 111 are connected by a line. This indicates that tilt mirror motor 111, rotational position sensor 112 connected to tilt mirror motor 111, and tilt mirror 113 also rotate in response to the rotation of pan axis rotation motor 108.

[0023] Tilt mirror 113 serves to send light rays for generating image data to image capture device 200. Specifically, tilt mirror 113 refracts external light rays and sends the light rays to telephoto optical system 201 of image capture device 200. The reflective surface of tilt mirror 113 corresponds to the "refracting optical surface" in the claims of the present invention. The reflective surface of tilt mirror 113 is located at a position where it intersects with the optical axis of image capture device 200.

[0024] The installation direction detection means 114 detects the installation direction of the imaging device 200 installed in the beam bending device 100. In this embodiment, the imaging device 200 installed in the beam bending device 100 has a degree of freedom of installation in 90-degree increments around the optical axis, as will be described in detail later. The installation direction detection means 114 is composed of a mechanical detection lever provided at an appropriate location on the beam bending device 100 where the imaging device 200 is installed, and a photointerrupter that detects the tilting state of the lever. The GPS 115 acquires the geographical position and orientation from a GPS system. The communication unit 116 communicates various information with the imaging device 200. The GPS 115 corresponds to the "orientation detection means" in the claims of the present invention.

[0025] Next, we will explain the configuration of the imaging device 200. As shown in Figure 2, the imaging device 200 includes a telephoto optical system 201, an image sensor 202, image processing hardware 203, a memory 204, recognition hardware 205, a CPU 206, an SD slot 207, an SD card 208, a GPS 209, an acceleration sensor 210, and a communication unit 211.

[0026] The telephoto optical system 201 acquires light rays with a telephoto angle of view via the tilt mirror 113. The image sensor 202 converts the light rays acquired by the telephoto optical system 201 into an electrical signal. The image processing HW 203 performs image processing such as conversion to a necessary and sufficient image size, noise reduction, and other image processing on image data based on the electrical signal converted by the image sensor 202, and stores the processed image data in the memory 204. The recognition HW 205 performs object recognition processing based on the image data stored in the memory 204. In this object recognition processing, as in the object recognition processing by the recognition HW 105 described above, an object area is detected from an image based on the image data. Coordinate information indicating the position of this object area within the image and movement information of this object area are also detected. The information detected by the recognition HW 205 is output to the CPU 206.

[0027] The CPU 206 includes a memory (not shown) and executes programs stored in the memory to perform various controls. The SD slot 207 stores image files. These image files include, for example, processed image data obtained by applying appropriate processing to image data recorded in the memory 204, and various information determined and calculated according to the flowchart of FIG. 3(b) described below. The GPS 209 acquires the geographical position and orientation from a GPS system. The GPS 209 corresponds to the "orientation detection means" in the claims of the present invention. The acceleration sensor 210 has the function of detecting gravitational acceleration in three axes and can three-dimensionally detect the tilt of the imaging device 200, etc. The communication unit 211 communicates various information with the light bending device 100.

[0028] Fig. 3 is a flowchart showing the procedure of the imaging control process executed by the imaging system 10 of Fig. 1. The flow of this imaging control process will be described below with reference to the flowchart of Fig. 3 and Figs. 4 to 10. This imaging control process is performed by the light bending device 100 and the imaging device 200. Fig. 3(a) is a flowchart showing the procedure of the control process by the light bending device 100. Fig. 3(b) is a flowchart showing the procedure of the control process by the imaging device 200. The imaging control process is realized by exchanging various information by communicating between the light bending device 100 and the imaging device 200.

[0029] First, a description will be given of the control process by the beam bending device 100 in Fig. 3(a). The control process by the beam bending device 100 is realized by the CPU executing a program stored in the memory of the CPU or the like.

[0030] In FIG. 3(a), in S301, the CPU 106 detects a power-on instruction from the user to the beam bending device 100. Note that the power-on instruction in this embodiment is assumed to be a power-on action by the user using a power button, etc. Next, the process proceeds to S302. At this time, the user issues an automatic shooting instruction to the imaging device 200. As a result, the control process by the imaging device 200 in FIG. 3(b) proceeds to S322.

[0031] 3(a), in S302, the CPU 106 acquires state information of the installation direction detection means 114. The processing of S302 will now be described in detail with reference to FIG.

[0032] Fig. 6 is a diagram showing a state in which imaging device 200 is installed in beam bending device 100 of Fig. 1. In Fig. 6, the optical axis of imaging device 200 is defined as the z-axis, the rotation axis of tilt mirror 113 is defined as the x-axis, and the axis perpendicular to the x-axis and z-axis is defined as the y-axis.

[0033] Comparing the two diagrams of FIG. 6(a) and FIG. 6(b), in FIG. 6(a) the finder unit 601 of the imaging device 200 is located in the + direction of the y-axis, and in FIG. 6(b) the finder unit 601 of the imaging device 200 is located in the + direction of the x-axis. Generally, when photographing with the imaging device 200 held by hand, horizontal photography is performed by holding the imaging device 200 horizontally (horizontal position), and vertical photography is performed by rotating the imaging device 200 90 degrees and holding it vertically (vertical position). In contrast, the configuration of FIG. 6(a) corresponds to a case where photographing is performed at a horizontal angle via the tilt mirror 113, and the configuration of FIG. 6(b) corresponds to a case where photographing is performed at a vertical angle via the tilt mirror 113. In this embodiment, the imaging device 200 can also be installed as shown in FIG. 6(c) to photograph at a horizontal angle via the tilt mirror 113. It is also possible to install imaging device 200 as shown in Fig. 6(d) so as to capture images at a vertical angle via tilt mirror 113. In this embodiment, it is possible to identify in which of the configurations shown in Fig. 6(a), 6(b), 6(c), and 6(d) imaging device 200 is installed, based on the status information of installation direction detection means 114. CPU 106 controls communication unit 116 to transmit the acquired status information of installation direction detection means 114 to imaging device 200.

[0034] Next, in S303, the CPU 106 performs object recognition processing by controlling the recognition HW 105. In this object recognition processing, the recognition HW 105 uses image data obtained from light rays captured by the wide-angle optical system 101 via the image sensor 102, the image processing HW 103, and the memory 104 to detect an object region from an image based on this image data.

[0035] Next, in S304, CPU 106 determines a main subject region to be captured from among the one or more detected subject regions. It is assumed that the method for determining the main subject region in S304 is predetermined. For example, there are various methods, such as determining a specific person region or a specific object region as the main subject region, but here, it is assumed that one main subject region is determined using any of these methods. At this time, information such as the position and size of the main subject region within the angle of view is calculated, and the type of the main subject region, such as a person or a dog, is identified. This information is stored in memory, etc. of CPU 106, and transmitted to image capture device 200. This information is used in the process of determining the main subject region in image capture device 200 (see S325, described later).

[0036] Next, in S305, CPU 106 generates a pan rotation position command for controlling pan axis rotation motor 108 and a tilt rotation position command for controlling tilt mirror motor 111. Specifically, based on the calculated position of the main subject area in the image and information from rotation position sensor 109 and rotation position sensor 112, it is calculated to what positions pan axis rotation motor 108 and tilt mirror motor 111 should be rotated to, respectively, so that the main subject area is positioned at the center of the angle of view. CPU 106 generates a pan rotation position command and a tilt rotation position command based on the calculation results.

[0037] Next, in S306, CPU 106 performs motor position feedback processing. In the motor position feedback processing, CPU 106 controls the drive voltage of each motor based on the difference between the current rotational position of each motor and the rotational position corresponding to each rotational position command generated in S305. This controls pan axis rotation motor 108 and tilt mirror motor 111 so that they each move to their desired rotational positions.

[0038] Next, in S307, CPU 106 determines whether or not an inquiry about angular position information of tilt mirror motor 111 has been received from imaging device 200. The process of imaging device 200 inquiring about angular position information of tilt mirror motor 111 corresponds to the process of S329 in FIG. 3(b), which will be described later. If it is determined that an inquiry has been received, the process proceeds to S308, and if it is determined that an inquiry has not been received, the process proceeds to S309.

[0039] In S308, CPU transmits information indicating the rotational position of tilt mirror motor 111 detected by rotational position sensor 112 to imaging device 200 via communication unit as angular position information of tilt mirror motor 111.

[0040] Next, in S309, the CPU 106 determines whether or not it has detected a user's instruction to power off the beam bending device 100. Note that the power off instruction in this embodiment is assumed to be a power off action by the user using a power button, or the like.

[0041] If it is determined that the instruction to power off the beam bending device 100 has not been detected, the process returns to S303. If it is determined that the instruction to power off the beam bending device 100 has been detected, the process proceeds to S310.

[0042] In S310, the CPU 106 performs a termination process for the beam bending device 100. In this termination process, information that is used in the processes being performed in the beam bending device 100 and that will be needed later is saved and stored, and various drive mechanisms are reset to their initial positions. After that, this process ends.

[0043] Next, a control process by the imaging device 200 in Fig. 3(b) will be described. The control process by the imaging device 200 is realized by the CPU 206 executing a program stored in the memory or the like of the CPU 206. Note that the control process by the imaging device 200 in Fig. 3(b) assumes that the imaging device 200 has already been started.

[0044] In FIG. 3B, in S321, the CPU 206 detects an automatic shooting instruction from the user.

[0045] Next, in S322, the CPU 206 determines whether or not it has received the status information of the installation direction detection means 114 that was acquired and transmitted in S302 by the beam bending device 100. The CPU 206 waits until it receives the status information of the installation direction detection means 114, and when it receives the status information of the installation direction detection means 114 (YES in S322), the process proceeds to S323.

[0046] In S323, the CPU 206 stores the received state information of the installation direction detection means 114 in the memory of the CPU 206 or the like.

[0047] Next, in S324, the CPU 206 controls the recognition HW 105 to perform object recognition processing on an image based on the image data recorded in the memory 204. In this embodiment, as described above in the description of FIG. 2, light rays acquired by the telephoto optical system 201 are converted into electrical signals by the image sensor 202. The image processing HW 203 then performs image processing on the image data based on these electrical signals, such as conversion to a necessary and sufficient image size, noise reduction, and other image processing, and the processed image data is recorded in the memory 204. In the object processing in S324, accompanying information such as the type of object area recognized in the image based on the image data recorded in the memory 204 and coordinates indicating its position in the image is calculated.

[0048] Next, in S325, CPU 206 controls recognition HW 105 to determine a main subject region from among one or more subject regions detected in S324. In S325, the main subject region is determined based on information transmitted from beam bending device 100 in S304, for example, information indicating the type of main subject region determined in S304. In this manner, the same subject region is determined as the main subject region in the processing of S304 by beam bending device 100 and the processing of S325 by imaging device 200. At this time, accompanying information such as the position and size of the main subject region within the angle of view is also calculated, and this accompanying information is stored in memory or the like of CPU 206.

[0049] Next, in S326, CPU 206 determines whether the accompanying information of the main subject region determined in S325 satisfies the shooting conditions. This shooting condition is, for example, a condition such as whether the main subject region is captured at a certain size or greater within a certain range within the angle of view. If it is determined that the accompanying information of the main subject region determined in S325 does not satisfy the shooting conditions, the process returns to S324. Thus, in this embodiment, the still image shooting sequence is not performed until the accompanying information of the main subject region satisfies the shooting conditions.

[0050] If it is determined in S325 that the accompanying information of the main subject region determined satisfies the photographing conditions, the process proceeds to S327. In S327, CPU 206 executes a still image capturing sequence. In this photographing sequence, the series of processes from the time when image data obtained based on light rays acquired by telephoto optical system 201 to the time when it is recorded in memory 204 is the same as the processes described above. However, this photographing sequence differs from the processes described above in that it also acquires high-resolution still image data that is intended to be saved in SD slot 207. As described above, in this embodiment, when the user issues an automatic photographing instruction, the still image capturing sequence is executed in accordance with the fact that the accompanying information of the main subject region satisfies the photographing conditions, even without issuing a photographing instruction such as pressing the shutter button.

[0051] Next, in S328, CPU 206 determines a value to be set as the Orientation value based on the status information of installation direction detection means 114. The Orientation value is a value of EXIF ​​information included in the image file together with the still image data generated in S327. The Orientation value is a value used for display control of this still image data, and more specifically, a value representing the image orientation when a still image based on this still image data is displayed. The determination method of S328 will be described in detail using FIGS. 4 to 8.

[0052] FIG. 4 is a diagram illustrating the Orientation values ​​used in this embodiment. Here, an example is described in which an image file is stored in an information processing device, such as a PC (not shown), and a still image based on the still image data contained in the image file is displayed on the display means of the information processing device. In this case, FIG. 4 shows how the images are aligned and displayed based on the Orientation value of the EXIF ​​information contained in the image file. The memory of the information processing device holds a group of pixel information for R rows by C columns for displaying a still image based on the still image data, and the storage coordinates of each pixel in memory are represented by (column, row).

[0053] For example, Fig. 4(a) shows a display layout when the Orientation value is "1." In this case, the display means of the information processing device displays a still image in an orientation such that, in the pixel information group arranged in the memory of the information processing device, the pixel information at stored coordinates (0, 0) is arranged in the upper left, the pixel information at stored coordinates (C, 0) is arranged in the upper right, the pixel information at stored coordinates (0, R) is arranged in the lower left, and the pixel information at stored coordinates (C, R) is arranged in the lower right.

[0054] Fig. 4(b) shows the display arrangement when the Orientation value is "2", which is rotated 90 degrees from Fig. 4(a). In this case, the display means of the information processing device displays a still image in an orientation such that, in the pixel information group arranged in the memory of the information processing device, the pixel information at stored coordinates (C, 0) is arranged in the upper left, the pixel information at stored coordinates (0, 0) is arranged in the upper right, the pixel information at stored coordinates (C, R) is arranged in the lower left, and the pixel information at stored coordinates (0, R) is arranged in the lower right. Figs. 4(c) to 4(h) show that the Orientation value is changed one by one, and still images are displayed with pixel arrangements corresponding to each change.

[0055] 5A and 5B are diagrams illustrating the state of the imaging device 200 when the imaging device 200 is handheld and photographed without using the beam bending device 100 of FIG. 2. FIG. 5A is a diagram illustrating the state when the imaging device 200 is positioned horizontally to photograph a subject. FIG. 5B is a diagram illustrating the state when the imaging device 200 is rotated 90 degrees counterclockwise around its optical axis from the state of FIG. 5A to photograph a subject in a vertical position to photograph a subject. FIG. 5C is a diagram illustrating the state when the imaging device 200 is rotated 90 degrees counterclockwise around its optical axis from the state of FIG. 5B. FIG. 5D is a diagram illustrating the state when the imaging device 200 is rotated 90 degrees counterclockwise around its optical axis from the state of FIG. 5C.

[0056] FIG. 6 is a diagram illustrating a state in which the imaging device 200 is installed in the beam bending device 100 of FIG. 2. FIG. 6(a) is a diagram illustrating a state in which the imaging device 200 is installed in the beam bending device 100 so as to take an image in a horizontal position. FIG. 6(b) is a diagram illustrating a state in which the imaging device 200 is installed in the beam bending device 100 so as to take an image in a vertical position. FIG. 6(c) is a diagram illustrating a state in which the imaging device 200 is installed after being rotated 90 degrees counterclockwise around its optical axis from the state of FIG. 6(b). FIG. 6(d) is a diagram illustrating a state in which the imaging device 200 is installed after being rotated 90 degrees counterclockwise around its optical axis from the state of FIG. 6(c). Note that in this embodiment, as described above, it is possible to identify in which of the configurations of FIG. 6(a), FIG. 6(b), FIG. 6(c), and FIG. 6(d) the imaging device 200 is installed, based on the state information of the installation direction detection means 114.

[0057] Fig. 7 is an example of still image data generated by capturing an image in either of the states shown in Fig. 5 or Fig. 6. Note that Fig. 7 shows still image data obtained by capturing an image of a skier skiing from the upper left to the lower right. Note that Fig. 7 also shows the storage coordinates of each pixel of the upper left, upper right, lower left, and lower right of the still image data in the memory of the information processing device.

[0058] FIG. 7(a) shows still image data obtained by capturing an image in the state shown in FIG. 5(a), i.e., horizontally capturing an image using a handheld imaging device 200 without using the beam bending device 100. FIG. 7(b) shows still image data obtained by capturing an image in the state shown in FIG. 6(a), i.e., horizontally capturing an image using the imaging device 200 installed on the beam bending device 100. FIG. 7(c) shows still image data obtained by capturing an image in the state shown in FIG. 5(c). FIG. 7(d) shows still image data obtained by capturing an image in the state shown in FIG. 6(c). FIG. 7(e) shows still image data obtained by capturing an image in the state shown in FIG. 6(d). FIG. 7(f) shows still image data obtained by capturing an image in the state shown in FIG. 5(d). FIG. 7(g) shows still image data obtained by capturing an image in the state shown in FIG. 6(b), i.e., vertically capturing an image using the imaging device 200 installed on the beam bending device 100. FIG. 7(h) shows still image data obtained by capturing an image in the state shown in FIG. 5(b), that is, by capturing an image vertically using the handheld imaging device 200 without using the beam bending device 100.

[0059] When a still image based on still image data obtained by handheld shooting with the imaging device 200 is displayed on the display means of the information processing device, if the Orientation value is determined based on the posture (horizontal position, vertical position) of the imaging device 200, the still image will be displayed in the same orientation as in the real world. As an example, a case will be described in which a still image based on the still image data of FIG. 7(a) is displayed on the display means of the information processing device. As described above, the still image data of FIG. 7(a) is still image data obtained by shooting in the state of FIG. 5(a), that is, by horizontal shooting with the handheld imaging device 200 without using the light bending device 100. When a still image is displayed based on such still image data, if the display layout is set to the Orientation value of "1" shown in FIG. 4(a), a still image of a skier gliding from the upper left to the lower right, as in the real world, will be displayed on the display means of the information processing device.

[0060] On the other hand, when a still image based on still image data obtained by imaging using the beam bending device 100 is displayed on the display means of an information processing device, if the Orientation value is determined based on the attitude of the imaging device 200, the still image will not be displayed in the same orientation as in the real world. As an example, a case will be described in which a still image based on the still image data of FIG. 7(b) is displayed on the display means of an information processing device. As described above, the still image data of FIG. 7(b) is still image data obtained by imaging in the state of FIG. 6(a), that is, by swift imaging using the imaging device 200 installed on the beam bending device 100. The still image data of FIG. 7(b) is still image data obtained by bending light rays from a subject with the tilt mirror 113 and causing them to enter the telephoto optical system 201 of the imaging device 200. At this time, since the light rays are reflected once by the tilt mirror 113, the light entering the telephoto optical system 201 is reversed left and right. For this reason, the still image data of Fig. 7(b) obtained by imaging using the beam bending device 100 is still image data that is obtained by left-right inverting the still image data of Fig. 7(a) obtained by the same side-by-side imaging and imaging using the handheld imaging device 200. When displaying a still image based on such still image data of Fig. 7(b), if the display layout is set to have an Orientation value of "1" as shown in Fig. 4(a), a still image that is left-right inverted from the real world will be displayed on the display means of the information processing device.

[0061] Therefore, in this embodiment, the value to be set in the Orientation value is determined based on whether the installation orientation of image capture device 200 with respect to tilt mirror 113 is one of FIGS. 6(a), 6(b), 6(c), and 6(d). As described above, whether the installation orientation of image capture device 200 with respect to tilt mirror 113 is one of FIGS. 6(a), 6(b), 6(c), and 6(d) can be identified based on the status information of installation orientation detection means 114. For example, if the installation orientation of image capture device 200 with respect to tilt mirror 113 is one of FIG. 6(a), the value to be set in the Orientation value is determined to be "2." As a result, when a still image is displayed based on the still image data of FIG. 7(b), the display layout is set to the Orientation value of "2" shown in FIG. 4(b), and a still image of a skier gliding from the upper left to the lower right, just like in the real world, is displayed on the display means of the information processing device. As described above, in this embodiment, an appropriate Orientation value is determined based on the installation direction of imaging device 200 relative to tilt mirror 113 so that a still image is displayed in the same orientation as in the real world for image data obtained by imaging using beam bending device 100. (See, for example, FIG. 8.) FIG. 8 is a table summarizing the correspondence between Orientation values, the display arrangement of pixel information when displayed on the display means of the information processing device shown in FIG. 4, the memory arrangement of pixel information when imaging shown in FIG. 7, and the physical arrangement of imaging device 200 shown in FIGS. 5 and 6.

[0062] 3, in S329, the CPU 206 controls the communication unit 211 to inquire of the beam bending device 100 about the angular position information of the tilt mirror motor 111. Upon receiving this inquiry, the beam bending device 100 transmits information indicating the rotational position of the tilt mirror motor 111 detected by the rotational position sensor 112 to the imaging device 200 as angular position information of the tilt mirror motor 111 in S308 described above.

[0063] In S330, CPU 206 determines whether or not it has received the angular position information of tilt mirror motor 111 inquired about in S329. CPU 206 waits until it receives the angular position information of tilt mirror motor 111, and when it receives the angular position information of tilt mirror motor 111 (YES in S330), the process proceeds to S331.

[0064] In S331, CPU 206 determines a value to be set for the elevation angle (Camera Elevation Angle value) of image capture device 200 defined in the EXIF ​​standard, based on angular position information of tilt mirror motor 111. Note that the Camera Elevation Angle value is also a value of EXIF ​​information included in the image file together with the still image data generated in S327. The processing of S331 will be described in detail with reference to FIG. 9.

[0065] 9(a) is a diagram showing a state in which tilt mirror 113 is placed at a reference position. In FIG. 9(a), the horizontal direction is the X axis, and the vertical direction is the Y axis, and tilt mirror 113 is shown tilted at 45 degrees with respect to the X axis and Y axis. At this time, optical axis 901 of imaging device 200 is reflected by tilt mirror 113. The optical axis reflected by tilt mirror 113 is referred to as reflected optical axis 903.

[0066] In FIG. 9(a), optical axis 901 and reflected optical axis 903 each form an angle of 45 degrees with respect to normal 902 of tilt mirror 113. The state of tilt mirror 113 in FIG. 9(a) is defined as a state in which the tilt mirror tilt angle is 0 degrees. When tilt mirror motor 111 is rotated in this state, tilt mirror 113 tilts and the direction of reflected optical axis 903 changes. Here, the clockwise rotation direction of tilt mirror motor 111 is defined as a positive angle direction. FIG. 9(b) shows a state in which the tilt mirror tilt angle is a positive value of α degrees.

[0067] In FIG. 9( b), optical axis 901 is reflected by tilt mirror 113 to become reflected optical axis 905, which is angled θ upward from reflected optical axis 903. When tilt mirror motor 111 tilts by α, normal 904 of tilt mirror 113 also tilts by α from normal 902 of tilt mirror 113. Due to the constraints of mirror reflection, the angle between normal 904 of tilt mirror 113 and optical axis 901 is the same as the angle between normal 904 of tilt mirror 113 and reflected optical axis 905, and this angle is defined as β. This leads to the relationship β = 45 degrees + α. Furthermore, the angle between optical axis 901 of imaging device 200 and reflected optical axis 905 can be expressed as 2β, which can also be expressed as 2β = 90 degrees + 2α.

[0068] Here, when the angle formed by optical axis 901 and reflected optical axis 905 of imaging device 200 is expressed using θ, it becomes 90 degrees + θ. Therefore, the relationship θ = 2α can be derived. In other words, when tilt mirror 113 is tilted α degrees from the reference position, the tilt of the reflected optical axis changes by an angle of 2α from the horizontal direction. Note that angle α in the above explanation corresponds to "information indicating the rotational position of the motor for rotating the bending optical surface" in the claims of the present invention, and angle θ corresponds to "elevation angle information" in the claims of the present invention.

[0069] That is, in S331, the CPU 206 acquires a value corresponding to α in FIG. 9 from the beam bending device 100 as angular position information of the tilt mirror motor 111, and determines the value obtained by doubling that value as the value to be set for the elevation angle (Camera Elevation Angle).

[0070] Next, in S332, CPU 206 calculates the orientation of the optical axis after bending based on the orientation information detected by GPS 209 included in imaging device 200 and the angular position information of tilt mirror motor 111 received in S330. The processing of S332 will be described in detail with reference to FIG.

[0071] FIG. 10(a) is a diagram illustrating a state in which landscape photography is performed by holding the imaging device 200 in a landscape position (normal position), similar to FIG. 5(a) described above. The GPS 209 is provided inside the imaging device 200 and can determine the north, south, east, and west directions in three dimensions: x, y, and z. For example, in FIG. 10(a), when the z direction is north, the GPS 209 determines that the front side of the imaging device 200 (the side having the telephoto optical system 201) is north. In this case, if the imaging device 200 is rotated around the x axis to change the orientation of the telephoto optical system 201 upward, the GPS 209 will determine that the bottom side of the imaging device 200 (the side opposite the top side having the viewfinder unit 1001) is north.

[0072] 10(b) and 10(c) are diagrams showing a state in which the imaging device 200 is installed in the beam bending device 100. In FIGS. 10(b) and 10(c), the installation direction of the imaging device 200 with respect to the tilt mirror 113 is different. In these two installation states, the installation direction of the imaging device 200 with respect to the tilt mirror 113 can be detected by the installation direction detection means 114. For example, in the case of FIG. 10(b), the optical axis after bending is optical axis 1002, and the orientation of optical axis 1002 coincides with the orientation of the top surface (the surface having the finder unit 1001) of the imaging device 200 among the orientations detectable by the GPS 209. Therefore, in this case, the orientation of the top surface of the imaging device 200 is selected from the three-dimensional orientation information detected by the GPS 209, and this orientation is calculated as the orientation of the optical axis after bending. 10(c), the optical axis after bending is optical axis 1003, and the orientation of optical axis 1003 coincides with the orientation of the right side of image capture device 200 among the orientations detectable by GPS 209. Therefore, in this case, the orientation of the right side of image capture device 200 is selected from the three-dimensional orientation information detected by GPS 209, and this orientation is calculated as the orientation of the optical axis after bending.

[0073] Next, in S333, the CPU 206 determines the orientation of the optical axis after bending calculated in S332 as the value to be set as the GPSImgDirection value defined in the EXIF ​​standard. The GPSImgDirection value is also a value of the EXIF ​​information included in the image file together with the still image data generated in S327.

[0074] Next, in S334, the CPU 206 generates an image file. This image file includes the still image data generated in the imaging sequence of S327 and the values ​​determined in S328, S331, and S333. The generated image file is written to the SD card 208 via the SD slot 207.

[0075] Next, in S335, the CPU 206 determines whether or not a user instruction to end automatic shooting has been detected. If it is determined that a user instruction to end automatic shooting has not been detected, the process returns to S324. If it is determined that a user instruction to end automatic shooting has been detected, the process ends.

[0076] According to the above-described embodiment, the installation direction of the imaging device 200 relative to the beam bending device 100 is detected, and an image file is generated that includes an Orientation value corresponding to the detected installation direction and still image data generated in the imaging sequence of S327. That is, the Orientation value is determined taking into consideration not only the posture of the imaging device 200 but also the beam bending device 100 provided outside the imaging device 200. This makes it possible to include an appropriate Orientation value that displays an image in the same orientation as in the real world in an image file generated by imaging using the beam bending device 100.

[0077] Furthermore, in the above-described embodiment, the installation direction of the imaging device 200 relative to the light bending device 100 can be changed around the optical axis of the imaging device 200. As a result, even if the installation direction of the imaging device 200 is changed around the optical axis of the imaging device 200, an appropriate Orientation value for displaying an image in the same orientation as in the real world can be included in an image file generated by imaging using the light bending device 100.

[0078] Furthermore, in the above-described embodiment, the installation direction of the imaging device 200 relative to the beam bending device 100 can be changed to a vertical position or a horizontal position around the optical axis of the imaging device 200. As a result, even if the installation direction of the imaging device 200 relative to the beam bending device 100 is changed to either a horizontal position or a vertical position, an appropriate Orientation value can be included in the image file generated by imaging using the beam bending device 100.

[0079] In the above-described embodiment, the image file also includes the Camera Elevation Angle value determined in S331. This allows the image file generated by capturing an image using the beam bending device 100 to include an appropriate Camera Elevation Angle value that takes into account the bending of the optical axis by the beam bending device 100.

[0080] Furthermore, in the above-described embodiment, the image file also includes the GPSImgDirection value determined in S333. This allows an image file generated by imaging using the beam bending device 100 to include an appropriate GPSImgDirection value that takes into account the bending of the optical axis by the beam bending device 100. Furthermore, by generating an image file that includes such a Camera ElevationAngle value and GPSImgDirection value, it becomes possible to identify the direction from which the image was captured when, for example, a still image based on the image file is displayed in association with a map such as Google Maps.

[0081] Furthermore, in the above-described embodiment, the beam bending device 100 includes the installation direction detection means 114 and a communication unit 116 that transmits status information of the installation direction detection means 114 to the imaging device 200. This allows the imaging device 200 to easily identify the installation direction of the imaging device 200 relative to the beam bending device 100. As a result, an image file generated by imaging using the beam bending device 100 can include an appropriate Orientation value that displays the image in the same orientation as in the real world.

[0082] In the above-described embodiment, the elevation angle is determined only from the angle of tilt mirror 113, but this is not limiting. For example, if image capture device 200 itself has a certain degree of tilt with respect to the XY plane, the elevation angle may be determined by taking into account the tilt of image capture device 200 detected by acceleration sensor 210 in addition to the angle of tilt mirror 113. This makes it possible to include an appropriate Camera ElevationAngle value that takes into account the tilt of image capture device 200 in an image file generated by capturing an image using beam bending device 100.

[0083] Furthermore, in the above-described embodiment, a configuration has been described in which the value to be set in the GPSImgDirection value is determined using the orientation detected by the GPS 209 included in the image capturing device 200, but the present invention is not limited to this configuration. For example, the value to be set in the GPSImgDirection value may be determined using the orientation detected by the GPS 115 included in the beam bending device 100. In this case, at least before performing step S333, a process is performed in which the image capturing device 200 inquires of the beam bending device 100 about and acquires orientation information detected by the GPS 115. Even with such a configuration, it is possible to achieve the same effects as those of the above-described embodiment.

[0084] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0085] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) An imaging system comprising: a light beam bending device that bends external light beams; an imaging device that generates image data based on the light beams bent by the light beam bending device; and means for detecting the installation direction of the imaging device relative to the light beam bending device, wherein the imaging device generates an image file including the image data and a value corresponding to the detected installation direction of the imaging device, the value being used to control the display of the image data. (Configuration 2) The imaging system described in Configuration 1, characterized in that the light beam bending device has a rotatable bending optical surface that bends the external light beam, the bending optical surface is positioned at a position where it intersects with the optical axis of the imaging device, and the installation direction of the imaging device can be changed around the optical axis of the imaging device. (Configuration 3) The imaging system according to Configuration 2, wherein the installation direction of the imaging device can be changed to a vertical position or a horizontal position around the optical axis of the imaging device. (Configuration 4) The imaging system described in Configuration 2 or 3, characterized in that the light bending device further includes a means for transmitting information indicating the rotational position of a motor for rotating the bending optical surface to the imaging device, and the imaging device generates elevation angle information of the optical axis bent by the bending optical surface based on the information indicating the rotational position, and generates an image file including a value corresponding to the detected installation direction of the imaging device and used for display control of the image data, the image data, and the elevation angle information. (Configuration 5) The imaging system described in Configuration 4, wherein the imaging device further comprises a means for detecting the tilt of the imaging device, and the imaging device generates elevation angle information of the optical axis bent by the bending optical surface based on the information indicating the rotational position and the detected tilt of the imaging device. (Configuration 6) The imaging system described in any one of configurations 1 to 5, characterized in that the light bending device includes the detecting means and a means for transmitting the detected installation direction of the imaging device to the imaging device. (Configuration 7) The imaging system according to any one of configurations 1 to 6, further comprising a means for detecting an orientation, wherein the imaging device calculates the orientation of the optical axis after bending by the light bending device based on the detected orientation and the detected installation direction of the imaging device, and the imaging device generates an image file including a value corresponding to the installation direction of the imaging device received from the imaging device and used for display control of the image data, the image data, and the calculated orientation. (Configuration 8) The imaging system according to configuration 7, wherein the beam bending device includes a means for detecting the orientation. (Configuration 9) The imaging system according to configuration 7, wherein the imaging device includes a means for detecting the orientation. [Explanation of symbols]

[0086] 10. Imaging System 100 Ray bending device 113 Tilt Mirror 114 Installation direction detection means 115 GPS 200 Imaging device 209 GPS

Claims

1. a light bending device for bending an external light beam; an imaging device that generates image data based on the light beam bent by the light beam bending device; a means for detecting an installation direction of the imaging device relative to the light bending device, An imaging system characterized in that the imaging device generates an image file including the image data and a value corresponding to the detected installation direction of the imaging device and used to control the display of the image data.

2. the light bending device includes a rotatable bending optical surface for bending the external light beam; the bending optical surface is disposed at a position where it intersects with an optical axis of the imaging device, 2. The imaging system according to claim 1, wherein the installation direction of the imaging device is changeable around the optical axis of the imaging device.

3. 3. The imaging system according to claim 2, wherein the installation direction of the imaging device can be changed to a vertical position or a horizontal position around the optical axis of the imaging device.

4. the beam bending device further comprises means for transmitting information indicating a rotation position of a motor for rotating the bending optical surface to the imaging device; 3. The imaging system according to claim 2, wherein the imaging device generates elevation angle information of the optical axis bent by the bending optical surface based on the information indicating the rotation position, and generates an image file including a value corresponding to the detected installation direction of the imaging device and used for display control of the image data, the image data, and the elevation angle information.

5. the imaging device further includes means for detecting a tilt of the imaging device; 5. The imaging system according to claim 4, wherein the imaging device generates elevation angle information of the optical axis bent by the bending optical surface based on the information indicating the rotational position and the detected tilt of the imaging device.

6. 2. The imaging system according to claim 1, wherein the light bending device comprises the detecting means and a means for transmitting the detected installation direction of the imaging device to the imaging device.

7. Further comprising means for detecting an orientation, the imaging device calculates an orientation of the optical axis after bending by the light beam bending device based on the detected orientation and the detected installation direction of the imaging device; The imaging system according to claim 1, characterized in that the imaging device generates an image file including a value corresponding to the installation direction of the imaging device received from the imaging device and used for display control of the image data, the image data, and the calculated orientation.

8. 8. The imaging system of claim 7, wherein the beam bending device includes means for detecting the orientation.

9. 8. The imaging system according to claim 7, wherein the imaging device includes a means for detecting the orientation.

10. An imaging device installed on a beam bending device, means for generating image data based on the light beam bent by the light beam bending device; means for acquiring an installation direction of the imaging device relative to the light bending device; An imaging device characterized by comprising: a value corresponding to the installation direction of the acquired imaging device, the value being used for display control of the image data, and a means for generating an image file including the image data.

11. A method for controlling an imaging device installed in a beam bending device, comprising: generating image data based on the light beam bent by the light beam bending device; obtaining an installation direction of the imaging device relative to the beam bending device; A control method for an imaging device, comprising a step of generating an image file including a value corresponding to the installation direction of the acquired imaging device, the value being used to control the display of the image data, and the image data.

12. A program for causing a computer to execute a control method for an imaging device installed in a light bending device, The method for controlling the imaging device includes: generating image data based on the light beam bent by the light beam bending device; obtaining an installation direction of the imaging device relative to the beam bending device; A program comprising a step of generating an image file including the image data and a value corresponding to the installation direction of the acquired imaging device, the value being used for display control of the image data.

Images