Control device, control method, and program

Abstract

To intuitively perform control to a desired imaging direction without user's consideration of the image rotation angle.SOLUTION: A control device includes: direction acquisition means for acquiring a movement direction instructed by a user with respect to an image captured by an imaging unit and displayed; angle acquisition means for acquiring at least one of a rotation angle resulting from rotation around an optical axis of an imaging optical system of the imaging unit or an imaging element of the imaging unit, and a rotation angle resulting from rotation around an image center of the displayed image; and control means for controlling first drive means for moving the imaging direction of the imaging unit in a first direction and second drive means for moving the imaging direction in a second direction different from the first direction. On the basis of the rotation angle, the control means controls movement of the imaging direction of the imaging unit by the first drive means and the second drive means so that the imaging direction of the imaging unit matches the movement direction instructed by the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a control technique for an imaging device. [Background technology]

[0002] Conventionally, an imaging device is mounted on a camera platform equipped with a pan driver and a tilt driver, and the shooting direction of the imaging device is changed by driving the pan driver and the tilt driver. Recently, imaging devices equipped with a rotation driver that can rotate the imaging element around the optical axis have become available. Furthermore, in imaging devices, the aspect ratio of an image captured by the imaging element is determined by the ratio of the number of horizontal pixels to the number of vertical pixels on the imaging surface. In imaging devices equipped with a rotation driver, an imaging element whose imaging surface is horizontally elongated relative to the horizontal direction can be rotated 90 degrees around the optical axis by the rotation driver to obtain a vertically elongated image.

[0003] Furthermore, Patent Document 1 discloses a technology that enables pan and tilt drive of a camera platform, as well as rotation drive that rotates an imaging device or an imaging element around an optical axis. Patent Document 1 allows a user to select between a camera platform operation mode that controls pan and tilt drive, and a rotation operation mode that controls rotation drive. In the rotation operation mode, a user can rotate the imaging device or the imaging element around the optical axis to any rotation angle, and in the camera platform operation mode, a user can operate the camera platform to any pan angle or tilt angle. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-114503 Summary of the Invention [Problem to be solved by the invention]

[0005] In the technology disclosed in the above-mentioned Patent Document 1, pan drive, tilt drive, and rotation drive are controlled separately in order to improve operability. However, when an image is rotated to an arbitrary angle by rotation drive, it can be difficult for a user to intuitively recognize how much to operate the pan / tilt direction of the camera platform in the camera platform operation mode in order to point the imaging device in the desired shooting direction.

[0006] Therefore, an object of the present invention is to enable a user to intuitively operate the camera in a desired shooting direction without being conscious of the angle of rotation of the image. [Means for solving the problem]

[0007] The control device of the present invention comprises a direction acquisition means for acquiring a movement direction instructed by a user with respect to an image captured by an imaging unit and displayed; an angle acquisition means for acquiring at least one of a rotation angle resulting from rotation around the optical axis of the imaging optical system of the imaging unit or the imaging element of the imaging unit, and a rotation angle resulting from rotation around the image center of the displayed image; and a control means for controlling a first driving means for moving the imaging direction of the imaging unit in a first direction and a second driving means for moving the imaging direction of the imaging unit in a second direction different from the first direction, and is characterized in that the control means controls the movement of the imaging direction of the imaging unit by the first driving means and the second driving means based on the rotation angle so that the imaging direction of the imaging unit matches the movement direction instructed by the user. [Effects of the Invention]

[0008] According to the present invention, the user can intuitively operate the camera in a desired shooting direction without being conscious of the angle of rotation of the image. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the functional configuration of an imaging system according to a first embodiment. [Figure 2]FIG. 10 is an explanatory diagram of pan driving on the circumference of a plurality of imaging units. [Figure 3] FIG. 2 is an explanatory diagram of tilt driving of one imaging unit. [Figure 4] FIG. 2 is an explanatory diagram of rotational driving of an imaging element. [Figure 5] FIG. 10 is an explanatory diagram of torsional driving in each imaging unit on the circumference. [Figure 6] FIG. 1 is a diagram illustrating an example of a schematic configuration of an imaging system. [Figure 7] FIG. 1 is a diagram illustrating an example of the hardware configuration of an imaging system according to a first embodiment. [Figure 8] FIG. 2 is a diagram used to explain the relationship between rotation drive and a subject. [Figure 9] 10A and 10B are diagrams showing examples of an image taken in landscape mode and an image taken in portrait mode. [Figure 10] 10A and 10B are diagrams showing changes in an image due to pan driving during rotation. [Figure 11] FIG. 10 is a diagram showing an example of a GUI for a rotation angle θ=0 degrees. [Figure 12] FIG. 10 is a diagram showing an example of a GUI for a rotation angle θ=+90 degrees. [Figure 13] FIG. 10 is a diagram showing an example of a GUI for a rotation angle θ=+45 degrees. [Figure 14] FIG. 10 is an explanatory diagram of a composite vector. [Figure 15] 10 is a flowchart of pan-tilt control according to a rotation angle. [Figure 16] FIG. 10 is a diagram showing an example of a drive amount selection button. [Figure 17] FIG. 10 is a diagram showing an example of a GUI for a rotation angle φ=+90 degrees. [Figure 18] FIG. 10 is a diagram showing an example of a GUI for a rotation angle φ=+45 degrees. [Figure 19] FIG. 10 is a diagram showing an example of a GUI for rotation angles φ=+90 degrees and θ=−90 degrees. [Figure 20] FIG. 10 is a diagram showing an example of a GUI when multiple imaging units are selected. [Figure 21]FIG. 10 is a diagram illustrating an example of the functional configuration of an imaging system according to a fourth embodiment. [Figure 22] FIG. 10 is a diagram illustrating an example of the hardware configuration of an imaging system according to a fourth embodiment. [Figure 23] FIG. 10 is a diagram illustrating an example of the external configuration of an imaging device according to a fourth embodiment. [Figure 24] FIG. 10 is an explanatory diagram of rotation drive when the tilt angle is +90°. [Figure 25] FIG. 10 is a diagram showing an example of a GUI in which the tilt angle is +90 degrees. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention, and not all of the combinations of features described in the present embodiments are necessarily essential to the solution of the present invention, and multiple features may be combined arbitrarily. The configurations of the embodiments may be modified or changed as appropriate depending on the specifications of the device to which the present invention is applied and various conditions (such as usage conditions and usage environment). Furthermore, in each of the following embodiments, the same or similar configurations and processing steps will be designated by the same reference numerals, and redundant explanations will be omitted.

[0011] <<First embodiment>> <Imaging system> The configuration and functions of the imaging system according to the first embodiment will be described below with reference to Fig. 1. Fig. 1 is a diagram showing an example of the functional configuration of the imaging system according to the first embodiment, and shows the functional configuration of a camera 100 together with a client device 200, which is an example of an information processing device. The camera 100 is connected to the client device 200 via a network 170. The camera 100 transmits captured image data to the client device 200, and also receives control signals from the client device 200.

[0012] Camera 100 is configured to have four imaging units, a drive unit provided independently for each imaging unit, image processing unit 151, control unit 152, communication unit 153, and storage unit 154. The four imaging units consist of first imaging unit 110, second imaging unit 120, third imaging unit 130, and fourth imaging unit 140. The drive units corresponding to first to fourth imaging units 110 to 140, respectively, consist of first drive unit 113, second drive unit 123, third drive unit 133, and fourth drive unit 143.

[0013] <First to fourth imaging units> The first imaging unit 110 is an imaging device having a first imaging optical system (lens) 111 and a first imaging element 112. Similarly, the second imaging unit 120 is an imaging device having a second imaging optical system 121 and a second imaging element 122, the third imaging unit 130 is an imaging device having a third imaging optical system 131 and a third imaging element 132, and the fourth imaging unit 140 is an imaging device having a fourth imaging optical system 141 and a fourth imaging element 142. In other words, the camera 100 of this embodiment is a multi-lens camera having four imaging elements.

[0014] The first imaging optical system 111 has a focus lens and a zoom lens that can be driven in the optical axis direction. Similarly, the second imaging optical system 121, the third imaging optical system 131, and the fourth imaging optical system 141 each have a focus lens and a zoom lens.

[0015] The first imaging optical system 111 forms an image of the transmitted light on the first image sensor 112, and the first image sensor 112 converts the formed image light into an electrical signal and outputs it to the image processing unit 151. Similarly, signals obtained by the second imaging optical system 121 and the second image sensor 122, the third imaging optical system 131 and the third image sensor 132, and the fourth imaging optical system 141 and the fourth image sensor 142 are sent to the image processing unit 151. Furthermore, the first imaging element 112 is configured to be rotatable around the optical axis of the imaging optical system 111. Similarly, the second imaging element 122, the third imaging element 132, and the fourth imaging element 142 are each configured to be rotatable around their optical axes.

[0016] <1st to 4th drive units> The first drive unit 113 includes a first pan drive unit 114, a first tilt drive unit 116, a first rotation drive unit 117, a first torsion drive unit 119, a first focus drive unit 118, and a first zoom drive unit 115. The second drive unit 123 includes a second pan drive unit 124, a second tilt drive unit 126, a second rotation drive unit 127, a second torsion drive unit 129, a second focus drive unit 128, and a second zoom drive unit 125. The third drive unit 133 includes a third pan drive unit 134, a third tilt drive unit 136, a third rotation drive unit 137, a third torsion drive unit 139, a third focus drive unit 138, and a third zoom drive unit 135. The fourth drive unit 143 includes a fourth pan drive unit 144, a fourth tilt drive unit 146, a fourth rotation drive unit 147, a fourth torsion drive unit 149, a fourth focus drive unit 148, and a fourth zoom drive unit 145.

[0017] <Control unit> By executing a control program, the control unit 152 comprehensively controls the first imaging unit 110 to the fourth imaging unit 140, the first driving unit 113 to the fourth driving unit 143, the image processing unit 151, the communication unit 153, and the storage unit 154. The control unit 152 is configured by a CPU (Central Processing Unit) and the like.

[0018] The control unit 152 determines the drive direction and drive amount of the first pan drive unit 114 and the first tilt drive unit 116, and controls the shooting direction of the first imaging unit 110, i.e., the direction of the shooting angle of view determined by the first imaging optical system 111 and the first imaging element 112. The control unit 152 also controls the first zoom drive unit 115 to drive the zoom lens of the first imaging optical system 111, thereby controlling the shooting range (i.e., the shooting angle of view) of the first imaging unit 110. The control unit 152 also controls the first focus drive unit 118 to drive the focus lens of the first imaging optical system 111, thereby controlling the focus position of the first imaging unit 110. The control unit 152 also controls the first rotation drive unit 117 to control the rotation angle, which is the angle of rotation of the first imaging element 112 around the optical axis. Similarly, the control unit 152 controls the second imaging unit 120 and the second driving unit 123, thereby controlling the imaging direction (direction of the imaging angle of view), imaging range (imaging angle of view), focus position, and rotation angle of the second imaging unit 110. The control unit 152 also controls the third imaging unit 130 and the third driving unit 133, and the fourth imaging unit 140 and the fourth driving unit 143 in the same manner, thereby controlling the imaging direction, imaging range, focus position, and rotation angle of the third imaging unit 130 and the fourth imaging unit 140. All or part of these functions performed by the control unit 152 may be realized by a control unit 202 of the client device 200, which will be described later.

[0019] <First to fourth pan drive units and first to fourth tilt drive units> The first pan driver 114 is equipped with the first imaging unit 110. Similarly, the second pan driver 124 is equipped with the second imaging unit 120, the third pan driver 134 is equipped with the third imaging unit 130, and the fourth pan driver 144 is equipped with the fourth imaging unit 140. The first to fourth pan drivers 114 to 144 move the imaging directions of the corresponding first to fourth imaging units 110 to 140 in the horizontal direction, which is the first direction. In this embodiment, the first to fourth pan drivers 114 to 144 perform pan driving to move the corresponding first to fourth imaging units 110 to 140 on the same circumference having the same rotation axis.

[0020] 2 is a schematic diagram showing an example of the arrangement and movement of the first to fourth imaging units 110 to 140 of the camera 100 of this embodiment on the same circumference. In FIG. 2, the clockwise direction is the + direction and the counterclockwise direction is the - direction, and the first to fourth imaging units 110 to 140 positioned on the same circumference are shown as viewed from above (the +Z axis side). The first to fourth imaging units 110 to 140 are mounted on corresponding first to fourth pan driving units 114 to 144, respectively, and the first to fourth pan driving units 114 to 144 move the first to fourth imaging units 110 to 140 on the same circumference.

[0021] The first pan driver 114 includes a motor and a gear, and the driving power of the motor is controlled by the control unit 152. Under the control of the motor driving power by the control unit 152, the first pan driver 114 moves the first imaging unit 110 in the pan direction indicated by the arrow 220 on a circumference 102 indicated by a dotted line and having an axis 101 as its axis of rotation. Similarly, the second to fourth pan drivers 124 to 144 each include a motor and a gear, and the driving power of each motor is controlled by the control unit 152. Under the control of the motor driving power by the control unit 152, the second to fourth pan drivers 124 to 144 move the second to fourth imaging units 120 to 140 in the pan direction indicated by the arrow 220 on the circumference 102 having an axis 101 as its axis of rotation.

[0022] The first to fourth pan driving units 114 to 144 can be driven independently, and the control unit 152 can simultaneously control the driving of one or more of the first to fourth pan driving units 114 to 144. The first to fourth pan driving units 114 to 144 are equipped with sensors such as photointerrupters and Hall elements, and the control unit 152 can obtain the pan positions of the first to fourth pan driving units 114 to 144 based on the output values ​​of these sensors.

[0023] Furthermore, the first to fourth pan driving units 114 to 144 move the corresponding first to fourth imaging units 110 to 140 in the direction of arrow 220 on the common circumference 102, but the relative positions of the first to fourth imaging units 110 to 140 are not interchanged. Furthermore, even when the first to fourth pan driving units 114 to 144 move the first to fourth imaging units 110 to 140 on the circumference 102, they do not move beyond the driving end 106. In other words, the first to fourth imaging units 110 to 140 move only on the circumference 102, not including the driving end 106. Note that the driving end 106 may be a physical driving end defined by hardware, or may be a driving end that is not instructed by the user on software.

[0024] 3 is a schematic diagram showing the first imaging unit 110 and the first tilt driving unit 116 that can change the imaging direction (direction of the imaging angle of view) of the first imaging unit 110 in the tilt direction. In FIG. 3, the first imaging unit 110 and the first driving unit 113 are taken as examples for explanation, but the same is true for the second to fourth imaging units 120 to 140 and the second to fourth driving units 123 to 143, and therefore explanations thereof will be omitted as appropriate.

[0025] In Fig. 3, the upward direction is the + direction and the downward direction is the - direction, and first imaging unit 110 is shown as viewed from the side (-X axis side). Note that axis 101 is the rotation axis shown in Fig. 2. First tilt driving unit 116 is configured to be able to rotate first imaging unit 110 in the direction indicated by arrow 330, with axis 103 as the rotation axis. Furthermore, first tilt driving unit 116 is capable of driving in the range of 0 degrees to +90 degrees, with the horizontal direction as the reference, and axis 103 as the rotation axis.

[0026] First tilt driver 116 includes a motor and gears, and the driving power of the motor is controlled by control unit 152. First tilt driver 116 can change the shooting direction (direction of the shooting angle of view) of first imaging unit 110 in the tilt direction under control of the driving power of the motor by control unit 152. Although not shown in the figures, second to fourth tilt drivers 126 to 146 similarly include motors and gears, and the driving power of each motor is controlled by control unit 152. Under control of the driving power of the motor by control unit 152, second to fourth tilt drivers 126 to 146 can rotate second to fourth imaging units 120 to 140 in the tilt direction indicated by arrow 330 around shaft 103 as the rotation axis. That is, the first to fourth tilt driving sections 116 to 164 move the imaging directions of the corresponding first to fourth imaging sections 110 to 140 in the vertical direction, which is the second direction.

[0027] Furthermore, the first to fourth tilt driving units 116 to 146 can be driven independently, and control unit 152 can simultaneously drive and control one or more of the first to fourth tilt driving units 116 to 146. Furthermore, the first to fourth tilt driving units 116 to 146 are equipped with sensors such as photointerrupters and Hall elements, and control unit 152 can obtain the tilt positions of the first to fourth tilt driving units 116 to 146 based on the output values ​​of these sensors.

[0028] Unlike the first to fourth pan driving units 114 to 144 described above, the first to fourth tilt driving units 116 to 146 do not physically interfere with each other, and therefore do not physically interfere with each other in the tilt direction with the first to fourth imaging units 110 to 140. The same is true for the first to fourth zoom driving units 115 to 145, the first to fourth focus driving units 118 to 148, and the first to fourth rotation driving units 117 to 147 described above, and they do not physically interfere with each other.

[0029] FIG. 3 also shows the dome 104 and the fixed unit 105. The dome 104 covers and houses the first to fourth imaging units 110 to 140, the first to fourth pan driving units 114 to 144, and the first to fourth tilt driving units 116 to 146. The dome 104 is made of a material such as transparent plastic or glass, and therefore the first to fourth imaging units 110 to 140 can capture images of the surroundings through the transparent dome 104. The fixed unit 105 is a fixed member that is attached to a ceiling, floor, wall, etc. That is, the dome 104, the shaft 101, the first to fourth imaging units 110 to 140, the first to fourth pan driving units 114 to 144, and the first to fourth tilt driving units 116 to 146 are attached to the ceiling, floor, wall, etc. via the fixed unit 105.

[0030] <Zoom drive unit and focus drive unit> The first zoom driver 115 includes a motor and gears, and drives the zoom lens of the first imaging optical system 111 under motor drive power control from the control unit 152. Similarly, the second to fourth zoom drivers 125 to 145 each include a motor and gears, and drive the zoom lenses of the corresponding second to fourth imaging optical systems 121 to 141 under motor drive power control from the control unit 152. This changes the imaging ranges (image angle of view, zoom magnification) of the first to fourth imaging units 110 to 140, respectively. In this embodiment, the telephoto end direction resulting from driving the zoom lens is defined as the + direction, and the wide-angle end direction is defined as the - direction. The first to fourth imaging optical systems 121 to 141 each include sensors such as photointerrupters and Hall elements, and the control unit 152 can obtain the positions of the zoom lenses of the first to fourth imaging optical systems 111 to 141 based on the output values ​​of these sensors. Furthermore, the first to fourth zoom driving units 115 to 145 can be driven independently, and the control unit 152 can drive and control one or more of the first to fourth zoom driving units 115 to 145 simultaneously.

[0031] The first focus driver 118 includes a motor and gears, and drives the focus lens of the first imaging optical system 111 under motor drive power control from the control unit 152. Similarly, the second to fourth focus drivers 128 to 148 each include a motor and gears, and drive the focus lenses of the corresponding second to fourth imaging optical systems 121 to 141 under motor drive power control from the control unit 152. This changes the focus positions of the first to fourth imaging units 110 to 140. In this embodiment, the farther direction as a result of driving the focus lens is defined as the + direction, and the closer direction is defined as the - direction. The first to fourth imaging optical systems 121 to 141 each include sensors such as photointerrupters and Hall elements, and the control unit 152 can obtain the positions of the focus lenses of the first to fourth imaging optical systems 111 to 141 based on the output values ​​of these sensors. Furthermore, the first to fourth focus drivers 118 to 148 can be driven independently, and the control unit 152 can drive and control one or more of the first to fourth focus drivers 118 to 148 simultaneously.

[0032] <Rotation drive unit> The first rotation driver 117 includes a motor and gears, and drives the first imaging element 112 to rotate about the optical axis under motor drive power control from the control unit 152. Similarly, the second to fourth rotation drivers 127 to 147 each include a motor and gears, and drive the corresponding second to fourth imaging elements 122 to 142 to rotate about the optical axis under motor drive power control from the control unit 152. This changes the rotation angles of the first to fourth imaging elements 112 to 142 about the optical axis. The first to fourth imaging elements 112 to 142 each include sensors such as photointerrupters and Hall elements, and the control unit 152 can obtain the rotation angles of the first to fourth imaging elements 112 to 142 about the optical axis based on the output values ​​of these sensors. Furthermore, the first to fourth rotation driving units 117 to 147 can be driven independently, and the control unit 152 can control the driving of one or more of the first to fourth rotation driving units 117 to 147 simultaneously.

[0033] Here, the aspect ratio of an image captured by an image sensor is determined by the ratio of the number of pixels in the horizontal direction to the number of pixels in the vertical direction. In this embodiment, for example, the rotation angle at which an image captured by the image sensor has a landscape aspect ratio relative to the horizontal direction is defined as a reference angle of 0°, and clockwise rotation around the optical axis is defined as a positive direction, and counterclockwise rotation is defined as a negative direction. Furthermore, the first to fourth rotation drive units 117 to 147 of this embodiment are capable of rotational drive around the optical axis by angles between -180° and +180°. For example, if the first rotation drive unit 117 rotates the first image sensor 112 by +90° around the optical axis from a landscape aspect ratio, the image captured by the first image sensor 112 will have a portrait aspect ratio.

[0034] 4 is a schematic diagram of the first imaging unit 110 as seen from the front side of the first imaging optical system 111, and for ease of understanding, the first imaging optical system 111 and the first imaging element 112 are shown overlapping (in perspective). Also shown in FIG. 4 is the fixed unit 105, but the pan head on which the first imaging unit 110 is mounted is not shown. Axis 107 indicates the rotation axis of the first rotation drive unit 117, and the first rotation drive unit 117 drives and rotates the first imaging element 112 in the direction indicated by arrow 440 around axis 107 as the center of rotation. It is desirable that axis 107 (rotation axis) of the first rotation drive unit 117 be at the center of the optical axis of the first imaging optical system 111. In Figure 4, the first imaging unit 110 and the first driving unit 113 are used as examples for explanation, but the same applies to the second to fourth imaging units 120 to 140 and the second to fourth driving units 123 to 143, so their explanations will be omitted as appropriate.

[0035] Furthermore, although the present embodiment has been described as an example in which the first image sensor 112 is rotated, the first image sensor 110 itself may be rotated about the optical axis of the first imaging optical system 111. Similarly, the second to fourth image sensors 120 to 140 themselves may be rotated about the optical axes of the corresponding second to fourth imaging optical systems 121 to 141. Even in these cases, the captured images output from the first to fourth image sensors 110 to 140 are images rotated about the optical axes, and the same captured images as those obtained when the first to fourth image sensors 112 to 142 are rotated about the optical axes can be obtained.

[0036] <Torsion drive unit> The first torsional driver 119 is a driver that rotationally drives the first imaging unit 110 in the pan direction independently of the first pan driver 114 described above. Similarly, the second to fourth torsional drivers 129 to 149 are drivers that rotationally drive the corresponding second to fourth imaging units 120 to 140 in the pan direction independently of the first to fourth pan drivers 124 to 144 described above. The first to fourth torsional drivers 119 to 149 each include a motor and a gear, and independently drive the corresponding first to fourth imaging units 110 to 140 in the pan direction under motor drive power control from the control unit 152. This changes the shooting direction of each of the first to fourth imaging units 110 to 140 to the pan direction.

[0037] Here, the first to fourth torsional driving units 119 to 149 differ from the first to fourth pan driving units 114 to 144 in their rotation axes. That is, the rotation axes of the first to fourth pan driving units 114 to 144 are common to axis 101 at the center (or near the center) of camera 100 as shown in FIG. 2, and therefore the first to fourth imaging units 110 to 140 move on the same circumference 102 described above. In contrast, the rotation axes of the first to fourth torsional driving units 119 to 149 are independent for each of the first to fourth imaging units 110 to 140. The first to fourth torsional driving units 119 to 149 each include a sensor such as a photointerrupter or a Hall element, and the control unit 152 can obtain the rotation angle (pan position) of each of the first to fourth torsional driving units 119 to 149 based on the output values ​​of these sensors. Furthermore, the first to fourth torsion driving units 119 to 149 can be driven independently, and the control unit 152 can control the driving of one or more of the first to fourth torsion driving units 119 to 149 simultaneously.

[0038] FIG. 5 is a diagram used to explain an example of independent rotational drive in the pan direction for each of the first to fourth imaging units 110 to 140 by the first to fourth torsional drive units 119 to 149. In FIG. 5, the clockwise direction is the positive direction and the counterclockwise direction is the negative direction, and similarly to FIG. 2 described above, the first to fourth imaging units 110 to 140 are shown on the same circumference 102 as viewed from above (the +Z axis side). Note that the tilt angles of the first to fourth imaging units 110 to 140 are assumed to be 0 degrees. Axis 161 indicates the rotation axis of the first torsional drive unit 119 corresponding to the first imaging unit 110, and the first torsional drive unit 119 can change the orientation of the first imaging unit 110 in the direction of arrow 541. Furthermore, first torsional driver 119 can change the imaging direction of first imaging unit 110 in the range of -30 degrees to +30 degrees, with the direction indicated by dotted line 108 passing through axes 101 and 161 being 0 degrees, which is the front direction (reference direction). Similarly, axes 162 to 164 indicate the rotation axes of second to fourth torsional drivers 129 to 149 corresponding to second to fourth imaging units 120 to 140, respectively. Therefore, second to fourth torsional drivers 129 to 149 can change the imaging directions of second to fourth imaging units 120 to 140 in the directions of arrows 542 to 544, respectively. In addition, the second to fourth torsional drive units 129 to 149 can change the orientation of the second to fourth imaging units 120 to 140 within a range of -30 degrees to +30 degrees, with the direction from axis 101 passing through the corresponding axes 162 to 164 being the front direction (reference direction) of 0 degrees.

[0039] <Additional information about the rotation axis> As described above, the first to fourth pan driving units 114-144 have a rotation axis (axis 101) that is common to the first to fourth imaging units 110-140. In contrast, the first to fourth tilt driving units 116-146 have independent rotation axes (axis 103) for the corresponding first to fourth imaging units 110-140, respectively. Similarly, the first to fourth torsion driving units 119-149 have independent rotation axes (axis 161-164) for the corresponding first to fourth imaging units 110-140, respectively. Similarly, the first to fourth rotation driving units 117-147 have independent rotation axes (axis 107) for the corresponding first to fourth imaging units 110-140, respectively.

[0040] <Image processing unit> The image processing unit 151 performs predetermined image processing on the imaging signals input from the first imaging unit 110, the second imaging unit 120, the third imaging unit 130, and the fourth imaging unit 140 to generate captured image data, and outputs each captured image data to the communication unit 153. For example, the image processing unit 151 generates each captured image data by performing development processing (calculation) on the imaging signals of the first to fourth imaging units 110 to 140 in accordance with a predetermined display format.

[0041] The image processing unit 151 also includes an event detection unit 171. The event detection unit 171 detects a predetermined event from the captured images of the first to fourth imaging units 110 to 140. For example, when detecting a moving subject in a captured image as an event, the event detection unit 171 calculates the difference in luminance between frames of the captured image to determine a change in luminance, and determines that a moving subject has been detected if there is a change in luminance. The predetermined event detected by the event detection unit 171 can also be instructed by the user of the client device 200, which will be described later, via an instruction unit 204.

[0042] Furthermore, the image processing unit 151 of this embodiment also generates OSD (On-screen display) display (superimposed display) data used for a GUI (Graphical User Interface) for the images captured by the first to fourth imaging units 110 to 140. In this embodiment, the GUI based on the OSD display is used as a UI (User Interface) that can be operated by a user in the client device 200, which will be described later. In this embodiment, the GUI based on the OSD display is used to enable a user of the client device 200 to operate the first to fourth driving units 113 to 143 and to visually confirm the imaging directions and positional relationships of the first to fourth imaging units 110 to 140. The OSD display data also includes symbols that represent the imaging directions and positional relationships of the first to fourth imaging units 110 to 140.

[0043] Furthermore, for example, when a plurality of captured images by the first to fourth imaging units 110 to 140 are to be displayed on the same screen, the image processing unit 151 generates an image in which the plurality of captured images are arranged vertically and horizontally, or in a single horizontal row or vertically. In addition, the image processing unit 151 can also perform image processing such as inverting and rotating the captured image. All or part of these functions of the image processing unit 151 may be realized by a control unit 202 of the client device 200, which will be described later.

[0044] <Communications Department> The communication unit 153 transfers the image data and OSD display data sent from the image processing unit 151 to the client device 200 via a network 170 such as a wired LAN or a wireless LAN. <Recording Department> The storage unit 154 is configured with a RAM (Random Access Memory) and a ROM (Read Only Memory), and stores various programs including a control program executed by the control unit 152. The storage unit 154 is also used as a temporary storage area for image data.

[0045] <Client device> The client device 200 is configured to include a communication unit 201, a control unit 202, a display unit 203, an instruction unit 204, and a recording unit 205. The client device 200 is, for example, a device such as a personal computer. The communication unit 201 communicates with the camera 100 via the network 170. The client device 200 may also include a configuration for supplying power to the camera 100 via the network 170. The display unit 203 displays, on the display screen of the display device, images transmitted from the camera 100, GUI based on the OSD display described above, and the like. The instruction unit 204 has a UI and receives operation information from a user using a mouse (pointing device), keyboard, or touch panel. The touch panel is provided on the screen of a display device, for example, and the user can give operation instructions through operations on the touch panel. The recording unit 205 is configured by RAM and ROM, and temporarily stores computer programs, and stores programs and the like that the control unit 202 uses to comprehensively control the client device 200.

[0046] The control unit 202 includes the functionality of a CPU and performs overall control of the client device 200 by executing a program recorded in the recording unit 205. In this embodiment, the control unit 202 generates a control signal for controlling the camera 100 in response to an instruction on the GUI or the like input by the user via the instruction unit 204. In this embodiment, the control signal is a signal for controlling the first to fourth image capturing units 110 to 140 and the first to fourth drive units 113 to 143 through control by the control unit 152 of the camera 100. In other words, the user can operate (control in response to the operation) the first to fourth image capturing units 110 to 140 and the first to fourth drive units 113 to 143 of the camera 100 through an operation on the GUI via the instruction unit 204 of the client device 200. As described above, some or all of the functions of the control unit 152 and image processing unit 151 of the camera 100 may be realized by the control unit 202 of the client device 200.

[0047] FIG. 6 is a diagram showing an example of the schematic configuration of an imaging system including the camera 100 and the client device 200 of this embodiment. Camera 100 is connected to client device 200 via network 170. While Fig. 6 shows an example of a desktop computer in which client device 200 and display device 260 are separate components, client device 200 may also be a notebook computer or tablet terminal with an integrated display.

[0048] FIG. 7 is a diagram showing an example of the hardware configuration of the camera 100 and the client device 200. As shown in FIG. The camera 100 includes the first to fourth imaging units 110 to 140, the first to fourth driving units 113 to 143, a network I / F 183, a CPU 180, a RAM 181, and a ROM 182 described above.

[0049] The ROM 182 stores a control program that the CPU 180 uses to comprehensively control the camera 100 . The RAM 181 temporarily stores a control program executed by the CPU 180. The RAM 181 also provides a work area used when the CPU 180 executes control and processing. The RAM 181 also functions as a frame memory and a buffer memory. The CPU 180 is a central processing unit that performs overall control of the camera 100. The CPU 180 also executes a control program loaded from the ROM 182 to the RAM 181, thereby realizing the operations of the respective functional units of the control unit 152, the image processing unit 151, and the communication unit 153 described above.

[0050] The network I / F 183 transmits images and OSD display data obtained by the CPU 180 executing processing related to the functions of the image processing unit 151 to the client device 200 via the network 170. Note that the image data captured by the first to fourth imaging units 110 to 140 may be stored in an internal storage device such as the RAM 181 or a removable storage medium (not shown) such as an SD card.

[0051] The client device 200 is an information processing device having a CPU 280 , a RAM 281 , a ROM 282 , an input I / F 284 , an output I / F 285 , and a network I / F 283 . The RAM 281 provides a work area used when the CPU 280 executes data processing. The RAM 281 also functions as a frame memory and a buffer memory. The ROM 282 stores a program for the CPU 280 to control the client device 200 and the like. The CPU 280 is a central processing unit, and executes a program loaded from the ROM 282 to the RAM 281. As a result, the CPU 280 performs overall control of the client device 200, and controls the first to fourth imaging units 110 to 140 and the first to fourth driving units 113 to 143 via the control unit 152 of the camera 100.

[0052] The input I / F 284 is connected to the instruction unit 204, and is an interface for receiving user instructions input to the client device 200 via the instruction unit 204. The input I / F 284 also receives operation information corresponding to user instructions for the camera 100 input via the instruction unit 204. In this way, the client device 200 can acquire user instructions for the first to fourth imaging units 110 to 140 and the first to fourth driving units 113 to 143 of the camera 100 via the input I / F 284.

[0053] The output I / F 285 is connected to the display unit 203 and is an interface for displaying the image sent from the camera 100 and the GUI by the OSD display on the screen of the display unit 203 . The network I / F 283 is connected to the camera 100 via the network 170. The network I / F 283 is an interface for transmitting control signals generated by the CPU 280 based on operation information for the camera 100 input via the input I / F 284, and for receiving image data, OSD display data, and the like sent from the camera 100.

[0054] <Explanation of how the image and subject tilt as a result of the rotation drive unit> As described above, the camera 100 of this embodiment can rotate the image sensor around the optical axis using the rotation drive unit. The following describes an example in which the first rotation drive unit 117 rotates the first image sensor 112 of the first image sensor 110 around the optical axis. 8, 9, and 10 are diagrams used to explain the tilt of the imaging surface of the first imaging element 112 and the subject when the first rotation drive unit 117 rotates the first imaging element 112 about the optical axis. In this embodiment, the rotation angle about the optical axis by the first rotation drive unit 117 is described as θ. Note that the control unit 152 can obtain the rotation angle θ of the first imaging element 112 about the optical axis based on the output value of a sensor such as a photointerrupter or a Hall element arranged in the first imaging element 112, as described above.

[0055] First, with reference to FIG. 8, the relationship between the image captured by the first image sensor 112 and the subject when the first image sensor 112 is rotated around the optical axis by the first rotation drive unit 117 will be described. FIG. 8(a) shows a state in which the rotation angle θ by the first rotation drive unit 117 is 0 degrees. A rotation angle of 0 degrees means that the imaging surface of the first imaging element 112 is landscape-oriented relative to the horizontal direction, and this state will be referred to as the landscape imaging state hereinafter. FIG. 8(a) shows a state in which an image of a subject 301 is captured by the first imaging element 112 in the landscape imaging state. Note that the image of the subject 301 formed on the first imaging element 112 is inverted vertically and horizontally by the lens of the first imaging optical system 111, as shown in FIG. 8(a). For the sake of explanation, in FIG. 8(a), a mark 302 is added to the position of the upper left pixel on the imaging surface of the first imaging element 112.

[0056] FIG. 8(b) shows a state in which the rotation angle θ by the first rotation drive unit 117 is +90 degrees (a state in which the camera is rotated 90 degrees in the positive direction shown in FIG. 4). When the rotation angle is +90 degrees, the imaging surface of the first imaging element 112 is vertically elongated relative to the horizontal direction, and this state will be referred to as the vertical imaging state hereinafter. FIG. 8(b) shows a state in which a subject 301 is imaged by the first imaging element 112 in the vertical imaging state. In FIG. 8(b) as well, the image of the subject 301 formed on the first imaging element 112 is inverted vertically and horizontally by the lens of the first imaging optical system 111. Also in FIG. 8(b), for the sake of explanation, a mark 302 is added to the position of the upper left pixel on the imaging surface of the first imaging element 112.

[0057] Next, display examples of an image obtained by capturing an image in landscape mode and an image obtained by capturing an image in portrait mode will be described with reference to Fig. 9. For the sake of explanation, in Fig. 9 as well, a mark 302 is added to the position of the upper left pixel in the image captured by the first image sensor 112. Fig. 9(a) shows an example of a display in which an image 401 captured in landscape orientation as shown in Fig. 8(a) and developed is displayed on a screen with a landscape aspect ratio relative to the horizontal direction. As shown in Fig. 8(a), the image of the subject 301 is captured with its image flipped upside down and left to right by the lens of the imaging optical system 111, so in Fig. 9(a), an image (developed image) that has been rotated 180 degrees to make the captured image easier to view is displayed.

[0058] Fig. 9(b) shows an example of a display in which image 402 captured in portrait mode and developed as shown in Fig. 8(b) is displayed on a screen with an aspect ratio that is landscape long relative to the horizontal direction. As shown in Fig. 8(b), the image of subject 301 is captured with the lens of imaging optical system 111 inverted up and down and left and right, and therefore Fig. 9(b) also displays an image (developed image) rotated 180 degrees to make the captured image easier to view.

[0059] Comparing the display examples of Figures 9(a) and 9(b), in the case of image 401 taken horizontally as shown in Figure 9(a), subject 301 is displayed upright on the screen with a horizontal aspect ratio. In contrast, in image 402 taken vertically as shown in Figure 9(b), subject 301 is displayed tilted by 90 degrees on the screen with a horizontal aspect ratio. In other words, depending on the rotation angle θ, subject 301 may be displayed tilted.

[0060] Next, referring to Figure 10, we will explain the change in the appearance of the displayed image when, for example, the first imaging unit 110 is panned by the first pan driving unit 114 in the landscape shooting state shown in Figure 8(a) and the portrait shooting state shown in Figure 8(b). FIG. 10(a) shows an example of a change in the image 410 displayed when the first image capture unit 110 is panned in the + direction of the X axis (to the right on the paper) by the first pan drive unit 114 during image capture in the landscape mode of FIG. 8(a). The subject 301a shown in the image 410 in FIG. 10(a) represents the subject before pan drive, and the subject 303b represents the subject after pan drive. That is, when the first image capture unit 110 is panned in the + direction of the X axis in the landscape mode, the shooting direction of the first image capture unit 110 moves in the + direction of the X axis (to the right on the paper), and the subject 301 moves in the - direction (to the left) relative to the first image capture unit 110. Therefore, on the display screen, the subject 301a before pan drive moves to the position of the subject 303b after pan drive.

[0061] FIG. 10(b) shows an example of a change in the image 420 displayed when the first image capture unit 110 is panned in the + direction of the X axis (downward on the paper) by the first pan drive unit 114 during image capture in the vertical capture state of FIG. 8(b). The subject 301a shown in the image 420 in FIG. 10(b) represents the subject before pan drive, and the subject 303b represents the subject after pan drive. That is, when the first image capture unit 110 is panned in the + direction of the X axis in the vertical capture state, the shooting direction of the first image capture unit 110 moves in the + direction of the X axis (downward on the paper), and the subject 301 moves in the - direction (upward on the paper) relative to the first image capture unit 110. Therefore, on the display screen, the subject 301a before pan drive moves to the position of the subject 303b after pan drive.

[0062] As described above, even when the first imaging unit 110 is panned in the same direction in the landscape and portrait shooting states, if the rotation angle of the first imaging unit 110 is different, the change in the position of the subject that accompanies the pan driving will appear differently on the displayed image. Note that, although the examples in Figures 8 to 10 have been given using the pan driving of the first imaging unit 110 as an example, similarly, if the rotation angle is different in the cases of the other second to fourth imaging units 120 to 140, the change in the position of the subject that accompanies the pan driving will appear differently.

[0063] 10 shows an example of panning the first imaging unit 110 by the first pan driver 114, but similarly, when the first imaging unit 110 is panned by the first torsional driver 119, the movement of the subject position appears different if the rotation angle is different. This also applies when the second to fourth imaging units 120 to 140 are panned by the second to fourth torsional drivers 129 to 149, respectively. Furthermore, in this embodiment, an example has been given in which the camera 100 is equipped with four imaging units, the first to fourth imaging units 110 to 140. However, even if there is only one imaging unit, as described above, the change in the position of the subject due to pan driving will appear differently if the rotation angle is different.

[0064] Therefore, when the rotation angles are different, the user must operate the imaging direction while taking into consideration the rotation angle at that time when instructing any of the first to fourth imaging units 110 to 140 to move to the desired imaging direction. In other words, when the rotation angles are different, it is difficult for the user to intuitively operate to the desired imaging direction.

[0065] Therefore, in this embodiment, even if the rotation angle θ is different, the appearance of the change in the position of the subject on the display screen due to the movement of the shooting direction of the imaging unit is controlled so as to be the same as the user's instruction operation to point in the desired direction. In other words, this embodiment enables the user to perform intuitive operation. Hereinafter, with reference to FIGS. 11, 12, and 13, examples of the GUI and drive control of the first to fourth drive units 113 to 143 when the rotation angle θ is different will be described.

[0066] Fig. 11(b) is a diagram showing an example of GUI 500 displayed on the screen of the display device of client device 200 using OSD display data generated by image processing unit 151 of camera 100 when rotation angle θ is 0 degrees (θ=0). Note that Fig. 11(b) shows GUI 500 displayed on display unit 203 of client device 200, but the OSD display data that is the basis of GUI 500 is data generated by image processing unit 151 of camera 100, as described above. Furthermore, a rotation angle θ of 0 degrees corresponds to the landscape shooting state as shown in Fig. 8(a) described above.

[0067] The GUI 500 includes an image display area 600 and an operation area 601 . An image captured and developed by any one of the first to fourth imaging units 110 to 140 is displayed in the image display area 600. Note that, although an example is given here in which the same image 401 as that shown in FIG. 9(a) captured by the first imaging unit 110 is displayed in the image display area 600, an image captured by any one of the second to fourth imaging units 120 to 140 may also be displayed. Note that, although specific display examples will be described later, the image display area 600 may display two or more images captured by two or more of the first to fourth imaging units 110 to 140.

[0068] The operation area 601 is an area where an operation GUI is displayed to enable the user of the client device 200 to operate the camera 100. In the operation area 601, for example, an image capture unit position display UI 602, an image capture unit operation UI 603, an image capture unit selection UI 604, and an operation button 605 are displayed.

[0069] The imaging unit selection UI 604 is a UI for the user to select an imaging unit to be operated from among the first to fourth driving units 113 to 143. The imaging unit selection UI 604 displays buttons A to D corresponding to the first to fourth imaging units 110 to 140, respectively. For example, button A corresponds to the first imaging unit 110, button B corresponds to the second imaging unit 120, button C corresponds to the third imaging unit 130, and button D corresponds to the fourth imaging unit 140. The user can select a desired button from among buttons A to B by clicking, touching, or the like to select an imaging unit to be the target of the drive operation, or conversely, to remove the imaging unit from the target of the drive operation. Not only one but multiple buttons A to D can be selected. When multiple buttons are selected, the multiple imaging units corresponding to the selected buttons are set as the targets of the drive operation. In addition, a number corresponding to the button selected in the imaging unit selection UI 604 may be superimposed on the image displayed in the image display area 600. 11(a) illustrates an example in which only the first imaging unit 110 is selected, and in this embodiment, the selection of button A corresponding to the first imaging unit 110 in the imaging unit selection UI 604 is indicated by diagonal lines. Also, the number "A" indicating button A is superimposed on the image 401 displayed in the image display area 600.

[0070] The imaging unit position display UI 602 displays symbols simulating the positions of the first to fourth pan drive units 114 to 144 of the camera 100, i.e., the positions of the first to fourth imaging units 110 to 140 on the circumference 102 described in FIG. 2. In the imaging unit position display UI 602, when the positions of the first to fourth imaging units 110 to 140 are changed by the driving of the first to fourth pan drive units 114 to 144, the positions of the symbols simulating each imaging unit are also changed. In FIG. 11(a), symbol 621 is a symbol corresponding to the first imaging unit 110, and similarly, symbol 622 corresponds to the second imaging unit 120, symbol 623 corresponds to the third imaging unit 130, and symbol 624 corresponds to the fourth imaging unit 140. The imaging unit position display UI 602 also displays symbol 625 corresponding to the drive end 106 shown in FIG. 2. Then, in the imaging unit position display UI 602, the display state of the symbol corresponding to the selected imaging unit is changed in conjunction with the selection of the imaging unit in the imaging unit selection UI 604. Fig. 11(a) shows an example in which the first imaging unit 110 is selected in the imaging unit selection UI 604, and therefore in the imaging unit position display UI 602, the symbol 621 corresponding to the first imaging unit 110 is changed to, for example, a solid display state in a specific color.

[0071] The imaging unit operation UI 603 is a GUI that allows the user to instruct the direction of movement of the imaging unit while viewing the image 401 captured by the imaging unit selected in the imaging unit selection UI 604 and displayed in the image display area 600. The imaging unit operation UI 603 provides pan / tilt buttons 701, 702, 703, 704, 705, 706, 707, and 708 that the user can use to instruct the movement of the imaging unit while viewing the image 401 in the image display area 600. Each of the pan / tilt buttons 701 to 708 has a shape that visually indicates a direction, such as a triangle. By selecting one of the pan / tilt buttons 701 to 708, the user can instruct the movement direction indicated by the shape of the button. In the example of FIG. 11(a), button A corresponding to the first imaging unit 110 is selected in the imaging unit selection UI 604. Therefore, the imaging unit operation UI 603 in the case of Figure 11(a) is a GUI that allows the user to instruct the direction of movement when moving the shooting direction of the first imaging unit 110 while viewing the image 401 captured by the first imaging unit 110 and displayed.

[0072] Of the pan tilt buttons 701 to 708, the pan tilt button 701 is a button that is clicked when the user instructs a direction to move the shooting direction to the right of the image 401 displayed in the image display area 600. When the user clicks the pan tilt button 701, the instruction unit 204 of the client device 200 sends the user instruction to the control unit 202, and the control unit 202 transmits a control signal based on the user instruction from the communication unit 201 to the camera 100.

[0073] The communication unit 153 of the camera 100 sends the received control signal to the control unit 152, and the control unit 152 acquires the control signal, that is, the control signal generated by the client device 200 in response to the user's clicking operation on the pan / tilt button 701. In this way, the control unit 152 has a direction acquisition function that acquires the movement direction instructed by the user to move the shooting direction of the imaging unit with respect to the image 401 in the image display area 600. Then, the control unit 152 controls the driving of the first driving unit 113 based on the control signal.

[0074] FIG. 11B illustrates an example of the GUI 500 when the rotation angle θ of the first imaging unit 110 is 0 degrees (θ=0). FIG. 11A illustrates the drive amount of the first driving unit 113 when the pan / tilt button 701 is clicked once when the rotation angle θ of the first imaging unit 110 is 0 degrees. Specifically, when the pan / tilt button 701 is clicked once when the rotation angle θ of the first imaging unit 110 is 0 degrees, the control unit 152 controls the first pan driving unit 114 to pan the shooting direction of the first imaging unit 110 to the right by a predetermined amount α. In this embodiment, for example, α=10 degrees. When the pan / tilt button 701 is clicked when the rotation angle θ of the first imaging unit 110 is 0 degrees, the control unit 152 does not drive the first tilt driving unit 116 (i.e., the tilt direction is driven to 0 degrees).

[0075] Among the pan-tilt buttons 701 to 708, the pan-tilt button 702 is a button that is clicked when the user instructs the camera 100 to move the imaging direction to the left with respect to the image 401 displayed in the image display area 600. When the user clicks the pan-tilt button 702, the client device 200 transmits a control signal based on the user instruction to the camera 100, and the control unit 152 of the camera 100 drives and controls the first driving unit 113 based on the control signal. For example, if the pan-tilt button 702 is clicked once when the rotation angle θ of the first imaging unit 110 is 0 degrees, the control unit 152 controls the first pan driving unit 114 to pan the imaging direction of the first imaging unit 110 to the left by a predetermined amount α (10 degrees). Note that even if the pan-tilt button 702 is clicked when the rotation angle θ of the first imaging unit 110 is 0 degrees, the control unit 152 does not drive the first tilt driving unit 116.

[0076] Among the pan-tilt buttons 701 to 708, the pan-tilt button 703 is a button that is clicked when the user instructs the camera 100 to move the image capturing direction upward relative to the image 401 displayed in the image display area 600. When the user clicks the pan-tilt button 703, the client device 200 transmits a control signal based on the user instruction to the camera 100, and the control unit 152 of the camera 100 drives and controls the first driving unit 113 based on the control signal. For example, if the pan-tilt button 703 is clicked once when the rotation angle θ of the first imaging unit 110 is 0 degrees, the control unit 152 controls the first tilt driving unit 116 to tilt the first imaging unit 110 upward by a predetermined amount α (10 degrees). Note that if the pan-tilt button 703 is clicked when the rotation angle θ of the first imaging unit 110 is 0 degrees, the control unit 152 does not drive the first pan driving unit 114.

[0077] Among the pan-tilt buttons 701 to 708, the pan-tilt button 704 is a button that is clicked when the user instructs the camera 100 to move the image capturing direction downward relative to the image 401 displayed in the image display area 600. When the user clicks the pan-tilt button 704, the client device 200 transmits a control signal based on the user instruction to the camera 100, and the control unit 152 of the camera 100 controls the driving of the first driving unit 113 based on the control signal. For example, if the pan-tilt button 704 is clicked once when the rotation angle θ of the first imaging unit 110 is 0 degrees, the control unit 152 controls the first tilt driving unit 116 to tilt the first imaging unit 110 downward by a predetermined amount α (10 degrees). Even if the pan-tilt button 704 is clicked when the rotation angle θ of the first imaging unit 110 is 0 degrees, the control unit 152 does not drive the first pan driving unit 114.

[0078] Similarly, pan / tilt button 705 is operated to move the image capturing direction in the upper right direction, pan / tilt button 706 is operated to move the image capturing direction in the upper left direction, pan / tilt button 707 is operated to move the image capturing direction in the lower left direction, and pan / tilt button 708 is operated to move the image capturing direction in the lower right direction. When the rotation angle θ of the first image capturing unit 110 is 0 degrees, the control unit 152 controls the first driving unit 113 to move the image capturing direction of the first image capturing unit 110 in the direction corresponding to which of the pan / tilt buttons 705 to 708 is clicked. In this way, when the image capturing direction of the first image capturing unit 110 is moved in an oblique direction such as the upper right direction, upper left direction, lower left direction, or lower right direction, the first pan driving unit 114 and the first tilt driving unit 116 are driven simultaneously (or) sequentially. When driving in these oblique directions, the control unit 152 calculates the driving amount of the resultant vector of the pan direction and the tilt direction to be a predetermined amount α. The method of calculating the resultant vector will be described later.

[0079] 11 shows an example in which the pan / tilt buttons 701-708 are arranged at 45-degree intervals, but the angle at which each pan / tilt button is arranged is not limited to 45 degrees and may be spaced at different angles for each pan / tilt button. The above description also gives an example of clicking a pan / tilt button, but if the pan / tilt button is continuously pressed, the control unit 152 may continue to drive the first pan driver 114 or the first tilt driver 116 in that direction within a drivable range. While FIG. 11 shows an example in which the first imaging unit 110 and the first driver 113 are used, if one of the second to fourth imaging units 120-140 is selected in the imaging unit selection UI 604, the control unit 152 similarly controls the selected imaging unit and driver.

[0080] The joystick button 709 is a button that is dragged by the user to specify a direction in which the image 401 captured by the imaging unit selected in the imaging unit selection UI 604 and displayed in the image display area 600 is moved in any direction. The joystick button 709 is displayed within a symbol 712 that resembles a controller equipped with a joystick. When the rotation angle θ is 0 degrees and the joystick button 709 is dragged to the right (the direction of the pan-tilt button 701), the control unit 152 of the camera 100 performs drive control similar to that performed when the pan-tilt button 701 described above is operated. When the joystick button 709 is operated to the left, up, down, right-up, left-up, left-down, or right-down, the control unit 152 performs drive control similar to that performed when the pan-tilt buttons 702 to 708 described above are operated. Note that the joystick button 709 can also be controlled to drive in directions other than the eight directions indicated by the pan-tilt buttons 701 to 708. Furthermore, when the joystick button 709 is not operated, the joystick button 709 is located at the center of the symbol 712, and at this time, the control unit 152 of the camera 100 does not perform drive control.

[0081] Furthermore, the control unit 202 of the client device 200 may send to the camera 100 a control signal according to the amount of operation of the joystick button 709 from the center of the symbol 712. In this case, the control unit 152 of the camera 100 may change the drive speed of the drive units (pan drive unit and tilt drive unit) according to the amount of operation of the joystick button 709 from the center of the symbol 712. Furthermore, the position of the joystick button 709 may be changed within the symbol 712 in conjunction with the click operation on the pan / tilt buttons 701 to 708.

[0082] The rotation button 710 is a button that is clicked by the user when the user wants to rotate the rotation drive unit corresponding to the imaging unit selected in the imaging unit selection UI 604 in the + direction. For example, when the rotation button 710 is clicked once, the rotation drive unit changes the rotation angle by +15 degrees. On the other hand, the rotation button 711 is a button that rotates the rotation drive unit in the - direction, and when clicked once, the rotation drive unit changes the rotation angle by -15 degrees.

[0083] The operation buttons 605 are configured by a menu button for calling up other setting screens, a recording button for starting recording, a display change button, etc. The display change buttons include a display rotation button according to a second embodiment described later, and a multiple image display button according to a third embodiment described later.

[0084] FIG. 12(b) shows an example of a GUI 500 displayed on the display device of the client device 200 based on OSD display data generated by the image processing unit 151 of the camera 100 when the rotation angle θ is +90 degrees (θ=+90). Similar to FIG. 11(b), FIG. 12(b) shows the GUI 500 displayed on the display device of the client device 200, but the OSD display data on which the GUI 500 is based is generated by the image processing unit 151 of the camera 100. A rotation angle θ of +90 degrees corresponds to the landscape shooting state shown in FIG. 8(b) above. Note that FIG. 12(b) also illustrates an example in which the first image capture unit 110 is selected by selecting button A in the image capture unit selection UI 604. FIG. 12(a) also shows the state in which the axis of the first pan drive unit 114 and the axis of the first tilt drive unit 116 are rotated by +90 degrees in response to rotation drive by the first rotation drive unit 117.

[0085] Here, when the rotation angle θ is +90 degrees, for example, driving the first pan driver 114 of the first imaging unit 110 in the right (+) direction will move the imaging direction of the first imaging unit 110 in the + direction of the X axis (downward on the paper), as described with reference to FIG. 10(b). In this case, the image in the image display area 600 will be displayed as if the subject 301 is moving upward. Therefore, if the pan / tilt buttons in the imaging unit operation UI 603 are used to indicate the drive directions of the pan driver and tilt driver, the movement direction indicated by the pan / tilt button will differ from the movement direction of the image in the image display area 600 depending on the rotation angle. In other words, assuming that the user will operate the device while viewing the image displayed in the image display area 600, the user will have to select the pan / tilt button while taking into account the current rotation angle and the drive directions of the pan driver and tilt driver. This makes it difficult for the user to intuitively operate the device while viewing the image displayed in the image display area 600.

[0086] For this reason, in this embodiment, the imaging unit operation UI 603 of the GUI 500 is a GUI that allows the user to instruct the movement direction when moving the shooting direction of the imaging unit with respect to the image displayed in the image display area 600. Then, the control unit 152 of the camera 100 controls the driving of the pan driving unit and the tilt driving unit based on the rotation angle so that the shooting direction of the imaging unit is moved in the same direction as the movement direction instructed by the user while viewing the image on the imaging unit operation UI 603.

[0087] To achieve such drive control of the pan driver and tilt driver, the control unit 152 first acquires the rotation angle θ of the rotation driver corresponding to the image capture unit selected in the image capture unit selection UI 604. That is, the control unit 152 performs angle acquisition processing to acquire the rotation angle of the image relative to the image display area 600 of the GUI 500 on the display screen. Note that the following description will also take as an example a case where the first image capture unit 110 is selected in the image capture unit selection UI 604. In this case, the control unit 152 acquires the rotation angle θ of the first image sensor 112 by the first rotation driver 117 as the angle acquisition processing.

[0088] Then, based on the rotation angle θ, the control unit 152 controls the first driving unit 113 so that the shooting direction of the first imaging unit 110 moves in the same direction as the movement direction instructed by the user by selecting the pan / tilt button 701. In other words, the control unit 152 controls the driving of at least one of the first pan driving unit 114 and the first tilt driving unit 116 so that the shooting direction of the first imaging unit 110 moves in the same direction as the direction indicated by the pan / tilt button 701 selected by the user.

[0089] For example, if the pan / tilt button 701 is selected when the rotation angle θ is 0 degrees, the control unit 152 drives the first pan driving unit 114 to pan rightward so that the shooting direction of the first imaging unit 110 moves rightward, as described in Fig. 11. At this time, the control unit 152 also controls the first tilt driving unit 116 not to drive it.

[0090] Furthermore, for example, if the pan / tilt button 701 is selected when the rotation angle θ is +90 degrees, the control unit 152 controls the first tilt driving unit 116, rather than the first pan driving unit 114, to drive it in the upward (+) direction by the driving amount α. As a result, the shooting direction of the first imaging unit 110 moves in the right (+) direction, and the image on the right side of the image 401 is displayed in the image display area 600 in FIG. 12(b). In other words, the user can intuitively perform the operation without being conscious of the rotation angle.

[0091] Furthermore, for example, when the pan / tilt button 702 is selected when the rotation angle θ is +90 degrees, the control unit 152 controls the first tilt drive unit 116 to drive the first tilt drive unit 116 in the downward (-) direction by the drive amount α. In this case, the shooting direction of the first imaging unit 110 moves in the left (-) direction with respect to the image displayed in the image display area 600. As a result, the image displayed in the image display area 600 in FIG. 12(b) shows an image on the left (-) side with respect to the image 401, which coincides with the direction of movement to the left instructed by the user by selecting the pan / tilt button 702.

[0092] Fig. 13(b) is a diagram showing an example of the GUI 500 displayed on the display device of the client device 200 by the OSD display data generated by the image processing unit 151 of the camera 100 when the rotation angle θ is +45 degrees (θ=+45). Note that the example of Fig. 13(b) also shows an example in which the first imaging unit 110 is selected by selecting button A in the imaging unit selection UI 604. Fig. 13(a) shows a state in which the axis of the first pan driving unit 114 and the axis of the first tilt driving unit 116 are rotated by +45 degrees by rotation driving of the first rotation driving unit 117.

[0093] If the pan / tilt button 701 is selected when the rotation angle θ is +45 degrees, the control unit 152 controls the drive of the first drive unit 113 so as to move the shooting direction of the first imaging unit 110 to the right with respect to the image 401 in the image display area 600. At this time, the control unit 152 controls the drive so that the resultant vector of the pan direction by the first pan drive unit 114 and the tilt direction by the first tilt drive unit 116 becomes a drive amount α. As a result, the shooting direction of the first imaging unit 110 moves to the right (+), and an image on the right (+) side with respect to the image 401 is displayed in the image display area 600 in FIG. 13(b).

[0094] In this way, the control unit 152 of the camera 100 changes the drive and drive ratio of the first pan drive unit 114 and the first tilt drive unit 116, which are controlled in response to the operation of the pan / tilt button, based on the rotation angle of the first rotation drive unit 117. This allows the user to intuitively give instructions (operations) to change the movement direction that moves the shooting direction of the imaging unit while looking at the displayed image, without being aware of the rotation angle of the rotation drive unit.

[0095] FIG. 14 is a diagram used to explain the above-mentioned method of calculating the composite vector. 14, an example in which the pan / tilt button 701 is operated will be described, but a resultant vector can also be calculated in the same way when a pan / tilt button indicating another direction is operated. In Fig. 14, an example in which the first imaging unit 110 is selected in the imaging unit selection UI 604 will be described.

[0096] The ratio of the drive amount of the first drive unit 113 corresponding to the first imaging unit 110 to the drive amount of the first pan drive unit 114 and the drive amount of the first tilt drive unit 116 can be calculated from the rotation angle θ and the drive amount α indicated by the magnitude of the resultant vector 801. If the pan drive amount of the first pan drive unit 114 is P and the tilt drive amount of the first tilt drive unit 116 is T, the pan drive amount P and the tilt drive amount T are expressed by the following equations (1) and (2) using trigonometric functions.

[0097] P=α*cosθ Equation (1) T=α*sinθ Equation (2)

[0098] By driving the first pan driving unit 114 with a pan driving amount P and the first tilt driving unit 116 with a tilt driving amount T, the shooting direction of the first imaging unit 110 can be controlled to the direction and driving amount of the composite vector 801. In the above example, the pan drive amount P and tilt drive amount T are calculated from the resultant vector 801. However, if the movement direction instructed by the user matches the movement direction relative to the image 401, the user can perform intuitive operations. Therefore, it is sufficient to obtain the same result as the ratio calculated by equations (1) and (2), and for example, equations (1) and (2) may be multiplied by the same coefficient. Furthermore, in practice, the first drive unit 113 (first pan drive unit 114 and first tilt drive unit 116) has limitations on drive control (minimum drive amount setting value and setting interval), so it is desirable to adjust and control to the closest drive amount within the range of drive controllability.

[0099] FIG. 15 is a flowchart of the control process when the control unit 152 performs the above-mentioned control. When the control signal transmitted from the client device 200 includes an instruction to change the rotation angle, the control unit 152 performs the process of the flowchart in FIG. First, in the process of step S901, the control unit 152 acquires a control signal sent from the client device 200. The control unit 152 determines whether the user instruction in the control signal includes an instruction to change the rotation angle of the image sensor, and if so, reads information about the rotation angle. Next, in the process of step S902, the control unit 152 calculates the drive amount (ratio) of the pan drive unit and the tilt drive unit according to the operation information of the pan-tilt button from the read rotation angle and the operation information of each pan-tilt button included in the control signal. The drive amount (ratio) at this time may be calculated as needed from the composite vector as described above, or may be read from a table calculated in advance. Then, the control unit 152 controls the drive of the pan drive unit and the tilt drive unit according to the calculated drive amount (ratio) of the pan drive unit and the tilt drive unit.

[0100] <Additional information about the pan / tilt buttons> In this embodiment, the type and shape of the UI are not limited to pan-tilt buttons, and various types and shapes may be used, as long as the direction of movement of the imaging unit is the same as the direction of movement instructed by the user on the GUI 500, regardless of the rotation angle. For example, the UI may be a button labeled "Right +30 degrees" (corresponding to the pan-tilt button 701), a joystick button 709, or a drive amount selection button (described later). The pan-tilt button may also be a symbol indicating the direction, such as a triangle or arrow, or text indicating the direction, such as "left," "right," "up," or "down." This may allow the user to perform operations more intuitively. Furthermore, the button placement may be associated with the movement direction, such as by placing a button indicating the right direction on the right side of the displayed image, thereby enabling the user to perform operations more intuitively.

[0101] <Drive amount selection button> FIG. 16 is a diagram showing an example of the drive amount selection button 730. As shown in FIG. The drive amount selection buttons 730 are displayed as a UI in, for example, the operation area 601. In FIG. 16, a left / right drive amount selection button 731 and a up / down drive amount selection button 732 are shown as examples of the drive amount selection buttons 730. The left / right drive amount selection buttons 731 are made up of six buttons, and are used to change the pan drive amount from the current pan drive position by the pan drive unit by -30 degrees, -20 degrees, -10 degrees, +10 degrees, +20 degrees, and +30 degrees from the left end. When the -30 degree button of the left / right drive amount selection buttons 731 is selected, the pan drive unit changes the pan drive position by 30 degrees in the left (-) direction, and when the +30 degree button is selected, the pan drive unit changes the pan drive position by 30 degrees in the right (+) direction. Similarly, the up / down drive amount selection buttons 732 are composed of six buttons, and are buttons for manipulating the tilt drive amount from the current tilt drive position by the tilt drive unit by -30 degrees, -20 degrees, -10 degrees, +10 degrees, +20 degrees, and +30 degrees from the left end. When the -30 degree button of the up / down drive amount selection buttons 732 is selected, the tilt drive unit changes the tilt drive position by 30 degrees in the downward (-) direction, and when the +30 degree button is selected, the tilt drive position changes by 30 degrees in the upward (+) direction.

[0102] The control unit 152 of the present embodiment described above can handle such drive amount selection buttons 730 in the same way as the pan-tilt buttons described above. For example, the third button from the right of the left / right drive amount selection buttons 731 (+10 degrees) is handled in the same way as the pan-tilt button 701, and when the third button is selected, the drive amount is determined to be three times (3*α) or 30 degrees.

[0103] Furthermore, apart from the pan / tilt buttons 701-708 and the drive amount selection button 730, there may be a button for driving only the pan drive unit or a UI for driving only the tilt drive unit. When these buttons are operated, the direction of movement of the shooting direction relative to the displayed image differs depending on the rotation angle. Therefore, when using these UIs, it is desirable to separate the area within the operation area 601 from the above-mentioned image capture unit operation UI 603 so that these UIs cannot be operated simultaneously. Furthermore, the camera 100 may have two different modes so that the user can switch between them depending on whether they want to specify the direction of movement relative to the displayed image or the drive unit. Furthermore, in this embodiment, the direction of movement of the shooting direction relative to the displayed image is the direction indicated by the pan / tilt buttons 701 to 708, but this is not limiting. For example, the direction of movement of the subject within the screen may be used instead of the direction of movement of the shooting direction relative to the displayed image. When the direction is the movement direction of the subject within the screen, the direction indicated by the pan / tilt buttons 701 to 708 is the exact opposite (180-degree rotated) to the direction of movement of the shooting direction relative to the displayed image. In other words, the UI for the user to instruct the direction of movement of the shooting direction relative to the displayed image, such as the pan / tilt buttons, and the UI indicating the movement direction of the subject in the image are UIs indicating the exact opposite directions.

[0104] <<Second embodiment>> In the first embodiment, an example was described in which the image sensor (or the image sensor) is rotationally driven by the rotation drive unit, whereas in the second embodiment, an example will be described in which not only the image sensor (or the image sensor) is rotationally driven, but also a display image is rotationally driven around the center of the image as the rotation axis. 17 and 18 are diagrams showing an example in which an image in the image display area 600 is rotated around the center of the image in the GUI 500 similar to those shown in Figures 11 to 13. In Figures 17 and 18, an example will be described in which the first imaging unit 110 is selected in the imaging unit selection UI 604.

[0105] As in the second embodiment, when rotating the image 401 in the image display area 600, the user operates a display rotation button (not shown) provided in the operation buttons 605. Then, the client device 200 sends a control signal including operation information according to the user instruction to the camera 100. Although the display rotation button is not shown, it may be, for example, eight buttons for specifying a rotation angle in 45-degree increments, similar to the pan / tilt buttons 701 to 708, as the rotation angle for rotating the display image. Also, for example, instead of the display rotation button, a text box for inputting the rotation angle of the display image as a numerical value may be used.

[0106] The control unit 152 of the camera 100 causes the image processing unit 151 to perform image processing such as rotating the image, based on a control signal including operation information of the rotation angle selected by a user instruction via the operation button 605. Then, the rotated image is sent to the client device 200, whereby the rotated image 401 is displayed in the image display area 600 of the GUI 500.

[0107] With reference to FIG. 17, a case where the rotation angle φ of the display image is +90 degrees (φ=+90 degrees) will be described. Here, the description will be made using the rotation angle θ set by the rotation driver and the rotation angle φ of the display image, but θ and φ are values ​​of the same dimension and can be added together. The sum of the rotation angle θ set by the rotation driver and the rotation angle φ of the display image is the rotation angle ψ in the second embodiment, which can be expressed by equation (3). Note that in the first embodiment, the rotation angle φ of the display image is 0 degrees (φ=0 degrees), so the added rotation angle ψ is the same as the rotation angle θ set by the rotation driver (ψ=θ).

[0108] θ+φ=ψ Equation (3)

[0109] FIG. 17 shows the image 401 rotated 90 degrees clockwise compared to the example in FIG. 11 described above, and it can be seen that the image 401 is now vertically elongated relative to the horizontal direction. The rotation angle φ at this time is +90 degrees. Note that the rotation angle θ set by the rotation driver is 0 degrees. Therefore, the total rotation angle ψ is +90 degrees. Furthermore, the total rotation angle ψ in the example in FIG. 17 matches the rotation angle θ in FIG. 12 (θ = ψ = +90 degrees). That is, as shown in FIG. 17, the subject 301 shown in the image 401 is tilted +90 degrees, similar to the example in FIG. 12. Therefore, when, for example, the pan / tilt button 701 is selected in the GUI 500 in FIG. 17, the control unit 152 can control the pan driver and tilt driver in the same way as when the pan / tilt button 701 is selected in the example in FIG. 12 described above.

[0110] Next, with reference to FIG. 18, a case where the rotation angle φ of the displayed image is +45 degrees (φ=+45 degrees) will be described. That is, FIG. 18 shows the state where image 401 has been rotated (rotated) 45 degrees clockwise compared to the example of FIG. 11 described above, and it can be seen that image 401 is tilted. In this case, the total rotation angle ψ in FIG. 18 matches the rotation angle θ in FIG. 13 (θ=ψ=+45 degrees). That is, as shown in FIG. 18, subject 301 reflected in image 401 has the same rotation inclination as in the example of FIG. 13. Therefore, in the example of GUI 500 in FIG. 18, control unit 152 only needs to control the pan driving unit and tilt driving unit in the same manner as in the example of FIG. 13 described above.

[0111] Next, referring to FIG. 19, a case where the rotation angle θ set by the rotation drive unit and the rotation angle φ of the display image are combined as in the first embodiment will be described. The example of FIG. 19 illustrates a state where the rotation angle θ of the rotation drive unit is driven by −90 degrees from a state where the rotation angle φ of the display image is +90 degrees and the rotation angle θ set by the rotation drive unit is 0 degrees. In this case, the image 401 becomes vertically elongated relative to the horizontal direction, and the subject 301 in the image is upright. The total rotation angle ψ at this time is 0 degrees, which is consistent with the rotation angle θ shown in FIG. 11 (θ = ψ = 0 degrees). That is, the total rotation angle ψ in the case of FIG. 19 is the same as the rotation angle in the example of FIG. 11. Therefore, in this example, it is clear that the control unit 152 may perform control similar to that of the drive unit shown in FIG. 11.

[0112] 17 to 19, in the second embodiment, the same control as in the first embodiment can be performed based on the rotation angle obtained by adding together the rotation angle of the display image and the rotation angle by the rotation drive unit. That is, in the second embodiment as well, the user can intuitively give instructions (operations) to change the movement direction of the display image while viewing the display image, regardless of the rotation angle of the display image. When the image display is mirrored (flipped horizontally or vertically), it is desirable to also reverse the drive direction (calculated resultant vector). This allows the user to intuitively operate the device without being aware of mirroring or rotation.

[0113] In the second embodiment, in step S901 of the flowchart in Fig. 15, control unit 152 acquires a control signal sent from client device 200. Control unit 152 determines whether the user instruction in the control signal includes an instruction to change the rotation angle or an instruction to change the rotation angle of the displayed image. If the instruction to change is included, control unit 152 reads information about the rotation angle. Next, in the process of step S902, the control unit 152 calculates the drive amount (ratio) of the pan drive unit and the tilt drive unit according to the operation information of the pan-tilt button from the read rotation angle and the operation information of each pan-tilt button included in the control signal. The drive amount (ratio) at this time may be calculated as needed from the composite vector as described above, or may be read from a table calculated in advance. Then, the control unit 152 controls the drive of the pan drive unit and the tilt drive unit according to the calculated drive amount (ratio) of the pan drive unit and the tilt drive unit.

[0114] <<Third embodiment>> Next, in a third embodiment, a case where the user selects a plurality of imaging units will be described. FIG. 20 shows the GUI 500 when the user selects all of the buttons A to D in the image capture unit selection UI 604, that is, when all of the first to fourth image capture units 110 to 140 are selected.

[0115] The user can select multiple image capture units through operations on the image capture unit selection UI 604, and can further instruct the display of multiple images in the image display area 600 by selecting, for example, a multiple image display button (not shown) provided in the operation buttons 605. In this way, when the user selects multiple image capture units through the image capture unit selection UI 604 and further instructs the display of multiple images using the multiple image display button, a control signal including information on the selection and instruction is sent from the client device 200 to the camera 100.

[0116] Based on a control signal including instructions to select multiple image capture units and display multiple images, the control unit 152 of the camera 100 causes the image processing unit 151 to perform image processing to generate an image in which multiple images captured by the multiple image capture units are arranged. As described above, examples of the arrangement of multiple images include a vertical and horizontal arrangement, a single horizontal line, or a single vertical arrangement. The multiple images arranged by the image processing unit 151 are then sent from the camera 100 to the client device 200 and displayed.

[0117] 20 shows an example of GUI 500 in which four images 401 to 404 captured by the first to fourth imaging units 110 to 140 are arranged vertically and horizontally and displayed in image display area 600. Image 401 is an image captured by the first imaging unit 110, image 402 is an image captured by the second imaging unit 120, image 403 is an image captured by the third imaging unit 130, and image 404 is an image captured by the fourth imaging unit 140. Numbers "A" to "D" corresponding to the buttons selected in imaging unit selection UI 604 are superimposed on the four images 401 to 404 captured by the first to fourth imaging units 110 to 140. This allows the user to recognize which of the first to fourth imaging units 110 to 140 the images 401 to 404 displayed in image display area 600 correspond to.

[0118] It is also assumed that the first to fourth imaging units 110-140 each have a different imaging direction, and that the first to fourth imaging units 110-140 or displayed images each have a different rotation angle. Note that subject 311 in image 401, subject 321 in image 402, subject 331 in image 403, and subject 341 in image 404 displayed in image display area 600 are all different subjects, but for the sake of simplicity, they are shown as the same subject.

[0119] In the example of Fig. 20, the rotation angle (ψ = φ + θ) of the first imaging unit 110 is assumed to be the same as in the example of Fig. 11 described above. The rotation angle (ψ = φ + θ) of the second imaging unit 120 is assumed to be the same as in the example of Fig. 12. The rotation angle (ψ = φ + θ) of the third imaging unit 130 is assumed to be the same as in the example of Fig. 17. The rotation angle (ψ = φ + θ) of the fourth imaging unit 140 is assumed to be the same as in Fig. 19.

[0120] Here, suppose that the user selects, for example, pan / tilt button 701 and a corresponding control signal is transmitted to camera 100. In this case, control unit 152 of camera 100 performs pan / tilt drive control on first to fourth drive units 113-143 in the same manner as described above, based on the rotation angles (ψ=φ+θ) of the first to fourth image capture units 110-140, respectively. That is, control unit 152 performs pan / tilt drive control on first drive unit 113 corresponding to first image capture unit 110 in the same manner as described with reference to FIG. 11. Furthermore, control unit 152 performs pan / tilt drive control on second drive unit 123 corresponding to second image capture unit 120 in the same manner as described with reference to FIG. 12. Similarly, control unit 152 performs pan / tilt drive control on third drive unit 133 corresponding to third image capture unit 130 in the same manner as described with reference to FIG. 17. Similarly, the control section 152 performs pan / tilt drive control on the fourth drive section 143 corresponding to the fourth imaging section 140 in the same manner as described with reference to FIG.

[0121] Thus, according to the third embodiment, even when multiple imaging units are selected, the user can intuitively instruct (operate) to change the shooting direction of each imaging unit while looking at the displayed image.

[0122] 20, all the symbols of the imaging units in the imaging unit position display UI 602 are selected, and therefore all of them are filled in with a specific color. The symbols displayed in the imaging unit position display UI 602 indicate the relative positions of the first to fourth pan drive units 114 to 144, and the shooting directions (directions of the shooting angle of view) of the first to fourth imaging units 110 to 140 are different. Therefore, when multiple imaging units are selected, it is desirable to emphasize and make the imaging unit position display UI 602 invisible to the user, for example, by graying out the imaging unit position display UI 602 (represented by diagonal lines in FIG. 20) or hiding it. This allows the user to concentrate on operations while viewing the image in the image display area 600, and the occurrence of erroneous operations can be reduced. When multiple imaging units are selected, it is desirable to similarly gray out the imaging unit position display UI 602, setting it to inoperable so that it does not accept any operations by the user.

[0123] <<Fourth embodiment>> In the first and second embodiments, the camera 100 is an example of a multi-lens camera having multiple imaging units, but in the fourth embodiment, a case will be described where the camera 100 is a monocular camera having a single imaging unit. FIG. 21 is a diagram showing an example of the functional configuration of an imaging system according to a fourth embodiment in which a camera 100 having a single imaging unit and a drive unit and a client device 200 are connected via a network 170. In FIG. 21, the single imaging unit and drive unit are exemplified by the first imaging unit 110 and the first drive unit 113 described above, and in the fourth embodiment, the imaging unit 110 and the drive unit 113 will be described as such. When the camera 100 has only one imaging unit 110 as in the present embodiment, there is no need to separate the pan drive unit and the torsion drive unit as in the previous embodiment, and therefore the drive unit 113 integrates them as the pan drive unit 114. Note that, apart from the fact that the camera 100 has a single imaging unit 110 and the drive unit 113, the configuration is generally similar to that of FIG. 1 described above, and therefore a description of the similar configuration will be omitted.

[0124] Fig. 22 is a diagram showing an example of the hardware configuration of an imaging system according to the fourth embodiment. In the case of Fig. 22, unlike the above-mentioned Fig. 7, the camera 100 has one imaging unit 110 and a driving unit 113. In Fig. 22, the configuration is also generally similar to the above-mentioned Fig. 7 except that the camera 100 has one imaging unit 110 and a driving unit 113, and therefore a description of the similar configuration will be omitted.

[0125] FIG. 23 is a diagram showing a schematic external configuration example of a camera 100 according to a fourth embodiment. Unlike the multiple-lens camera shown in the first embodiment, the camera 100 of the fourth embodiment is a monocular camera having a single imaging unit 110. FIGS. 23(a) and 23(b) show examples of two types of cameras 100 with different designs. Both the cameras 100 shown in FIGS. 23(a) and 23(b) are installed on a wall, ceiling, or the like by a fixing unit 105, and the imaging unit 110 is driven in pan and tilt directions by a driving unit 113.

[0126] In the case of a camera 100 having a single imaging unit 110 and drive unit 113 as in the fourth embodiment, as in the previous embodiments, the user can intuitively give instructions (operations) to change the shooting direction while looking at the displayed image, without being aware of the rotation angle. Note that when the imaging unit 110 is configured as a single monocular camera, as in the camera 100 of this embodiment, the imaging unit selection UI 604 of the GUI 500 described above is unnecessary.

[0127] <<Fifth Embodiment>> In the fifth embodiment, a case where the tilt angle is within a predetermined threshold angle range will be described in particular. In this embodiment, +90 degrees (+90 degrees relative to the horizontal direction) will be described as an example of the predetermined threshold angle range for the tilt angle. The camera 100 of the fifth embodiment has first to fourth imaging units 110 to 140 and first to fourth driving units 113 to 143. The functional configurations and hardware configurations of the camera 100 and the client device 200 are the same as those of the first to third embodiments, and therefore description thereof will be omitted. In the fifth embodiment, as described above, the first imaging unit 110 and the first driving unit 113 will be described as representative examples.

[0128] 24(a) is a diagram showing that the first imaging unit 110 appears to rotate when the first imaging unit 110 is panned by the first pan driving unit 114 while the tilt angle of the first tilt driving unit 116 is set to +90 degrees. FIG. 24(a) is a diagram similar to FIG. 3 described above, but unlike the example of FIG. 3, the tilt angle of the first imaging unit 110 is set to +90 degrees. Note that FIG. 24(a) also depicts the lenses of the imaging optical system 111 to clearly show the shooting direction of the first imaging unit 110.

[0129] FIG. 24(b) is a diagram showing examples of positions of the first imaging unit 110 when the first pan driving unit 114 moves the first imaging unit 110 on the circumference 102. Similar to FIG. 2, FIG. 24(b) is a schematic diagram focusing only on the first imaging unit 110 when pan driving is performed by the first pan driving unit 114, and does not show the second to fourth imaging units 120 to 140 other than the first imaging unit 110. That is, FIG. 24(b) shows the first imaging unit 110 at positions 911 to 914 when the first imaging unit 110 is moved by the first pan driving unit 114 in 90-degree increments on the circumference 102. Position 911 shown in FIG. 24(b) indicates the position of the first imaging unit 110 at its initial position (0 degrees) when pan driving is performed by the first pan driving unit 114. Position 912 in Figure 24(b) indicates the position where the first imaging unit 110 is panned by +90 degrees from the initial position, and similarly, position 912 indicates the position where it is panned by +180 degrees from the initial position, and position 914 indicates the position where it is panned by +270 degrees from the initial position.

[0130] 24(b), in order to clearly show that the first image capture unit 110 appears to rotate when moved on the circumference 102 by the first pan drive unit 114, a mark 304 is drawn at a fixed position in the upper right corner when viewed from the front of the first image capture unit 110. When viewing the mark 304 of the first image capture unit 110 from the side of the subject being photographed (the front side of the page), it can be seen that when the first image capture unit 110 is panned in the + direction, the mark 304 appears to rotate clockwise around the first image capture unit 110. In this case, the subject viewed from the first image capture unit 110 appears to rotate in the same direction as described in FIG. 9 (the + direction of the first rotation drive unit 117), that is, rotate clockwise. For this reason, when the tilt angle of the first image capture unit 110 is +90 degrees, it is desirable not to perform panning by the first pan drive unit 114. That is, when the tilt angle of the first imaging unit 110 is +90 degrees, the first pan driving unit 114 is not driven, thereby preventing an image with unnecessary rotation from being captured. Alternatively, when the tilt angle of the first imaging unit 110 is +90 degrees, the first rotation driving unit 117 may be driven to rotate in a rotation direction that cancels out the rotation that appears to occur due to the pan driving by the first imaging unit 110.

[0131] Furthermore, when the tilt angle of the first imaging unit 110 is in a +90-degree state, even if the user wants to move the imaging direction of the first imaging unit 110 horizontally, it is expected that the user will not know how to operate it. In this case, it is expected that the user will try to operate the first tilt driving unit 116 so as to change the tilt angle of the first imaging unit 110 toward 0 degrees. However, because the pan / tilt buttons 701 to 708 of the GUI 500 described above are not buttons that directly specify and drive the first tilt driving unit 116, it is expected that the user will not know how to cancel the +90-degree tilt angle state.

[0132] Therefore, in the fifth embodiment, in order to inform the user how to resolve the state in which the tilt angle is +90 degrees, the direction in which the tilt angle becomes 0 degrees is displayed in the GUI 500. 25 is a diagram showing an example of GUI 500 that displays arrow 305 indicating the direction in which the tilt angle is set to 0 degrees when, for example, the tilt angle of the first image capture unit 110 is +90 degrees. In this case, for example, pan / tilt buttons that the user cannot operate may be grayed out (or filled in with a specific color) to indicate that they are inoperable, or the pan / tilt buttons themselves may be hidden, thereby clearly indicating to the user only the operable drive directions. In this way, when the tilt angle of the image capture unit is +90 degrees, changing the display of GUI 500 to match the drivable directions allows the user to perform intuitive operations.

[0133] In the fifth embodiment, unnecessary rotation of the image can be suppressed by not driving the pan driving unit when the tilt angle is +90 degrees, but it is also possible to not drive the rotation driving unit instead of the pan driving unit. In this case, on the GUI 500, instead of graying out or hiding the pan / tilt button, the rotation buttons 710 and 711 may be grayed out or hidden.

[0134] In the fifth embodiment, an example was given in which the threshold angle range was a tilt angle of +90 degrees, but the threshold angle range is not limited to +90 degrees. For example, the threshold angle range may be a tilt angle range of +80 degrees to +100 degrees. The threshold angle range may be expanded beyond +80 degrees to +100 degrees, but the maximum angle range when the threshold angle range is expanded is +45 degrees to +135 degrees. This is because when the tilt angle is +45 degrees or less, the imaging unit also captures images in the horizontal direction, and panning by the pan driving unit appears to be movement rather than rotation.

[0135] In this embodiment, the movable range of the tilt drive unit is from 0 degrees to +90 degrees, but it is not limited to this and can be changed as appropriate. In addition, this embodiment can be similarly applied when the tilt angle is -90 degrees. Furthermore, the fifth embodiment has been described using an example of an imaging device having multiple imaging units as in the first to third embodiments, but it can also be applied to an imaging device having only a single imaging unit as shown in the fourth embodiment.

[0136] As shown in FIG. 24( b), the first imaging unit 110 moves on the circumference 102 due to the panning of the first pan driving unit 114. Strictly speaking, each position of the first imaging unit 110 when it moves sequentially on the circumference 102 is a different position, and therefore, the imaging range of the first imaging unit 110 at each position also moves. On the other hand, the smaller the distance between the position of the first imaging unit 110 and the axis 101 of the circumference 102, the more the first imaging unit 110 appears to rotate rather than move. For example, in the case of a monocular camera with only one imaging unit, there is no need for a common rotation axis for multiple imaging units as in a multi-lens camera. Since the rotation axis and the imaging unit are closer to each other, the first imaging unit 110 appears to rotate. Furthermore, for example, the wider the imaging optical system of the imaging unit, the less the effect of the distance between the imaging unit and the rotation axis on the angle of view, and therefore the first imaging unit 110 also appears to rotate.

[0137] <<Other embodiments>> The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. The above-described embodiments are merely examples of specific embodiments for carrying out the present invention, and the technical scope of the present invention should not be construed as being limited by these embodiments. In other words, the present invention can be carried out in various forms without departing from its technical concept or main features.

[0138] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) a direction acquisition means for acquiring a moving direction instructed by a user with respect to an image captured by the imaging unit and displayed; an angle acquisition means for acquiring at least one of a rotation angle of the imaging unit or an imaging element of the imaging unit around an optical axis of an imaging optical system, and a rotation angle of the displayed image around an image center; a control means for controlling a first driving means for moving the imaging direction of the imaging unit in a first direction and a second driving means for moving the imaging direction of the imaging unit in a second direction different from the first direction; and The control means controls the movement of the imaging direction of the imaging unit by the first driving means and the second driving means based on the rotation angle so that the imaging direction of the imaging unit matches the movement direction instructed by the user. (Configuration 2) 2. The control device according to configuration 1, wherein the control means controls a rotation means that rotates the imaging section or an imaging element of the imaging section around an optical axis of an imaging optical system. (Configuration 3) an image processing means for performing image processing to rotate the image captured by the imaging unit around the center of the image; 3. The control device according to configuration 1 or 2, wherein the control means controls image processing by the image processing means to rotate the image around the center of the image. (Configuration 4) The control device according to any one of configurations 1 to 3, wherein the angle acquisition means calculates the rotation angle based on a rotation of the imaging unit or an imaging optical system of an imaging element of the imaging unit around an optical axis, and a rotation of an image captured by the imaging unit and displayed around an image center. (Configuration 5) 5. The control device according to any one of configurations 1 to 4, further comprising an image processing means for generating user interface data for the user to instruct the movement direction for the image captured by the imaging unit and displayed, or user interface data indicating the movement direction of a subject shown in the image. (Configuration 6) The control device according to configuration 5, wherein the user interface for the user to instruct the movement direction of the image captured by the imaging unit and displayed and the user interface showing the movement direction of the subject shown in the image are user interfaces showing opposite directions. (Configuration 7) The control device according to any one of configurations 1 to 6, characterized in that the control means calculates a resultant vector of the first direction by the first drive means and the second direction by the second drive means based on the rotation angle, and determines a ratio of drive by the first drive means and the second drive means based on the resultant vector. (Configuration 8) 8. The control device according to configuration 7, wherein the control means determines the drive amounts of the first drive means and the second drive means based on the resultant vector. (Configuration 9) the first driving means includes a plurality of first driving means that individually move the imaging directions of the plurality of imaging units in the first direction, the second driving means includes a plurality of second driving means that individually move the imaging directions of the plurality of imaging units in the second direction, a plurality of rotation means for rotating the plurality of imaging units or the imaging elements of each imaging unit around the optical axis of the imaging optical system; the direction acquisition means acquires the movement direction instructed by the user with respect to an image captured by one or more imaging units selected by the user from the plurality of imaging units and displayed; 9. The control device according to any one of configurations 1 to 8, wherein the control means controls the plurality of first drive means, the plurality of second drive means, and the plurality of rotation means. (Configuration 10) an image processing means for generating user interface data for a user to instruct the plurality of movement directions; The control device according to configuration 9, characterized in that when a user selects multiple imaging units from the multiple imaging units, the image processing means displays a display that prevents the user from operating the user interface or hides the user interface. (Configuration 11) the first direction is a direction in which the imaging direction of the imaging unit is moved horizontally, the second direction is a direction in which the imaging direction of the imaging unit is moved vertically, The control device according to configuration 9 or 10, wherein the control means controls so as not to drive the first drive means or the rotation means when the shooting direction of the imaging unit moved in the second direction is within a predetermined angle range with respect to the horizontal direction. (Configuration 12) an image processing means for generating user interface data for a user to instruct the plurality of movement directions; The control device according to configuration 11, characterized in that when the control means controls the first driving means or the rotation means so as not to drive, the image processing means displays a display that prevents the user from operating the user interface or hides the user interface. (Configuration 13) The control device according to configuration 11, further comprising an image processing means for generating user interface data representing a direction for changing the shooting direction of the imaging unit to the horizontal direction when the shooting direction of the imaging unit moved in the second direction is within a predetermined angle range with respect to the horizontal direction. (Configuration 14) an image processing means for mirror-inverting the image captured by the imaging unit; 14. The control device according to any one of configurations 1 to 13, wherein the control means reverses the movement direction of the shooting direction by the first drive means and the second drive means when the mirror inversion is performed by the image processing means. (Method 1) a direction acquisition step of acquiring a movement direction instructed by a user with respect to an image captured by the imaging unit and displayed; an angle acquisition step of acquiring at least one of a rotation angle of the imaging unit or an imaging element of the imaging unit around an optical axis of an imaging optical system, and a rotation angle of the displayed image around an image center; a control step of controlling a first driving means for moving the imaging direction of the imaging unit in a first direction and a second driving means for moving the imaging direction of the imaging unit in a second direction different from the first direction; and The control method is characterized in that the control step controls the movement of the imaging direction of the imaging unit by the first driving means and the second driving means based on the rotation angle so that the imaging direction of the imaging unit matches the movement direction instructed by the user. (Program 1) A program for causing a computer to function as the control device according to any one of configurations 1 to 14. [Explanation of symbols]

[0139] 100: imaging device, 110-140: first to fourth imaging units, 113-143: first to fourth drive units, 114-144: first to fourth pan drive units, 115-145: first to fourth zoom drive units, 116-146: first to fourth tilt drive units, 117-147: first to fourth rotation drive units, 118-148: first to fourth focus drive units, 119-149: first to fourth torsion drive units, 151: image processing unit, 152: control unit, 153: communication unit, 154: recording unit, 200: client device

Claims

1. a direction acquisition means for acquiring a moving direction instructed by a user with respect to an image captured by the imaging unit and displayed; an angle acquisition means for acquiring at least one of a rotation angle of the imaging unit or an imaging element of the imaging unit around an optical axis of an imaging optical system, and a rotation angle of the displayed image around an image center; a control means for controlling a first driving means for moving the imaging direction of the imaging unit in a first direction and a second driving means for moving the imaging direction of the imaging unit in a second direction different from the first direction; and The control means controls the movement of the imaging direction of the imaging unit by the first driving means and the second driving means based on the rotation angle so that the imaging direction of the imaging unit matches the movement direction instructed by the user.

2. 2. The control device according to claim 1, wherein the control means controls a rotation means that rotates the image pickup unit or an image pickup element of the image pickup unit around an optical axis of an imaging optical system.

3. an image processing means for performing image processing to rotate the image captured by the imaging unit around the center of the image; 2. The control device according to claim 1, wherein the control means controls image processing by the image processing means to rotate the image around the center of the image.

4. The control device according to claim 1, characterized in that the angle acquisition means calculates the rotation angle based on a rotation about an optical axis of the imaging optical system of the imaging unit or the imaging element of the imaging unit, and a rotation about an image center of the image captured by the imaging unit and displayed.

5. The control device according to claim 1, further comprising an image processing means for generating user interface data for the user to indicate the movement direction for the image captured by the imaging unit and displayed, or user interface data indicating the movement direction of a subject appearing in the image.

6. The control device according to claim 5, wherein the user interface for the user to indicate the direction of movement for the image captured by the imaging unit and displayed and the user interface indicating the direction of movement of the subject shown in the image are user interfaces indicating opposite directions.

7. 2. The control device according to claim 1, wherein the control means calculates a resultant vector of the first direction by the first drive means and the second direction by the second drive means based on the rotation angle, and determines a ratio of drive by the first drive means and drive by the second drive means based on the resultant vector.

8. 8. The control device according to claim 7, wherein the control means determines the drive amounts of the first drive means and the second drive means based on the resultant vector.

9. the first driving means includes a plurality of first driving means that individually move the photographing directions of the plurality of image capturing units in the first direction, the second driving means includes a plurality of second driving means that individually move the photographing directions of the plurality of image capturing units in the second direction, a plurality of rotation means for rotating the plurality of imaging units or the imaging elements of each imaging unit around the optical axis of the imaging optical system; the direction acquisition means acquires the movement direction instructed by the user with respect to an image captured by one or more imaging units selected by the user from the plurality of imaging units and displayed; 2. The control device according to claim 1, wherein the control means controls the plurality of first driving means, the plurality of second driving means, and the plurality of rotating means.

10. an image processing means for generating user interface data for a user to instruct the plurality of movement directions; 10. The control device according to claim 9, wherein when a user selects multiple imaging units from the multiple imaging units, the image processing means displays a display that prevents the user from operating the user interface or hides the user interface.

11. the first direction is a direction in which the imaging direction of the imaging unit is moved horizontally; the second direction is a direction in which the imaging direction of the imaging unit is moved vertically; 10. The control device according to claim 9, wherein the control means controls so as not to drive the first drive means or the rotation means when the shooting direction of the imaging unit moved in the second direction is within a predetermined angle range with respect to the horizontal direction.

12. an image processing means for generating user interface data for a user to instruct the plurality of movement directions; The control device according to claim 11, characterized in that when the control means controls the first driving means or the rotation means so as not to drive, the image processing means displays a display that prevents the user from operating the user interface or hides the user interface.

13. 12. The control device according to claim 11, further comprising an image processing means for generating user interface data representing a direction in which the shooting direction of the imaging unit moved in the second direction is set to the horizontal direction when the shooting direction of the imaging unit is within a predetermined angle range with respect to the horizontal direction.

14. an image processing means for mirror-inverting the image captured by the imaging unit; 2. The control device according to claim 1, wherein the control means reverses the moving directions of the first and second driving means in the photographing direction when the mirror inversion is performed by the image processing means.

15. a direction acquisition step of acquiring a movement direction instructed by a user with respect to an image captured by the imaging unit and displayed; an angle acquisition step of acquiring at least one of a rotation angle of the imaging unit or an imaging element of the imaging unit around an optical axis of an imaging optical system, and a rotation angle of the displayed image around an image center; a control step of controlling a first driving means for moving the imaging direction of the imaging unit in a first direction and a second driving means for moving the imaging direction of the imaging unit in a second direction different from the first direction; and The control method is characterized in that the control step controls the movement of the imaging direction of the imaging unit by the first driving means and the second driving means based on the rotation angle so that the imaging direction of the imaging unit matches the movement direction instructed by the user.

16. Computer, a direction acquisition means for acquiring a moving direction instructed by a user with respect to an image captured by the imaging unit and displayed; an angle acquisition means for acquiring at least one of a rotation angle of the imaging unit or an imaging element of the imaging unit around an optical axis of an imaging optical system, and a rotation angle of the displayed image around an image center; a control means for controlling a first driving means for moving the imaging direction of the imaging unit in a first direction and a second driving means for moving the imaging direction of the imaging unit in a second direction different from the first direction; and The control means is a program that functions as a control device that controls the movement of the shooting direction of the imaging unit by the first driving means and the second driving means based on the rotation angle so that the shooting direction of the imaging unit matches the movement direction instructed by the user.

Images