Control device, control method, and program

Abstract

The goal is to enable users to intuitively operate an imaging device equipped with multiple imaging units. [Solution] A control device for controlling an imaging apparatus comprising a plurality of imaging means, a pan driving means for panning the plurality of imaging means, a tilt driving means for tilting the plurality of imaging means, and a rotation driving means for rotating the plurality of imaging means around an optical axis, wherein the control device has a display control means for displaying a plurality of captured images captured by the plurality of imaging means and an operation UI for operating at least one of the pan driving means and the tilt driving means, and the display control means rotates at least one of the captured images included in the plurality of captured images and the operation UI according to the rotation driving state by the rotation driving means in at least one of the selected imaging means among the plurality of imaging means.

Description

Technical Field

[0001] The present invention relates to a control device, a control method, and a program for an imaging device.

Background Art

[0002] An imaging control system including an imaging device such as a surveillance camera and a control device for controlling the imaging device is known. The control device is an example of an information processing device. In the imaging control system, the pan-tilt drive mechanism can be controlled by an operation on the control device. And an image captured by the imaging device can be displayed on the control device.

[0003] Further, the control device can control a rotation drive mechanism that rotates an imaging unit including an imaging sensor and a lens of the imaging device around the optical axis. When the imaging device is installed, the image displayed on the control device may be in an oblique state, but since the imaging device has a rotation drive mechanism, this can be corrected to a upright state. Generally, since the imaging sensor is rectangular, the angle of view in the long side direction is wide. By having a rotation drive mechanism, the angle of view can be widened in an arbitrary direction.

[0004] When a pan-tilt operation is performed, the image displayed on the control device usually moves in the operated direction. The user can look at the displayed image and perform a tilt operation upward, for example, if the shooting range is desired to be upward. On the other hand, in the state where the imaging unit is rotated, the shooting range moves in the rotated direction. For example, when a tilt operation is performed in a state rotated by 45 degrees, the shooting range moves in the 45-degree direction instead of the vertical direction. For this reason, there is a problem that the operation and the moving direction of the shooting range do not match and the user cannot intuitively operate. In contrast, in Patent Document 1, the operation UI (user interface) of the pan-tilt is rotated according to the rotation state.

Prior Art Documents

Patent Documents

[0005] [Patent Document 1] Japanese Patent Publication No. 2012-23434 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] Patent Document 1 describes a single imaging unit and a single pan-tilt operation UI. In recent years, multi-lens imaging devices equipped with multiple imaging units have emerged to simultaneously capture and monitor multiple directions. Many multi-lens imaging devices allow for simultaneous operation of multiple imaging units to enable rapid operation. Furthermore, to increase the area of ​​the image displayed on the control device, there is often only one operation UI rather than one for each image. In multi-lens imaging devices, each imaging unit may be rotated to a different angle, so it is not possible to rotate a single pan-tilt operation UI to match the rotation state, as in Patent Document 1.

[0007] As described above, in imaging control systems that include an imaging device equipped with multiple imaging units and a control device that controls them, there was a problem in that the pan-tilt operation and the direction of movement of the imaging range did not coincide, making it difficult for the user to operate intuitively.

[0008] Therefore, the objective of the present invention is to enable users to intuitively operate an imaging device equipped with multiple imaging units. [Means for solving the problem]

[0009] A control device according to one embodiment of the present invention is A control device for controlling an imaging apparatus comprising a plurality of imaging means, a pan driving means for panning the plurality of imaging means, a tilt driving means for tilting the plurality of imaging means, and a rotation driving means for rotating the plurality of imaging means around an optical axis, wherein the control device has a display control means for displaying a plurality of captured images captured by the plurality of imaging means and an operation UI for operating at least one of the pan driving means and the tilt driving means, and the display control means rotates at least one of the captured images included in the plurality of captured images and the operation UI according to the rotation driving state by the rotation driving means in at least one of the selected imaging means among the plurality of imaging means. [Effects of the Invention]

[0010] According to the present invention, an imaging device equipped with multiple imaging units can be operated intuitively by the user. [Brief explanation of the drawing]

[0011] [Figure 1] This figure shows the configuration of the imaging control system according to Embodiment 1 of the present invention. [Figure 2] This is a block diagram showing the hardware configuration of an imaging control system according to Embodiment 1 of the present invention. [Figure 3] This is a block diagram showing the functional configuration of an imaging control system according to Embodiment 1 of the present invention. [Figure 4] This flowchart shows the processing procedure of the imaging control system according to Embodiment 1 of the present invention. [Figure 5] This figure shows an example of the display of the imaging control system according to Embodiment 1 of the present invention. [Figure 6] This figure shows another example of the display image of the imaging control system according to Embodiment 1 of the present invention. [Figure 7] This figure shows an example of the display of the imaging control system according to Embodiment 2 of the present invention. [Figure 8]This figure shows an example of the display of the imaging control system according to Embodiment 3 of the present invention. [Modes for carrying out the invention]

[0012] Embodiments of the present invention will be described below with reference to the attached drawings. Note that the configurations shown in the following embodiments are merely examples and are not limited to those illustrated.

[0013] (Embodiment 1) Figure 1 is a diagram showing the configuration of an imaging control system according to Embodiment 1 of the present invention. The imaging control system 10 according to Embodiment 1 is configured to include an imaging device 100 and a control device 120.

[0014] The imaging device 100 has imaging units 105, 106, 107, and 108 arranged along its outer circumference. In this embodiment, the case in which the imaging device 100 has four imaging units is described, but the present invention is not limited to this, and the imaging device 100 may have any configuration having multiple imaging units. The imaging units 105 to 108 can be panned in the direction of the outer circumference of the imaging device 100 indicated by the arrow 115. The control device 120 has a display unit 126 located at the top of the control device 120 and an operation unit 124 located at the bottom of the control device 120. The imaging device 100 and the control device 120 are connected by a network 130. The network 130 is, for example, a wireless communication network.

[0015] Next, the hardware configuration of the imaging control system 10 will be described. Figure 2 is a block diagram showing the hardware configuration of the imaging device 100 and control device 120 that constitute the imaging control system 10 according to Embodiment 1 of the present invention.

[0016] First, the hardware configuration of the imaging device 100 will be described while referring to FIG. 2(A). FIG. 2(A) is a block diagram showing the hardware configuration of the imaging device 100. The imaging unit 211 captures an image. Specifically, the imaging unit 211 includes an imaging element such as a CCD sensor or a CMOS sensor, and photoelectrically converts the subject image formed through the lens of the imaging device 100 to generate an electrical signal. CCD is an abbreviation for Charge Coupled Device. CMOS is an abbreviation for Complementary Metal-Oxide-Semiconductor. Then, the imaging unit 211 generates an image through a process of converting the photoelectrically converted electrical signal into a predetermined digital signal. The imaging unit 211 constitutes the plurality of imaging units 105, 106, 107, and 108 shown in FIG. 1.

[0017] The pan-tilt-rotation drive unit 212 is composed of a mechanical drive system and a motor as a drive source capable of performing operations in the pan direction, tilt direction, and rotation operation regarding the shooting direction of the imaging device 100. The pan-tilt-rotation drive unit 212 controls the pan-tilt-rotation of the imaging device 100.

[0018] The lens drive unit 213 is composed of a drive system for the focus lens and the zoom lens, and controls the zoom and focus of the imaging device 100. The network I / F 214 communicates with the control device 120 via the network 130. I / F is an abbreviation for Interface. TCP / IP is an abbreviation for Transmission Control Protocol / Internet Protocol.

[0019] The CPU 215 is a central processing unit that overall controls the imaging device 100. CPU is an abbreviation for Central Processing Unit. The RAM 216 temporarily stores computer programs executed by the CPU 215. RAM is an abbreviation for Random Access Memory. Also, the RAM 216 provides a work area used when the CPU 215 executes processing. Further, the RAM 216 can function, for example, as a frame memory or as a buffer memory. The ROM 217 stores programs and the like for the CPU 215 to control the imaging device 100. ROM is an abbreviation for Read Only Memory. The HDD 218 is a storage device that records image data and the like. HDD is an abbreviation for Hard Disk Drive.

[0020] Next, the hardware configuration of the control device 120 will be described while referring to FIG. 2(B). FIG. 2(B) is a block diagram showing the hardware configuration of the control device 120. The network I / F 201 communicates with the imaging device 100 via the network 130. The input unit 202 has buttons, switches, a keyboard, or the like, and receives input from the user.

[0021] The CPU 203 is a central processing unit that overall controls the control device 120. The RAM 204 temporarily stores computer programs executed by the CPU 203. Also, the RAM 204 provides a work area used when the CPU 203 executes processing. Further, the RAM 204 can function, for example, as a frame memory or as a buffer memory. The ROM 205 stores programs and the like for the CPU 203 to control the control device 120. The HDD 206 is a storage device that records image data and the like.

[0022] The display 207 is composed of an LCD, liquid crystal panel, or organic EL panel, and displays images transmitted from the imaging device 100 and the UI screen described later. LCD is an abbreviation for Liquid Crystal Display. EL is an abbreviation for Electro Luminescence. The display 207 may also consist of a touch panel or the like that accepts user input in addition to displaying information to the user. In this case, the display 207 and the input unit 202 are configured as a single unit.

[0023] In Embodiment 1, an example is described in which the control device 120 is equipped with a display 207. However, the invention is not limited to this example, and the control device 120 and the display 207 may be housed in separate enclosures. In this case, the control device 120 and the display 207 are connected via cables such as HDMI® or SDI. HDMI is an abbreviation for High-Definition Multimedia Interface. SDI is an abbreviation for Serial Digital Interface.

[0024] Figure 3 is a block diagram showing the functional configuration of the imaging control system according to Embodiment 1 of the present invention. The description of Figure 3 will explain each component and its general function. Detailed processing will be explained later.

[0025] First, let's explain the configuration of the imaging device 100. The imaging device 100 includes a control unit 101, a pan drive unit 102, a tilt drive unit 103, a rotation drive unit 104, imaging units 105, 106, 107 and 108, an image processing unit 109, a storage unit 110, a communication unit 111, and a control bus 112.

[0026] The control unit 101 and the image processing unit 109 are configured as hardware or software within a computing device such as a microcomputer. Specifically, the control unit 101 and the image processing unit 109 are configured as shown in Figure 2(A) by the CPU 215, RAM 216, ROM 217, and HDD 218. The control unit 101 is connected to each component within the imaging device 100 by the control bus 112 and performs various calculations necessary for controlling each component and operating the imaging device 100.

[0027] The pan drive unit 102 is composed of a drive mechanism such as a motor and drives the imaging units 105-108 in the horizontal direction. The pan drive unit 102 also notifies the control unit 101 of the pan drive state. The tilt drive unit 103 is composed of a drive mechanism such as a motor and drives the imaging units 105-108 in the vertical direction. The tilt drive unit 103 also notifies the control unit 101 of the tilt drive state. The rotation drive unit 104 is composed of a drive mechanism such as a motor and rotates the optical system (not shown), such as the imaging sensors and lenses of the imaging units 105-108, around the optical axis. The rotation drive unit 104 also notifies the control unit 101 of the rotation drive state. The pan drive unit 102, tilt drive unit 103, and rotation drive unit 104 are composed of the pan-tilt-rotation drive unit 212 shown in Figure 2(A). In this embodiment, the rotation drive unit 104 is described as rotating the optical system within the imaging units 105-108, but it may also be configured to rotate the imaging units 105-108 themselves.

[0028] The imaging units 105-108 consist of optical systems such as imaging sensors and lenses, and circuits that control them. The imaging units 105-108 are driven horizontally, vertically, and around the optical axis by the pan drive unit 102, tilt drive unit 103, and rotation drive unit 104, respectively. The images captured by the imaging units 105-108 are output to the image processing unit 109.

[0029] The image processing unit 109 performs various image processing operations. Examples of operations performed by the image processing unit 109 include scaling, color gamut conversion, and brightness correction.

[0030] The memory unit 110 consists of RAM 216, ROM 217, and a storage device such as HDD 218 or SSD (Solid State Drive). The memory unit 110 holds various information necessary for the operation of the imaging device 100.

[0031] The communication unit 111 consists of terminals for communication standards such as USB, wireless LAN, or wired LAN, and their processing circuits. USB is an abbreviation for Universal Serial Bus. LAN is an abbreviation for Local Area Network. The communication unit 111 transmits and receives data with the communication unit 123 of the control device 120. The communication unit 111 consists of the network I / F 214, etc., as shown in Figure 2(A).

[0032] Next, the configuration of the control device 120 will be described. In this embodiment, the control device 120 will be described as hardware, but it may also be implemented as software on a device such as a PC or smartphone. PC is an abbreviation for Personal Computer. The control device 120 includes a control unit 121, a storage unit 122, a communication unit 123, an operation unit 124, a UI generation unit (display control means) 125, a display unit 126, and a control bus 127.

[0033] The control unit 121 and the UI generation unit 125 are composed of a computing device such as a microcomputer. Specifically, the control unit 121 and the UI generation unit 125 are composed of the CPU 203, RAM 204, ROM 205, and HDD 206 shown in Figure 2(B). The control unit 121 is connected to each component in the control device 120 by a control bus 127 and performs various calculations necessary for controlling each component and operating the control device 120.

[0034] The memory unit 122 consists of RAM 204, ROM 205, and a storage device such as HDD 206 or SSD. The memory unit 122 holds various information necessary for the operation of the control device 120.

[0035] The communication unit 123 consists of terminals for communication standards such as USB, wireless LAN, or wired LAN, and their processing circuits. The communication unit 123 transmits and receives data with the communication unit 111 of the imaging device 100. The communication unit 123 is composed of the network I / F 201, etc., as shown in Figure 2(B).

[0036] The operation unit 124 consists of operating devices such as buttons and switches, and their control circuits. The operation unit 124 is composed of the input unit 202 shown in Figure 2(B), etc. The operation unit 124 outputs information indicating user operations performed by the user to the UI generation unit 125.

[0037] The UI generation unit 125 generates a user interface (UI) for the user to operate the imaging device 100. The UI generation unit 125 updates the UI in response to user operations received from the operation unit 124 and outputs information indicating the user operations performed by the user to the communication unit 123. The UI generation unit 125 also receives the captured image taken by the imaging device 100 through the communication unit 123 and incorporates it into the UI. A specific example will be explained later. The UI generation unit 125 displays the generated UI image on the display unit 126. In this way, the UI generation unit 125 functions as a display control means.

[0038] The display unit 126 consists of an image display panel such as a liquid crystal panel or an organic EL panel, and their control circuits. The display unit 126 is composed of the display 207 shown in Figure 2(B), etc. The display unit 126 displays the UI image generated by the UI generation unit 125.

[0039] Figure 4 is a flowchart showing the processing procedure of the imaging control system according to Embodiment 1 of the present invention. Figure 5 is a diagram showing an example of the display of the imaging control system according to Embodiment 1 of the present invention, and is a diagram showing an example of the display of a UI image on the display unit 126. The processing procedure when a user operates the operation unit 124 and how the UI image changes at that time will be explained using Figures 4 and 5.

[0040] Flow control is performed by the control unit 121, which controls each component block. When the control unit 121 controls each component of the imaging device 100, a control command is sent from the communication unit 123 to the communication unit 111, which is received by the control unit 101, and the control command is output to the component that is actually being controlled. When the control unit 121 receives data from each component of the imaging device 100, after sending a command as described above, the control unit 101 receives the data from each component and outputs it to the communication unit 111, which sends it from the communication unit 111 to the communication unit 123, and then to the control unit 121. The intermediate control processes are omitted from here on.

[0041] Figure 5(A) shows an example of the UI display on the display unit 126 at startup. The background image 400, shown in gray, is the background of the image to be displayed. Display images 401, 402, 403, and 404 are images captured by the imaging units 105, 106, 107, and 108, respectively. Display image 401 shows a door, display image 402 shows a device, display image 403 shows a passageway, and display image 404 shows a shelf.

[0042] Camera selection UIs 405, 406, 407, and 408 are buttons for selecting the imaging unit to operate, and are indicated by circled numbers 1, 2, 3, and 4, respectively. The camera selection UI is an example of an imaging means selection UI. Camera selection UIs 405, 406, 407, and 408 correspond to imaging units 105, 106, 107, and 108, respectively. Camera selection UIs 405 to 408 are indicated by white circled numbers when not selected, and by black circled numbers when selected. Multiple camera selection UIs 405 to 408 can be selected simultaneously. In Figure 5(A), all camera selection UIs 405 to 408 are in a not-selected state, and all of them are indicated by white circled numbers. Camera selection UIs 405, 406, 407, and 408 are UIs for selecting the target imaging unit from among multiple imaging units to be operated by the pan / tilt operation UI 409 and the rotation operation UIs 410 and 411.

[0043] The Pan / Tilt Operation UI 409 is a UI for performing pan and tilt operations. The Pan / Tilt Operation UI 409 is an example of an operation UI for operating at least one of the pan drive unit 102 and the tilt drive unit 103. When the end of the arrow (triangular part) of the Pan / Tilt Operation UI 409 is selected, the imaging unit selected in the Camera Selection UI 405-408 is driven to pan or tilt in that direction. Hereafter, when referring to the direction of movement, the vertical upward direction will be considered as 0 degrees, the horizontal left direction as 90 degrees, the vertical downward direction as 180 degrees (= -180 degrees), and the horizontal right direction as -90 degrees.

[0044] Rotation operation UIs 410 and 411 are UIs for performing rotation operations that drive the imaging unit around the optical axis. When rotation operation UI 410 is selected, the imaging unit selected by camera selection UIs 405 to 408 is rotated counterclockwise, and when rotation operation UI 411 is selected, it is rotated clockwise. In this embodiment, the rotation operation can be performed counterclockwise from 0 to 90 degrees. 0 degrees is when the imaging sensors in imaging units 105 to 108 are horizontal in the horizontal direction, and 90 degrees is when they are vertical in the vertical direction.

[0045] In Figure 5(A), the rotation drive states of the imaging units 105, 106, 107, and 108 are assumed to be 60 degrees, 0 degrees, 30 degrees, and 0 degrees, respectively. The imaging units 105 and 107 have been pre-rotated so that display images 401 and 403 appear facing each other. In other words, the imaging units 105 and 107 have been pre-rotated to 60 degrees, 0 degrees, and 30 degrees, respectively, so that display images 401 and 403 appear upright, similar to display images 402 and 404. As a result, the captured images are displayed as they are without rotation, but all display images 401 to 404 are upright.

[0046] Let's return to Figure 4 for explanation. At the start, the user selects the imaging unit they want to operate using the camera selection UI 405-408. In step S301, the control unit 121 determines the number of imaging units selected by the user. If the control unit 121 determines that the number of imaging units selected by the user is 0 or 1, the process in step S302 is executed. If the control unit 121 determines that the number of imaging units selected by the user is 2 or more, the process in step S303 is executed.

[0047] In step S302, the control unit 121 obtains the rotation drive state of the selected imaging unit from the rotation drive unit 104. The UI generation unit 125 then rotates the pan / tilt operation UI 409 according to the rotation drive state from the control unit 121.

[0048] For example, if the user selects the imaging unit 107, the UI display example will be as shown in Figure 5(B). Since the imaging unit 107 is selected, the camera selection UI 407 is displayed with a number enclosed in a black circle. And since the rotation drive state of the imaging unit 107 is 30 degrees, the pan / tilt operation UI 409 is displayed rotated by 30 degrees. When the user performs a tilt operation with the pan / tilt operation UI 409 in this state, the displayed image 403 moves in the same direction as the tilt of the pan / tilt operation UI 409, by 30 degrees.

[0049] If the user does not select any imaging units, the UI display will look like Figure 5(A). Since no imaging units are selected, the pan / tilt operation UI409 will be in a forward-facing position (0-degree rotation).

[0050] In step S303, the control unit 121 obtains the rotation drive status of the selected imaging units from the rotation drive unit 104. The UI generation unit 125 then rotates the display images 401 to 404 corresponding to the selected images according to the rotation drive status from the control unit 121. At this time, the pan / tilt operation UI 409 is set to a frontal orientation (0-degree rotation).

[0051] For example, if the user selects imaging unit 105 and imaging unit 107, the UI display example will be as shown in Figure 5(C). Since imaging unit 105 and imaging unit 107 are selected, the camera selection UI 405 and 407 are displayed as numbers enclosed in black circles. The rotation drive states of imaging unit 105 and imaging unit 107 are 60 degrees and 30 degrees, respectively. Therefore, the display images 401 and 403, which are images captured by imaging unit 105 and imaging unit 107, are displayed rotated by -60 degrees and -30 degrees, which is the opposite of the rotation drive state. At this time, display images 401 and 403 are enlarged or reduced so as not to overlap with adjacent display images or to exceed the area of ​​background image 400. If the user performs a tilt operation in this state, both display images 401 and 403 move in the 0-degree direction, similar to the tilt of the pan / tilt operation UI 409. The process in step S303 is an example of a control means that enlarges or reduces the size of the display area when rotating the display image.

[0052] In step S304, the control unit 121 acquires captured images from the imaging units 105-108 via the communication unit 123 and outputs them to the UI generation unit 125. The UI generation unit 125 updates the display images 401-404 using the captured images from the control unit 121.

[0053] In step S305, the control unit 121 determines whether or not a user operation has been performed. If the control unit 121 determines that a user operation has been performed, the process in step S306 is executed. If the control unit 121 determines that no user operation has been performed, the process in step S304 is executed.

[0054] In steps S306, S307, and S308, the control unit 121 branches the control according to the user operation performed. If the user operation was a camera selection operation (Yes in step S306), the process in step S301 is executed. If the user operation was a rotation operation (Yes in step S307), the process in step S309 is executed. If the user operation was a pan / tilt operation (Yes in step S308), the process in step S310 is executed. If the user operation was none of the above (No in step S308), the process in step S304 is executed.

[0055] In step S309, the control unit 121 causes the communication unit 123 to send a rotation drive command. The communication unit 111 receives the transmitted rotation drive command. The control unit 101 controls the rotation drive unit 104 based on the received rotation drive command. After the processing in step S309, the processing in step S304 is executed.

[0056] In step S310, the control unit 121 causes the communication unit 123 to send pan drive commands and tilt drive commands. The communication unit 111 receives the transmitted pan drive commands and tilt drive commands. The control unit 101 controls the pan drive unit 102 or the tilt drive unit 103 based on the received pan drive commands and tilt drive commands. After the processing in step S310, the processing in step S304 is executed.

[0057] The display image rotated in steps S302 and S303 and updated in step S304 may be displayed as shown in Figure 6, for example. Figure 6 is a different display example of the display image, which is an image captured by the imaging unit 107, from the display image 403 in Figure 5(C). In the example in Figure 6, a cross corresponding to the pan / tilt operation UI and circles indicating the pan drive state and tilt drive state are displayed transparently on top of the display image, which is an image captured by the imaging unit 107. In this example, the pan drive state is -3 and the tilt drive state is +4. There is a range in which the pan drive and tilt drive can be driven. By performing this transparent display, it is possible to show how far the selected imaging unit can be driven in each direction. The processing in the example in Figure 6 is an example of a control means that superimposes an image on the display image that represents at least one of the pan drive state of the plurality of imaging means by the pan drive means and the tilt drive state of the plurality of imaging means by the tilt drive means.

[0058] Furthermore, in this embodiment, the object to be rotated and displayed is different in step S302 and step S303. The present invention is not limited to this, and in step S302 as in step S303, the display images 401 to 404 corresponding to the selected image may be rotated so that the pan / tilt operation UI 409 displays in a frontal orientation.

[0059] As described above, according to this embodiment, the orientation of the display images or pan / tilt operation UI of multiple imaging units can be rotated and aligned according to the rotation drive state. This makes it possible for the user to intuitively perform pan and tilt operations.

[0060] (Embodiment 2) In Embodiment 1, the displayed image or pan-tilt operation UI was rotated according to the number of selected imaging units and the rotation drive state. In Embodiment 2 of the present invention, the displayed image or pan-tilt operation UI is rotated according to the number of selected imaging units, the order in which the imaging units were selected, and the rotation drive state. The configuration of the imaging control system in this embodiment is the same as that of Embodiment 1. The processing procedure of the imaging control system in this embodiment is also the same as that of Embodiment 1, except for a few parts, so only the differences will be explained.

[0061] In step S301 of Figure 4, the control unit 121 stores the order in which the user selected the imaging units. If the user deselects an imaging unit, the order of selection of the imaging units at that time is restored.

[0062] The processing in step S302 is equivalent to that in Embodiment 1. For example, if the imaging unit 107 is selected first, the display example will be as shown in Figure 5(B).

[0063] In step S303, the control unit 121 obtains the rotation drive states of the selected imaging units from the rotation drive unit 104. The control unit 121 outputs the rotation drive states of the imaging units to the UI generation unit 125. The UI generation unit 125 orients the display image corresponding to the first selected imaging unit and rotates the pan / tilt operation UI 409 according to the rotation drive state of that imaging unit. The UI generation unit 125 also rotates the display images corresponding to the second and subsequent selected imaging units according to the rotation drive state of the first selected imaging unit and the rotation drive state of the second and subsequent selected imaging units.

[0064] For example, if the imaging unit 107 is selected first and the imaging unit 105 is selected second, the display example will be as shown in Figure 7. The display image 403 corresponding to the imaging unit 107, which was selected first, is displayed facing the viewer. Since the rotation drive state of the imaging unit 107 is 30 degrees, the pan-tilt operation UI 409 is displayed rotated by 30 degrees. Since the rotation drive state of the imaging unit 105 is 60 degrees, the display image 401 is displayed rotated by 30-60=-30 degrees to match the rotation of the pan-tilt operation UI 409. This makes it possible to match the direction of operation with the direction of movement of the display images 401 and 403 when the pan-tilt operation UI 409 is operated.

[0065] As described above, in the imaging control system according to this embodiment, the displayed image or pan / tilt operation UI is rotated according to the number of selected imaging units, the order in which the imaging units were selected, and the rotation drive state. This allows the user to add and operate imaging units while keeping the displayed image of the imaging unit that was initially selected and operated facing the correct direction.

[0066] (Embodiment 3) In Embodiment 1, the displayed image was in landscape orientation when the imaging unit was not selected. Embodiment 3 of the present invention describes the case where landscape and portrait orientations are mixed when the imaging unit is not selected. The configuration of the imaging control system according to this embodiment is the same as that of Embodiment 1. The processing procedure of the imaging control system in this embodiment is also the same as that of Embodiment 1, except for a few parts, so only the differences will be described.

[0067] At the start of the flowchart in Figure 4, the rotation drive states of the imaging units 105, 106, 107, and 108 are assumed to be 90 degrees, 0 degrees, 90 degrees, and 30 degrees, respectively. An example of the UI display on the display unit 126 at the start of the flowchart is shown in Figure 8(A). In Figure 8(A), display images 401 and 403 are displayed in portrait orientation, while display images 402 and 404 are displayed in landscape orientation. The portrait orientation is obtained by rotating the landscape orientation image by 90 degrees. By using portrait orientation, the image captured by rotating the imaging unit to widen the field of view in the vertical direction can be displayed vertically.

[0068] In step S302, the UI generation unit 125 rotates the pan / tilt operation UI 409 according to the rotation drive state of the selected imaging unit and the display state of the displayed image. For example, if the user selects the imaging unit 105, the UI display example will be as shown in Figure 8(B). Since the rotation drive state of the imaging unit 105 is 90 degrees, the pan / tilt operation UI 409 is displayed rotated 90 degrees. When the user performs a tilt operation in this state, the displayed image 403 moves in the same direction as the tilt of the pan / tilt operation UI 409, by 90 degrees.

[0069] In step S303, the UI generation unit 125 rotates the display images 401-404 corresponding to the selected image, according to the rotation drive state of the selected imaging unit and the predominance of vertical and horizontal display images. If the same number of vertical and horizontal displays are selected, it is assumed that the horizontal display is more predominant. The UI generation unit 125 rotates the pan / tilt operation UI 409 according to the predominance of vertical and horizontal display images. If the horizontal display is more predominant, the vertical direction becomes the tilt operation, and if the vertical display is more predominant, the vertical direction becomes the pan operation. For example, if the user selects imaging units 105, 106, and 107, the UI display example will be as shown in Figure 8(C). Since the majority of the display images are vertical, the pan / tilt operation UI 409 rotates 90 degrees so that the vertical direction becomes the pan operation. Then, the display image is rotated according to the orientation of the pan / tilt operation UI 409 and the rotation drive state of the selected imaging unit. The rotation drive state of imaging units 105 and 107 is 90 degrees. Since the images have already been rotated 90 degrees by being displayed in portrait mode, display images 401 and 403 are displayed rotated 90-90=0 degrees. The rotation drive state of the imaging unit 108 is 30 degrees. Display image 404 is displayed rotated 90-30=60 degrees.

[0070] As described above, according to this embodiment, the displayed image corresponding to the selected image is rotated depending on whether portrait or landscape orientation is more prevalent. This allows the user to operate the image in a orientation that is more intuitively understandable to them, whether in portrait or landscape orientation.

[0071] In Embodiments 1, 2, and 3 described above, the user selected the imaging unit to be operated by the user, but this can be done automatically. For example, in Embodiment 3, if the user selects one imaging unit corresponding to a vertically oriented display image, all imaging units corresponding to vertically oriented display images can be automatically selected as the target for operation. Also, if imaging units are panned to adjacent positions, if the user selects one of them, all adjacent imaging units can be automatically selected.

[0072] (Other embodiments) The present invention can also be realized by a process in which one or more processors read and execute a program that implements one or more of the functions of the above embodiments. The program may be supplied to a system or device having a processor via a network or storage medium. Furthermore, the present invention can also be realized by a circuit such as an ASIC (Application Specific Integrated Circuit) that implements one or more of the functions of the above embodiments.

[0073] Furthermore, while embodiments of the present invention have been described, the embodiments described above are merely examples of concrete implementations of the present invention, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various forms without departing from its technical concept or its main features. For example, combinations of the embodiments are also included in the disclosure of this specification.

[0074] Preferred embodiments of the present invention have been described above. However, the present invention is not limited to the embodiments described above. That is, the present invention includes embodiments that have been modified in accordance with the spirit of the present invention, and these embodiments are not excluded from the scope of the present invention.

[0075] This embodiment includes the following configurations, methods, and programs. (Composition 1) A control device for controlling an imaging apparatus comprising: a plurality of imaging means; a pan driving means for panning the plurality of imaging means; a tilt driving means for tilting the plurality of imaging means; and a rotation driving means for rotating the plurality of imaging means around the optical axis, The display control means displays a plurality of captured images captured by the plurality of imaging means and an operating UI for operating at least one of the pan drive means and the tilt drive means on a display means. The display control means rotates at least one of the captured images included in the plurality of captured images and the operation UI, in accordance with the rotation drive state by the rotation drive means in at least one of the selected imaging means among the plurality of imaging means. A control device characterized by the following features. (Configuration 2) The display control means causes the display means to display an imaging means selection UI for selecting the target imaging means to be operated on by the operation UI from among the plurality of imaging means, The display control means controls the operation UI to rotate according to the rotation drive state of the operation target imaging means by the rotation drive means, if there is only one target imaging means selected by the imaging means selection UI. The control device according to configuration 1, characterized by the above. (Composition 3) The display control means causes the display means to display an imaging means selection UI for selecting the target imaging means to be operated on by the operation UI from among the plurality of imaging means, If there are multiple target imaging means selected in the imaging means selection UI, the display control means controls the rotation of the image captured by the target imaging means according to the rotation drive state of the target imaging means by the rotation drive means. A control device according to configuration 1 or configuration 2, characterized by the above. (Composition 4) The display control means enlarges or reduces the image to match the size of the display area when the captured image is rotated. The control device according to configuration 3, characterized by the above. (Composition 5) The display control means superimposes on the captured image an image representing at least one of the pan drive state of the plurality of imaging means by the pan drive means and the tilt drive state of the plurality of imaging means by the tilt drive means. A control device according to any one of configurations 1 to 4, characterized by the above. (Composition 6) The display control means causes the display means to display an imaging means selection UI for selecting the target imaging means to be operated on by the operation UI from among the plurality of imaging means, If there are multiple target imaging means selected in the imaging means selection UI, the display control means controls the rotation of the operation UI according to the rotation drive state of the first selected target imaging means. A control device according to any one of configurations 1 to 5, characterized by the above. (Composition 7) The display control means causes the display means to display the captured image in which portrait and landscape orientations are mixed. The display control means controls the rotation of the captured image according to the orientation of the more numerous of the captured images, which are displayed vertically or horizontally, and the rotation drive state of the imaging means corresponding to the more numerous captured image. A control device according to any one of configurations 1 to 6, characterized by the above. (Composition 8) The display control means causes the display means to display an imaging means selection UI for selecting the target imaging means to be operated on by the operation UI from among the plurality of imaging means, The display control means is If the image captured by the target imaging means selected in the imaging means selection UI is displayed in landscape orientation, then the imaging means corresponding to the other images displayed in landscape orientation will be used as the target for operation in the operation UI. If the image captured by the target imaging means selected in the imaging means selection UI is displayed in portrait orientation, then the imaging means corresponding to the other images displayed in portrait orientation will be used as the target for operation in the operation UI. The control device according to configuration 7, characterized by the features described above. (Composition 9) The display control means causes the display means to display an imaging means selection UI for selecting the target imaging means to be operated on by the operation UI from among the plurality of imaging means, If there is an imaging device with a similar pan drive state to the imaging device selected in the imaging device selection UI, the display control means will also make the imaging device with the similar pan drive state a target for operation using the operation UI. A control device according to any one of configurations 1 to 8, characterized by the above. (Method 1) A control method for controlling an imaging device comprising: a plurality of imaging means; a pan driving means for panning the plurality of imaging means; a tilt driving means for tilting the plurality of imaging means; and a rotation driving means for rotating the plurality of imaging means around the optical axis, The system includes a display control step of displaying a plurality of captured images captured by the plurality of imaging means and an operating UI for operating at least one of the pan drive means and the tilt drive means on a display means. The display control step rotates at least one of the captured images included in the plurality of captured images and the operation UI, in accordance with the rotation drive state by the rotation drive means in at least one of the selected imaging means among the plurality of imaging means. A control method characterized by comprising: (Program 1) A program for controlling an imaging device comprising: a plurality of imaging means; a pan driving means for panning the plurality of imaging means; a tilt driving means for tilting the plurality of imaging means; and a rotation driving means for rotating the plurality of imaging means around the optical axis, Computers, Display control means for displaying a plurality of captured images captured by the plurality of imaging means and an operating UI for operating at least one of the pan drive means and the tilt drive means on a display means. To make it function as, The display control means rotates at least one of the captured images included in the plurality of captured images and the operation UI, in accordance with the rotation drive state by the rotation drive means in at least one of the selected imaging means among the plurality of imaging means. A program characterized by the following features. [Explanation of Symbols]

[0076] 100 Imaging device 101 Control Unit 102 Pan drive unit 103 Tilt drive unit 104 Rotation drive unit 105-108 Imaging section 109 Image Processing Unit 110 Storage section 111 Communications Department 120 Control device 121 Control Unit 122 Storage section 123 Communications Department 124 Operation section 125 UI generation section 126 Display section

Claims

1. A control device for controlling an imaging apparatus comprising: a plurality of imaging means; a pan driving means for panning the plurality of imaging means; a tilt driving means for tilting the plurality of imaging means; and a rotation driving means for rotating the plurality of imaging means around the optical axis, The display control means displays a plurality of captured images captured by the plurality of imaging means and an operating UI for operating at least one of the pan drive means and the tilt drive means on a display means. The display control means rotates at least one of the captured images included in the plurality of captured images and the operation UI, in accordance with the rotation drive state by the rotation drive means in at least one of the selected imaging means among the plurality of imaging means. A control device characterized by the following features.

2. The display control means causes the display means to display an imaging means selection UI for selecting the target imaging means to be operated by the operation UI from among the plurality of imaging means, The display control means controls the operation UI to rotate according to the rotation drive state of the operation target imaging means by the rotation drive means when one operation target imaging means is selected by the imaging means selection UI. The control device according to feature 1.

3. The display control means causes the display means to display an imaging means selection UI for selecting the target imaging means to be operated by the operation UI from among the plurality of imaging means, If there are multiple target imaging means selected by the imaging means selection UI, the display control means controls the rotation of the image captured by the target imaging means according to the rotation drive state of the target imaging means by the rotation drive means. The control device according to feature 1.

4. The display control means enlarges or reduces the image to match the size of the display area when the captured image is rotated. The control device according to claim 3.

5. The display control means superimposes on the captured image an image representing at least one of the pan drive state of the plurality of imaging means by the pan drive means and the tilt drive state of the plurality of imaging means by the tilt drive means. The control device according to feature 1.

6. The display control means causes the display means to display an imaging means selection UI for selecting the target imaging means to be operated by the operation UI from among the plurality of imaging means, If there are multiple target imaging means selected by the imaging means selection UI, the display control means controls the rotation of the operation UI according to the rotation drive state of the initially selected target imaging means. The control device according to feature 1.

7. The display control means causes the display means to display the captured image in which portrait and landscape orientations are mixed. The display control means controls the rotation of the captured image according to the orientation of the more numerous of the captured images, which are displayed vertically or horizontally, and the rotation drive state of the imaging means corresponding to the more numerous captured image. The control device according to feature 1.

8. The display control means causes the display means to display an imaging means selection UI for selecting the target imaging means to be operated by the operation UI from among the plurality of imaging means, The display control means is If the image captured by the target imaging means selected in the imaging means selection UI is displayed in landscape orientation, then the imaging means corresponding to the other images displayed in landscape orientation will be used as the target for operation in the operation UI. If the image captured by the target imaging means selected in the imaging means selection UI is displayed in portrait orientation, then the imaging means corresponding to other images displayed in portrait orientation will be used as the target for operation in the operation UI. The control device according to feature 7.

9. The display control means causes the display means to display an imaging means selection UI for selecting the target imaging means to be operated by the operation UI from among the plurality of imaging means, If there is an imaging device whose pan drive state is similar to the imaging device selected by the imaging device selection UI, the display control means will also make the imaging device with the similar pan drive state a target for operation using the operation UI. The control device according to feature 1.

10. A control method for controlling an imaging device comprising: a plurality of imaging means; a pan driving means for panning the plurality of imaging means; a tilt driving means for tilting the plurality of imaging means; and a rotation driving means for rotating the plurality of imaging means around the optical axis, The system includes a display control step that causes a plurality of captured images captured by the plurality of imaging means and an operating UI for operating at least one of the pan drive means and the tilt drive means to be displayed on a display means. The display control step rotates at least one of the captured images included in the plurality of captured images and the operation UI, in accordance with the rotation drive state by the rotation drive means in at least one of the selected imaging means among the plurality of imaging means. A control method characterized by comprising:

11. A program for controlling an imaging device comprising: a plurality of imaging means; a pan driving means for panning the plurality of imaging means; a tilt driving means for tilting the plurality of imaging means; and a rotation driving means for rotating the plurality of imaging means around the optical axis, Computers, Display control means for displaying a plurality of captured images captured by the plurality of imaging means and an operating UI for operating at least one of the pan drive means and the tilt drive means. To make it function as, The display control means rotates at least one of the captured images included in the plurality of captured images and the operation UI, in accordance with the rotation drive state by the rotation drive means in at least one of the selected imaging means among the plurality of imaging means. A program characterized by the following features.

Images