Control device, control method for the same, and program

Abstract

To make it possible to easily grasp the relationship between an image on which an object is detected and other images.SOLUTION: A control device has a detection unit configured to detect objects from each image captured by multiple imaging units and a control unit configured to display, on a display unit, an image in which the object was detected and an image captured by an imaging unit adjacent to the imaging unit that captured the image in which the object was detected and to superimpose the image related to the object onto the image of the adjacent imaging unit.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to a control device, a control method for a control device, and a program. [Background technology]

[0002] To enable a single camera to capture images in multiple directions, there are cameras (hereinafter referred to as multi-lens cameras) that are configured with multiple image capturing units. Among multi-lens cameras, there are imaging devices that have a drive mechanism that enables each image capturing unit to be independently driven on a common drive axis, such as on the same circumference. A display device in an imaging system that includes this imaging device displays images captured by each image capturing unit side by side (Patent Document 1). Furthermore, imaging devices that have the function of detecting moving objects and other objects are also known. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-148984 Summary of the Invention [Problem to be solved by the invention]

[0004] In the display of the display device of Patent Document 1, when a moving object is detected by one of the imaging units, it is difficult to see the relationship between the moving object and imaging units other than the imaging unit that detected the moving object.

[0005] An object of the present disclosure is to make it possible to easily understand the relationship between an image in which an object is detected and other images. [Means for solving the problem]

[0006] The control device has a detection unit that detects an object from each image captured by a plurality of imaging units, and a control unit that controls a display unit to display an image in which the object is detected and an image from an imaging unit adjacent to the imaging unit that captured the image in which the object is detected, and to superimpose an image related to the object on the image from the adjacent imaging unit. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to easily understand the relationship between an image in which an object is detected and other images. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an imaging system. [Figure 2] FIG. 1 is a diagram illustrating an imaging device. [Figure 3] FIG. 2 is a side view of the imaging device. [Figure 4] 10 is a flowchart for detecting an object. [Figure 5] FIG. 10 is a diagram showing a display example of an OSD display. [Figure 6] FIG. [Figure 7] FIG. 10 is a diagram showing a photographing area after pan / tilt driving. [Figure 8] FIG. 10 is a diagram showing an example of an OSD display. [Figure 9] FIG. 10 is a diagram illustrating an example of complementary photography. [Figure 10] FIG. [Figure 11] FIG. 10 is a diagram showing a display example in which images are shifted one by one. [Figure 12] FIG. 10 is a diagram showing a display example in which the positions of images are exchanged. [Figure 13] FIG. 10 is a diagram showing a display example in which images from adjacent cameras are arranged. [Figure 14] FIG. 10 is a diagram showing an example of a display layout for vertical shooting. [Figure 15] 10A and 10B are diagrams showing examples of display of a rearrangement method and a driving method of an adjacent camera. [Figure 16] 10 is a flowchart illustrating how to change the layout of an image. [Figure 17] FIG. 10 is a diagram showing an example in which the subject has moved. [Figure 18] FIG. 1 is a diagram illustrating an example of the configuration of an imaging system. [Figure 19] FIG. 1 is a diagram illustrating an example of the configuration of an imaging system. [Figure 20] 10 is a flowchart for detecting an object. DETAILED DESCRIPTION OF THE INVENTION

[0009] Preferred embodiments will be described in detail below with reference to the drawings. Note that the following embodiments do not limit the scope of the claims. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential, and multiple features may be combined arbitrarily. Furthermore, in the drawings, identical or similar components are designated by reference numerals, and redundant explanations will be omitted.

[0010] (First embodiment) <Imaging system> The configuration of an imaging system 190 according to the first embodiment will be described below with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an imaging system 190 according to the first embodiment. The imaging system 190 includes an imaging device 100 and a client device 200.

[0011] The imaging device 100 has four imaging units 110, 120, 130, and 140, independent driving units 113, 123, 133, and 143 for each imaging unit, an image processing unit 151, a control unit 152, a communication unit 153, and a recording unit 154.

[0012] Specifically, the imaging device 100 has a first imaging section 110, a second imaging section 120, a third imaging section 130, and a fourth imaging section 140. The imaging device 100 also has a first driving section 113, a second driving section 123, a third driving section 133, and a fourth driving section 143.

[0013] In the imaging system 190, the imaging device 100 is connected to a client device 200 via a network 170, and is capable of transmitting image data captured by the imaging device 100 and receiving control signals for the imaging device 100.

[0014] <Image capture unit> The first imaging section 110 has an imaging optical system 111 and a solid-state imaging element 112. The second imaging section 120 has an imaging optical system 121 and a solid-state imaging element 122. The third imaging section 130 has an imaging optical system 131 and a solid-state imaging element 132. The fourth imaging section 140 has an imaging optical system 141 and a solid-state imaging element 142.

[0015] Light transmitted through the imaging optical systems 111, 121, 131, and 141 forms images on the solid-state imaging elements 112, 122, 132, and 142, respectively, and is converted into electrical signals, which are then output as image data through processing in the image processing unit 151. The driving of the solid-state imaging elements 112, 122, 132, and 142 and the signal readout are controlled by the control unit 152.

[0016] The first driver 113 has a first pan driver 114 , a first zoom driver 115 , a first tilt driver 116 , a first rotation driver 117 , and a first focus driver 118 .

[0017] The second driver 123 has a second pan driver 124 , a second zoom driver 125 , a second tilt driver 126 , a second rotation driver 127 , and a second focus driver 128 .

[0018] The third driver 133 has a third pan driver 134 , a third zoom driver 135 , a third tilt driver 136 , a third rotation driver 137 , and a third focus driver 138 .

[0019] The fourth driver 143 has a fourth pan driver 144 , a fourth zoom driver 145 , a fourth tilt driver 146 , a fourth rotation driver 147 , and a fourth focus driver 148 .

[0020] The imaging units 110, 120, 130, and 140 each have a zoom lens that can drive the imaging optical systems 111, 121, 131, and 141 in the optical axis direction. The control unit 152 controls the first zoom driving unit 115, the second zoom driving unit 125, the third zoom driving unit 135, and the fourth zoom driving unit 145, thereby making it possible to control the shooting range of the imaging units 110, 120, 130, and 140.

[0021] The imaging units 110, 120, 130, and 140 each have a focus lens that can be driven in the optical axis direction by the imaging optical systems 111, 121, 131, and 141. The control unit 152 controls the first focus drive unit 118, the second focus drive unit 128, the third focus drive unit 138, and the fourth focus drive unit 148, thereby enabling the focus of the imaging units 110, 120, 130, and 140 to be controlled.

[0022] Furthermore, the imaging units 110, 120, 130, and 140 each have a rotation driver 117, 127, 137, and 147 that can rotate the solid-state imaging elements 112, 122, 132, and 142 about a rotation axis in the optical axis direction. The control unit 152 controls the first rotation driver 117, the second rotation driver 127, the third rotation driver 137, and the fourth rotation driver 147, thereby controlling the imaging angles of the imaging units 110, 120, 130, and 140.

[0023] <Pan / tilt drive unit> The driving units 113, 123, 133, and 143 have pan driving units 114, 124, 134, and 144 and tilt driving units 116, 126, 136, and 146. The pan driving units 114, 124, 134, and 144 and tilt driving units 116, 126, 136, and 146 are driving units for changing the shooting direction.

[0024] The first drive unit 113, the second drive unit 123, the third drive unit 133, and the fourth drive unit 143 each have a first pan drive unit 114, a second pan drive unit 124, a third pan drive unit 134, and a fourth pan drive unit 144. Similarly, the first drive unit 113, the second drive unit 123, the third drive unit 133, and the fourth drive unit 143 each have a first tilt drive unit 116, a second tilt drive unit 126, a third tilt drive unit 136, and a fourth tilt drive unit 146, respectively.

[0025] The first pan driving unit 114, the second pan driving unit 124, the third pan driving unit 134, and the fourth pan driving unit 144 have the same rotation axis, and the imaging units are arranged on the same circumference and rotate on the same circumference. This will be specifically described with reference to FIG. 2.

[0026] 2 is a diagram showing the imaging device 100 according to the first embodiment. Hereinafter, to simplify the explanation, the first imaging unit 110 will be used as a representative for the overlapping contents of the imaging units (first imaging unit 110, second imaging unit 120, third imaging unit 130, and fourth imaging unit 140), and explanations of the other imaging units will be omitted. Similarly, the driving units will be described using driving unit 113 as a representative, and explanations of the other driving units will be omitted. Differences between the imaging units will be clearly stated.

[0027] 2 is a structural diagram of the imaging device 100 as seen from above (the +Z axis side). Each pan driving unit 114, 124, 134, and 144 includes a motor and a gear, and drives each imaging unit 110, 120, 130, and 140 by controlling the power that drives the motor.

[0028] The pan driving units 114, 124, 134, and 144 are configured to be rotatable around a shaft 101. A circumference 102 indicates a common circumference that can drive the imaging units 110, 120, 130, and 140 around the shaft 101. The power that drives the motors is controlled by a control unit 152.

[0029] Furthermore, the imaging device 100 is capable of independently driving each of the imaging units 110, 120, 130, and 140, and is also capable of simultaneously independently driving one or more pan driving units 114, 124, 134, and 144. Furthermore, because all of the imaging units 110, 120, 130, and 140 are driven on the same circumference 102, the relative positions of the imaging units 110, 120, 130, and 140 do not change. The multiple imaging units 110, 120, 130, and 140 are arranged on the same circumference 102, and the pan angles on the same circumference 102 can be changed independently.

[0030] The tilt drivers 116, 126, 136, and 146 will be described with reference to Fig. 3. Fig. 3 is a side view (negative X-axis side) of the imaging device 100 according to the first embodiment. Each of the tilt drivers 116, 126, 136, and 146 includes a motor and a gear, and drives each of the imaging units 110, 120, 130, and 140 by controlling the power that drives the motor.

[0031] Tilt drive unit 116 is configured to be rotatable around shaft 103. The power that drives the motor is controlled by control unit 152. Furthermore, imaging device 100 is capable of independently driving each of imaging units 110, 120, 130, and 140, and is also capable of simultaneously and independently driving one or more tilt drive units 116, 126, 136, and 146.

[0032] The dome 104 is made of a transparent material such as plastic or glass, and enables the imaging unit 110 to capture an image of the outer periphery of the imaging device 100. The fixed unit 105 is a fixed member that can be attached to a ceiling, floor, or wall.

[0033] Furthermore, unlike the pan driving units 114, 124, 134, and 144, the tilt driving units 116, 126, 136, and 146 do not physically interfere with the imaging units 110, 120, 130, and 140. Similarly, the zoom driving units 115, 125, 135, and 145, the focus driving units 118, 128, 138, and 148, and the rotation driving units 117, 127, 137, and 147 do not physically interfere with the imaging units 110, 120, 130, and 140.

[0034] <Zoom drive unit / Focus drive unit> Each of the zoom drivers 115, 125, 135, and 145 includes a motor and a gear, and by controlling the power that drives the motor, it is possible to drive the zoom lens of each of the imaging optical systems 111, 121, 131, and 141. Furthermore, the position of the zoom lens can be obtained using a photointerrupter, a hall element, or the like.

[0035] Driving the zoom driving units 115, 125, 135, and 145 makes it possible to change the angle of view (zoom magnification) of the imaging units 110, 120, 130, and 140. The power for driving the motors is controlled by the control unit 152. Furthermore, the imaging device 100 can drive each of the imaging units 110, 120, 130, and 140 independently, and can drive one or more of the zoom driving units 115, 125, 135, and 145 simultaneously.

[0036] Each focus driver 118, 128, 138, and 148 includes a motor and a gear, and can drive the focus lens of each imaging optical system 111, 121, 131, and 141 by controlling the power that drives the motor. Furthermore, the position of the focus lens can be acquired using a photointerrupter, a Hall element, or the like. Driving the focus drivers 118, 128, 138, and 148 can change the focus positions (in-focus positions) of the imaging units 110, 120, 130, and 140. The power that drives the motor is controlled by a control unit 152. Furthermore, the imaging device 100 can independently drive each imaging unit 110, 120, 130, and 140, and can simultaneously drive one or more focus drivers 118, 128, 138, and 148.

[0037] <Rotation drive unit> Each rotation drive unit 117, 127, 137, and 147 includes a motor and gears, and can be independently controlled for each of the imaging units 110, 120, 130, and 140 by controlling the power that drives the motor. By tilting the rotation drive units 117, 127, 137, and 147 by 90 degrees around the optical axis direction, it is possible to change the imaging area from a landscape aspect ratio to a portrait aspect ratio. Furthermore, the rotation drive units 117, 127, 137, and 147 can rotate not only by 90 degrees but also within a range of 0 degrees to 360 degrees. The following description will be given with 0 degrees as the reference, where 0 degrees represents the landscape aspect ratio.

[0038] <Communications Department> The communication unit 153 transfers the image sent from the image processing unit 151 to the client device 200 via a wired or wireless network 170. The communication unit 153 also receives instructions from the client device 200.

[0039] <Control unit> As described above, the control unit 152 includes a CPU (Central Processing Unit) that controls the imaging units 110, 120, 130, and 140, the driving units 113, 123, 133, and 143, the image processing unit 151, and the communication unit 153. The control unit 152 performs overall control of the imaging device 100.

[0040] <Recording Department> The recording unit 154 has a RAM (Random Access Memory) and a ROM (Read Only Memory), and temporarily stores computer programs and stores programs for the control unit 152 to control the imaging device 100.

[0041] <Client device> The client device 200 includes a communication unit 201 , a control unit 202 , a display unit 203 , an instruction unit 204 , and a recording unit 205 .

[0042] A communication unit 201 of the client device 200 can communicate with the image capturing device 100 via a network 170 .

[0043] The display unit 203 is a display device such as a display, and can display the display image transmitted from the imaging device 100.

[0044] The instruction unit 204 has a user interface, accepts operations of a mouse (pointing device) or keyboard performed by a user, and generates a control signal for controlling the imaging device 100 by the control unit 202. The control signal controls, for example, each of the driving units 113, 123, 133, and 143.

[0045] That is, the user can control the pan / tilt drive units and zoom magnification of each of the image capture units 110, 120, 130, and 140 from the client device 200 via the network 170. Also, the user can control the image capture device 100 as described above by operating a graphical user interface (GUI) in the displayed image via a mouse. Here, a mouse is used as an example, but other means such as touch panel operation may also be used.

[0046] The client device 200 is, for example, a device such as a personal computer, and the network 170 is configured with a wired LAN, a wireless LAN, etc. The client device 200 may also be configured to supply power to the imaging device 100 via the network 170.

[0047] The control unit 202 includes the functions of a CPU and performs overall control of the client device 200. The recording unit 205 is configured with RAM and ROM, and temporarily stores computer programs and stores programs used by the control unit 202 to control the client device 200.

[0048] <Image processing unit> The image processing unit 151 performs development processing (calculation) in accordance with a display format for the imaging data (captured images) captured by each of the imaging units 110, 120, 130, and 140. For example, if there are four imaging units 110, 120, 130, and 140 and it is desired to simultaneously display four images, the image processing unit 151 arranges the imaging data in a square hexagonal pattern and develops them into a single display image.

[0049] The image processing unit 151 also includes an object detection unit 161. The object detection unit 161 analyzes the acquired captured image by image processing. For example, when detecting a moving object as an object, the object detection unit 161 calculates the difference in brightness between frames of the captured image, and determines that a moving object has been detected if there is a change in brightness. The user can instruct via the instruction unit 204 what to detect as an object.

[0050] Furthermore, the image processing unit 151 can display (superimpose) a graphical user interface on the captured image data as an OSD (On-Screen Display). The superimposed OSD display is a graphical user interface that can be operated by the user.

[0051] 18 shows an example of the configuration of an imaging system 190. Fig. 18 is a diagram showing an example of the configuration of an imaging system 190 according to the first embodiment. The imaging system 190 includes an imaging device 100, a network 170, a client device 200, and a display unit 203.

[0052] The imaging device 100 is connected to a client device 200 via a network 170. In Fig. 18, the client device 200 is shown as a desktop computer, but the client device 200 may be a notebook computer or a tablet in which the client device 200 and a display unit 203 are integrated.

[0053] 19 shows detailed configurations of the image capturing apparatus 100 and the client apparatus 200. Fig. 19 is a diagram showing a detailed configuration example of the image capturing apparatus 100 and the client apparatus 200 according to the first embodiment.

[0054] The imaging device 100 includes imaging units 110, 120, 130, and 140, driving units 113, 123, 133, and 143, a network I / F 183, a CPU 180, a RAM 181, and a ROM 182.

[0055] The CPU 180 is a central processing unit that performs overall control of the imaging device 100. The CPU 180 also controls the control unit 152, image processing unit 151, and communication unit 153 shown in FIG.

[0056] The RAM 181 temporarily stores computer programs executed by the CPU 180. The RAM 181 also provides a work area used when the CPU 180 executes processing. The RAM 181 also functions as a frame memory and a buffer memory.

[0057] The ROM 182 stores a program for the CPU 180 to control the imaging device 100 and the like.

[0058] The network I / F 183 transmits the developed display image to the client device 200 via the network 170. Image data captured by each of the imaging units 110, 120, 130, and 140 may be stored in an internal storage device such as a RAM 181 or a ROM 182, which will be described later, or in a removable storage medium (not shown) such as a memory card.

[0059] The client device 200 is an information processing device having a CPU 280 , a RAM 281 , a ROM 282 , an input I / F 284 , an output I / F 285 , and a network I / F 283 .

[0060] The CPU 280 is a central processing unit that controls the client device 200 .

[0061] The RAM 281 provides a work area used when the CPU 280 executes data processing. The RAM 281 also functions as a frame memory and a buffer memory.

[0062] The ROM 282 stores a program for the CPU 280 to control the client device 200 and the like.

[0063] 1 is an interface for receiving operations for the client device 200 input from a user via the instruction unit 204. Operation information for the imaging device 100 can also be received by the input I / F 284.

[0064] The output I / F 285 is an interface that is connected to the display unit 203 in FIG. 18 and causes the display unit 203 to display the display image output from the imaging device 100 .

[0065] The network I / F 283 is an interface that connects to the imaging device 100 via the network 170 and inputs operation information for the imaging device 100 that is input via the input I / F 284 to the imaging device 100. The network I / F 283 also receives a display image output from the imaging device 100.

[0066] <Flowchart> Hereinafter, the operation of the imaging device 100 when detecting an object will be described with reference to Fig. 4. Fig. 4 is a flowchart showing a control method for the imaging device 100 according to the first embodiment, and is a flowchart when detecting an object. This flowchart is realized by the CPU 180 of the imaging device 100 executing a program loaded in RAM. Here, the imaging device 100 is an example of a control device. Multiple imaging units 110, 120, 130, and 140 are built into the imaging device 100.

[0067] In step S401, the object detection unit 161 detects an object from each image captured by the multiple image capture units 110, 120, 130, and 140, and determines whether or not an object has been detected. For example, as shown in FIG. 5(b), the object detection unit 161 detects a moving object 505 in an image 501 captured by the image capture unit 110 as an object. If an object has not been detected, the process returns to step S401 and enters a standby state. If an object has been detected, the process proceeds to step S402.

[0068] In step S402, CPU 180 calculates the position of the pan driving section of the driving section of one of imaging sections (cameras) 110, 120, 130, and 140 that has detected the object.

[0069] In step S403, CPU 180 controls display unit 203 to perform an OSD display showing the direction in which the object was detected on the image of an imaging unit (hereinafter referred to as the object detection camera) adjacent to the imaging unit that detected the object (hereinafter referred to as the adjacent camera). An OSD display showing the direction in which the object was detected is, for example, an arrow pointing in the direction of the object detection camera as seen from the adjacent camera. Details of this will be described later using Figures 5(a) and (b).

[0070] For example, CPU 180 functions as a control unit and controls display unit 203 to display image 501 in which moving body (object) 505 is detected, and images 502 and 504 of imaging units 120 and 140 adjacent to both sides of imaging unit 110 that captured image 501 in which moving body 505 is detected, and image 503. Furthermore, CPU 180 controls display unit 203 to superimpose arrows (images) 507 and 508 indicating the direction of moving body 505 on images 502 and 504 of imaging units 120 and 140 adjacent to both sides.

[0071] In step S404, CPU 180 determines whether the user has operated (clicked or tapped) the arrow displayed on the OSD on client device 200. If the user has not operated, the process returns to step S404 and enters a standby state. If the user has operated, the process proceeds to step S405.

[0072] In step S405, CPU 180 drives the pan drive unit of the adjacent camera whose arrow has been operated in a direction (the direction of the arrow) that brings it closer to the object detection camera.

[0073] When the arrow 507 or 508 is operated, the CPU 180 instructs the imaging area 602 or 504 of the imaging unit 120 or 140 corresponding to the image 502 or 504 in which the operated arrow 507 or 508 is displayed to move closer to the imaging area 601 of the imaging unit 110 that captured the image 501 in which the moving object 505 is detected.

[0074] Specifically, when arrow 507 or 508 is operated, CPU 180 instructs that the pan angle of imaging unit 120 or 140 corresponding to image 502 or 504 in which the operated arrow 507 or 508 is displayed should approach the pan angle of imaging unit 110 that captured image 501 in which moving object 505 is detected.

[0075] Furthermore, when the arrow 507 or 508 is operated, the CPU 180 instructs that at least one of the tilt angle, zoom magnification, focus position, and rotation angle of the imaging unit 120 or 140 corresponding to the image 502 or 504 in which the operated arrow 507 or 508 is displayed be the same as that of the imaging unit 110 that captured the image 501 in which the moving object 505 is detected.

[0076] Furthermore, when the arrow 507 or 508 is operated, the CPU 180 instructs that at least one of the exposure, gain, shutter speed, white balance, and gamma of the imaging unit 120 or 140 corresponding to the image 502 or 504 in which the operated arrow 507 or 508 is displayed be made the same as that of the imaging unit 110 that captured the image 501 in which the moving object 505 is detected.

[0077] The OSD display will be described with reference to Fig. 5(a) and (b). Fig. 5(a) and (b) are display examples of the OSD display according to the first embodiment. Fig. 5(a) is a structural diagram similar to Fig. 2. Fig. 5(b) shows how images from the imaging units 110, 120, 130, and 140 are displayed on the display unit 203 of the client device 200.

[0078] 5(b) is a display image developed by the image processing unit 151, and is displayed on the display of the display unit 203. Also, image 501 is an image captured by the imaging unit 110. Similarly, image 502 is an image captured by the imaging unit 120. Image 503 is an image captured by the imaging unit 130. Image 504 is an image captured by the imaging unit 140.

[0079] 5(b) shows an example in which an object is detected by the imaging unit 110. A moving object 505 is captured within the angle of view of the imaging unit 110, and the object detection unit 161 detects the moving object 505 when the moving object 505 moves and a difference in brightness occurs between frames. When the moving object 505 is detected, the CPU 180 displays the detection result in the image 501 so that the detection result can be seen. In the image 501, a frame 506 of the moving object 505 is displayed as the detection result.

[0080] When an object is detected by the imaging unit 110, the CPU 180 displays an arrow indicating the direction of the object (an arrow indicating the direction in which the moving object 505 is detected) on the image 502 or 504, which is an image from the adjacent camera. Here, an example in which the arrow is displayed on both the image 502 and the image 504 will be described.

[0081] In image 504, CPU 180 displays, on the OSD, arrow 507 indicating the direction of the object. Arrow 507 is an OSD display that indicates the direction in which the object is detected as seen from the imaging unit 140. The pan direction of arrow 507 indicates the right direction, which is the direction of imaging unit 110 that detected the object. The tilt direction of arrow 507 is desirably determined based on the positional relationship between imaging unit 140 and imaging unit 110 and the position at which the object is detected by imaging unit 110. Arrow 507 indicates the case where the center of the imaging angle of view of imaging unit 140 and the position in the tilt direction of detected frame 506 are the same, and arrow 507 is displayed pointing only in the pan direction. The positional relationship between pan and tilt will be described with reference to FIGS. 6 and 7.

[0082] In image 502, CPU 180 displays, on the OSD, an arrow 508 indicating the direction of the object. Arrow 508 is an OSD display that indicates the direction in which the object is detected as seen from the imaging unit 120. The horizontal direction of arrow 508 indicates the left direction, which is the direction of the imaging unit 110 that detected the object. Arrow 508 illustrates a case in which the center of the imaging angle of view of the imaging unit 120 and the tilt direction position of the detected frame 506 differ. When the center of the imaging angle of view of the imaging unit 120 is located above the detected frame 506, arrow 508 points downward and left, as shown in FIG. 5(b). Conversely, when the center of the imaging angle of view of the imaging unit 120 is located below the detected frame 506, arrow 508 points upward and left (not shown).

[0083] In this way, by displaying the direction in which the object was displayed on the image of the adjacent camera as an OSD, the user can know the direction in which the object was located. This improves user operability and makes it easier to capture the area around the detected object.

[0084] Furthermore, in the OSD display, arrows 507 and 508 are not just displays indicating directions, but are also displayed as a user-operable graphical user interface for driving pan / tilt, thereby further improving user operability. When the OSD display of arrow 507 or 508 is operated (tapped / clicked), imaging device 100 controls the pan driving unit, tilt driving unit, zoom driving unit, focus driving unit, and rotation driving unit of the imaging unit displaying the operated arrow. At this time, the imaging unit whose OSD display arrow is operated is driven based on the states of the pan driving unit, tilt driving unit, zoom driving unit, focus driving unit, and rotation driving unit of the imaging unit that detected the object.

[0085] For example, the tilt driver, zoom driver, focus driver, and rotation driver of the adjacent camera are driven to the same positions as the object detection camera. However, the pan driver cannot be positioned in the same position due to interference, so it is driven to the closest position within the driveable range.

[0086] This allows the user to easily observe the area around the object by clicking the arrow. This state in which the area around the camera that detected the object is captured by an adjacent camera is called supplementary capture. By displaying the OSD display as a graphical user interface and automatically performing supplementary capture when the OSD is operated, user operability is further improved, making it easier to capture the area around the detected object.

[0087] The positional relationship, imaging area, and detection area of each of the imaging units 110, 120, 130, and 140 will be described with reference to Fig. 6. Fig. 6 is a diagram showing the imaging areas of the imaging units 110, 120, 130, and 140 according to the first embodiment. Fig. 6 shows coordinates in which the horizontal axis represents the pan angle and the vertical axis represents the tilt angle.

[0088] The shooting area 601 corresponds to the shooting angle of view of the image 501 in Fig. 5(b). Similarly, the shooting area 602 corresponds to the shooting angle of view of the image 502 in Fig. 5(b). The shooting area 603 corresponds to the shooting angle of view of the image 503 in Fig. 5(b). The shooting area 604 corresponds to the shooting angle of view of the image 504 in Fig. 5(b).

[0089] The centers of the imaging areas 601-604 indicate the pan angle and tilt angle positions of the imaging units 110, 120, 130, and 140, respectively. The sizes of the imaging areas 601-604 are determined by the sizes of the solid-state imaging elements 112, 122, 132, and 142 and the optical characteristics of the imaging optical systems 111, 121, 131, and 141, respectively, and the sizes of the imaging areas 601-604 change depending on the zoom magnification. The imaging device 100 stores a table for determining the sizes of the imaging areas 601-604, making it possible to calculate the pan angle positions and tilt angle positions of the boundaries of each of the imaging areas 601-604.

[0090] 5A. Photographing area 601 and photographing area 602 are shown in the same positional relationship as in the description of FIG. 5A. Photographing area 601 and photographing area 604 are positioned at the same tilt angle. Photographing area 601 and photographing area 602 are positioned at different tilt angles. Photographing area 601 also shows the detected frame 506 in FIG. 5B as a detected area 605.

[0091] The position where an object is detected may be determined to be the center of the shooting area 601 or the center of the detection area 605. Here, a case where the position where an object is detected is determined to be the center of the shooting area 601 will be described.

[0092] At this time, the positions of the pan driving unit 114 and tilt driving unit 116 of the imaging unit 110 become the center coordinates of the shooting area 601. When the shooting area 601 is viewed from the shooting area 604, it can be seen that the shooting area 601 is located to the right of the shooting area 604. When the shooting area 601 is viewed from the shooting area 602, it can be seen that the shooting area 601 is located to the lower left of the shooting area 602. In this way, it is possible to notify the user of the positional relationship between the shooting areas 601 to 604 using arrows.

[0093] The angle of the arrow displayed on the OSD screen may be determined by calculating a vector from the relative positions of the pan driver and tilt driver for each imaging unit. Alternatively, the angle of the arrow displayed on the OSD screen may be displayed as a discrete angle, such as 45 degrees. Furthermore, imaging device 100 may determine whether the imaging units are close to each other based on the length of this vector.

[0094] Shooting areas 601 to 604 after the user clicks an arrow on the OSD display to perform pan / tilt driving will be described with reference to Fig. 7. Fig. 7 is a diagram showing shooting areas 601 to 604 after pan / tilt driving according to the first embodiment. Fig. 7 shows shooting areas 601 to 604 after driving the imaging units 120 and 140, in contrast to Fig. 6.

[0095] Here, a pan and tilt driving method will be described. The tilt angle of the shooting area 604 is the same as that of the shooting area 601, so the tilt driving unit 146 of the imaging unit 140 is not driven. The pan angle of the shooting area 604 is such that there is a driving margin between the shooting areas 601 and 604, so the imaging unit 140 is driven by the pan driving unit 144 in the direction of the imaging unit 110 until it is just before interfering with the imaging unit 110 (to the closest position within the driving range).

[0096] In the shooting area 602, similarly to the shooting area 604, the image capturing unit 120 is driven by the pan driving unit 124 toward the image capturing unit 110 to the closest position within the driving range. Furthermore, the image capturing unit 120 is driven by the tilt driving unit 126 to the same tilt driving unit position as the image capturing unit 110.

[0097] The case where the position where an object is detected is determined to be the center of the shooting area 601 has been described, but the case where OSD display and driving of the drive unit are performed at the center position of the detection area 605 will be described.

[0098] The image processing unit 151 calculates the position of the pixel located at the center of the detection area 605 where the object is detected. As shown in Fig. 6 and Fig. 7, the pan angle and tilt angle of each imaging unit are controlled by the imaging device 100, so the positional relationship between the imaging areas 601 to 604 is self-evident.

[0099] The image processing unit 151 corrects the positions of the pan driving unit and the tilt driving unit based on the position within the angle of view of the detected detection area 605, the size of the solid-state imaging device, the optical characteristics of the imaging optical system, and the zoom magnification. This enables the image processing unit 151 to calculate the positions of the pan driving unit and the tilt driving unit relative to the position of the detection area 605.

[0100] Control after pan / tilt driving will be described with reference to Figures 8(a) and (b). Figures 8(a) and (b) show an example of an OSD display for ending complementary shooting by the pan / tilt driving unit according to the first embodiment.

[0101] The end button 801 and the end button 802 are graphical user interfaces for ending the complementary photographing state in which the photographing regions 601 to 604 in FIG. 7 have been moved, and for returning the photographing regions to the positions of the photographing regions 601 to 604 in FIG.

[0102] When the arrow 507 or 508 is operated, the CPU 180 controls the display unit 203 to display end buttons 801 and 802 that can be operated by the user. When the end button 801 or 802 is operated, the CPU 180 instructs the imaging unit 120 or 140 to return the imaging area 602 or 604 corresponding to the operated arrow 507 or 508 to its original position.

[0103] The end button 801 and the end button 802 may be represented graphically as shown in FIG. 8(b), or may be represented by text that means "end," "reference position," "home position," or the like. The end button 801 and the end button 802 may display a graphical user interface outside the image area, rather than an OSD display on the image. The supplementary shooting state may also be terminated after a certain period of time has elapsed. Only one end button may be displayed, and when that end button is operated, the positions of all the imaging units may be returned to the positions shown in FIG. 6. While an example in which the arrow graphical user interface is not displayed has been described here, the arrow graphical user interface may also be displayed simultaneously with the end button.

[0104] The operation of each driving unit during complementary photography will be further described with reference to Figures 9(a) to 9(f), which are diagrams showing an example of complementary photography according to the first embodiment.

[0105] In complementary shooting, an adjacent camera is driven in the pan direction of the imaging unit that detected the object. However, there is no unique driving method. For example, Figure 7 shows an example in which the pan driving unit is driven to the closest position within its driving range, but Figures 9(a) to (f) introduce six examples of complementary shooting that differ from Figure 7.

[0106] 9(a) to 9(f) also illustrate examples of the operation of drive units other than the pan drive unit. These examples of complementary shooting can be modified or combined. As with FIG. 6, FIGS. 9(a) to 9(f) illustrate a case where an object is detected in a shooting area 601 and complementary shooting is performed in a shooting area 602. For clarity of explanation, other shooting areas are omitted. Shooting areas indicated by dotted lines indicate the shooting area before complementary shooting. Shooting areas indicated by solid lines indicate the shooting area during complementary shooting. In FIGS. 9(a) to 9(f), solid lines alone indicate that the shooting area remains unchanged before and after complementary shooting.

[0107] FIG. 9(a) shows an example in which the pan angle of the imaging area 602 is moved to a position where the imaging area 602 approaches the imaging area 601 and the boundaries of the imaging areas 601 and 602 are aligned. The boundaries of the imaging areas can be calculated from the size of the solid-state imaging element, the optical characteristics of the imaging optical system, and the zoom magnification. In FIG. 9(a), the zoom magnification and tilt angle of the imaging area 602 are set to be aligned with those of the imaging area 601, and the pan angle is determined to match the boundary of the imaging area 601. This makes it possible to prevent blind spots from occurring.

[0108] 9(b) is an example in which the shooting areas 601 and 602 overlap with the example in FIG. 9(a). In FIG. 9(b), the pan angle may be brought closer to the shooting area 601 within a driveable range, and the amount by which the shooting areas 601 and 602 overlap may be determined to be, for example, 10% of the horizontal angle of view. This makes it easier to capture the scene near the boundary between the shooting areas 601 and 602.

[0109] FIG. 9(c) shows an example in which only the pan drive unit is driven during complementary shooting. As shown in FIG. 9(c), by bringing the pan angle of the shooting area 602 closer to the shooting area 601, it is possible to capture a position closer to the shooting area 601 than the shooting area of the shooting area 602 indicated by the dotted line. The pan angle in this case is the same as in FIG. 9(a) or FIG. 9(b). In this case, the zoom magnification is shown as being the same before and after complementary shooting, but it may be changed. Furthermore, by changing the zoom magnification to the wide-angle side, it is possible to reduce blind spots and make it easier to capture the area around the object detection camera.

[0110] FIG. 9(d) shows an example in which the tilt angle is adjusted to the same angle as the imaging area 601, as compared to FIG. 9(c). As with the pan angle, by bringing the tilt angle of the imaging area 602 closer to that of the imaging area 601, it is possible to capture a complementary image of the periphery of the imaging area 601. In this case, by not only bringing the tilt angles closer but also making them the same angle, the positional relationship of the tilt angles of the imaging units is aligned, making it easier for the user to view the image. Furthermore, the tilt angle of the imaging area 602 may be aligned with the center of the imaging area 601 or with the detection area 605.

[0111] FIG. 9(e) shows an example in which not only the zoom magnification of the adjacent camera but also the zoom magnification of the camera that detected the object is changed. If the zoom magnification is high during complementary shooting, the adjacent cameras will get close to each other, causing physical interference between the imaging units, resulting in a gap (a blind spot). This creates a gap between the shooting area 601 and the shooting area 604 in FIG. 7, creating a blind spot in the pan angle direction. Therefore, during complementary shooting, the zoom magnification of the shooting area 601 is changed to the wide-angle side to reduce the blind spot. In this case, it is desirable to change the zoom magnification to the wide-angle side until the blind spot disappears, as shown in FIG. 9(a) or 9(b). The shooting area for which the zoom magnification is changed may be either the shooting area 601 or the shooting area 602, or both.

[0112] 9(f) shows an example in which the imaging area 601 is rotated by +90 degrees from 0 degrees. In this case, the imaging area 602 is also rotated by +90 degrees to capture a complementary image. If the angles of the rotation drive units are different, setting the rotation angle of the adjacent camera to the same angle as the rotation angle of the imaging unit that detected the object makes it possible to display the positional relationship in an easy-to-understand manner when the user compares the two images.

[0113] Furthermore, by aligning the position of the focus drive unit of the adjacent camera with the position of the focus drive unit of the object detection camera during complementary photography, it becomes easier to focus on the periphery of the detected object.

[0114] Furthermore, when capturing supplementary images, the exposure of the adjacent camera, the gain of the solid-state image sensor, the shutter speed, the white balance, the gamma, and other image-related parameters may be matched with those of the object detection camera, thereby reducing discrepancies in brightness and image quality between images.

[0115] Also, Figure 9(e) explains what happens when a blind spot occurs in the pan angle direction. If the rotation angle is 90 degrees, the angle of view in the pan angle direction can be widened by setting it to 0 degrees. In this way, when there is a blind spot, the blind spot in the pan angle direction can be reduced by driving the rotation angle to a landscape aspect ratio.

[0116] A supplementary explanation of this embodiment will be given below.

[0117] <Object detection method> Although a method for detecting an object from the brightness difference between frames has been shown, other methods may also be used. For example, the object detection method may be attribute authentication that determines whether the object is a person or a vehicle from the edge shape of the acquired image. Furthermore, when the detection target for which edge detection has been performed is a moving object, it is possible to calculate the moving direction (movement vector) of the moving object by taking the difference between frames. Here, one example of an object detection method has been shown, but detection may also be performed using a different method.

[0118] <Supplementary information when the driving range is limited> A case where the driving range is limited will be described with reference to Figures 10(a) and 10(b), which are diagrams showing the driving range according to the first embodiment.

[0119] Fig. 10(a) shows an example in which a drive range limit 1001 is added to Fig. 5(a). The drive range limit 1001 is a position that cannot be driven by hardware due to physical interference, or a position that cannot be instructed by software (control unit 152).

[0120] Since the imaging unit cannot move to a position within drive range limit 1001, the closest that imaging unit 140 can get to imaging unit 110 is up to drive range limit 1001. Therefore, when drive range limit 1001 exists between the imaging units in this way, imaging unit 140 cannot be driven close to imaging unit 110, so OSD display for the adjacent camera is not required. In this case, imaging unit 120 is the only adjacent camera that displays OSD, and arrow 508 is displayed only on image 502.

[0121] Furthermore, when the imaging unit 110 is located near the limit 1001 of the driving range, the imaging area 601 and the imaging area 604 may overlap by moving the imaging unit 140 closer to the imaging unit 110. In this case, even if the limit 1001 of the driving range exists, the OSD display may be performed on the image 504 of the imaging unit 140.

[0122] As described above, there may be a driving range limit 1001. In this case, one of the imaging units 120 and 140 adjacent to the imaging unit 110 that captured the image 501 in which the moving object 505 is detected cannot move on the same circumference 102 in a direction approaching the imaging unit 110 that captured the image 501 in which the moving object 505 is detected.

[0123] Furthermore, the other of the imaging units 120 and 140 adjacent to the imaging unit 110 that captured the image 501 in which the moving object 505 was detected can move on the same circumference 102 in a direction approaching the imaging unit 110 that captured the image 501 in which the moving object 505 was detected.

[0124] In this case, the CPU 180 performs control so that an arrow 508 indicating the direction of the moving object 505 is superimposed on the image 502 of the other imaging unit 120 .

[0125] <Addition of coordinates where object was detected> 6, a case where the position where an object is detected is determined to be the center of the shooting area 601. A case where the position where an object is detected is determined to be the center of the detection area 605 will be described.

[0126] When calculating the coordinates of an object from the position of the pixels where the object in the imaging unit that has detected the object is detected, it is desirable for the imaging device 100 to have a coordinate system common to each imaging unit.

[0127] When the detected position of the object is set as the center of the detection area 605, coordinates are calculated from the zoom magnification and the position of the detected pixels in addition to the rotation angles of the pan and tilt of the imaging unit.

[0128] <Supplement of the effect> Supplement the reason for performing OSD display on adjacent cameras. Assuming that there are four imaging units, if OSD display is performed on non - adjacent cameras (=opposite cameras), even if trying to move the camera facing in that direction, it will interfere with the adjacent cameras and cannot be moved. Therefore, by performing OSD display on adjacent cameras, the operability of the user can be improved.

[0129] <Supplement of OSD display> In FIG. 5(b), the heptagonal arrows 507 and 508 are given as examples of OSD displays indicating the direction in which the object is detected. However, as long as it is clear which direction it is facing, the number and shape of the arrows 507 and 508 are not limited. As long as the directions of the arrows 507 and 508 are clear, they can be triangles or quadrilaterals.

[0130] In the arrows 507 and 508 in FIG. 5(b), the length may be changed according to the position from the relative positions of the object - detecting camera and the adjacent camera. When the relative position is far, the arrow can be made longer, and when the relative distance is close, the arrow can be made shorter, etc. This makes it easier for the user to grasp the positional relationship with the camera that has detected the object (or the area where the object has been detected).

[0131] When performing an OSD display indicating the direction in which the object is detected, separately from the OSD display, a graphical user interface that allows the user to operate the pan / tilt, etc. may be displayed outside the frames of the images 501, 502, 503, 504.

[0132] <Supplement of the positional relationship> As shown in the configuration of Figure 2, the physical arrangement of the imaging units does not change, so the adjacent cameras relative to the object detection camera do not change positions. Also, the direction of the object detection camera relative to the adjacent cameras is uniquely determined. Therefore, although the pan angle has been used in the explanations in Figures 6, 7, and 9, it is possible to display an OSD showing the direction of an object (right / left) without calculating the pan angle.

[0133] <Linked operation during supplementary shooting> When capturing supplementary footage, it may be necessary to maintain the relative positions of the object detection camera and the adjacent camera. When the detection target is a moving object, the user may drive the pan / tilt to match the movement of the moving object. In this case, when the user operates the drive unit of the object detection camera, it is desirable that the adjacent camera that is capturing supplementary footage in conjunction with the object detection camera be controlled as well.

[0134] For example, if the pan angle of the object detection camera is driven by +10 degrees, the pan angle of the adjacent camera is also driven by +10 degrees. This eliminates the need to operate multiple imaging units when taking supplementary images, improving operability. Also, while an example in which the user drives the camera in accordance with a moving object has been shown here, the same applies to driving the camera by automatic tracking. Automatic tracking is a technology that keeps the target subject captured at a predetermined position in the angle of view (for example, the center of the angle of view), and performs pan / tilt driving by calculating the amount and direction of movement of the target subject.

[0135] When the shooting area 601 of the imaging unit 110 that captured the image 501 in which the moving object 505 is detected moves, the CPU 180 instructs the shooting areas 602 and 604 of the adjacent imaging units 120 and 140 to also move in conjunction with the movement.

[0136] In this way, by operating the drive unit of the adjacent camera in conjunction with the operation of the drive unit of the object detection camera, the user does not need to operate the adjacent camera, improving operability.

[0137] Here, the interlocking operation for the pan drive is shown, but the present invention can be similarly applied to drive units other than the pan drive unit.

[0138] <How to operate the OSD display of the adjacent camera> 5(b) shows an example in which an OSD display is performed on both adjacent cameras when there are two adjacent cameras, but it is also possible to display an OSD on only one of them. If an OSD display is performed on both cameras, the number of images to be focused on will increase, and it is possible that the user will become confused about how to operate the camera. Therefore, four methods will be described below for the imaging device 100 to determine which adjacent camera to display the OSD on.

[0139] In the first method, the movement vector of the moving object 505 is calculated using the inter-frame difference, and an OSD is displayed on an adjacent camera located in the traveling direction (pan direction) of the moving object 505. The object detection unit 161 detects the movement direction of the moving object 505. The CPU 180 controls the image capture unit 110 to capture an image 501 in which the moving object 505 is detected, and to superimpose an arrow 507 or 508 indicating the direction of the moving object 505 on the image 502 or 504 of the image capture unit 120 or 140 located in the movement direction of the moving object 505, out of the image capture units 120 and 140 adjacent to both sides of the image capture unit 110 that captured the image 501 in which the moving object 505 was detected. This makes it easier for the adjacent camera to capture the moving object 505 even if the moving object 505 moves out of the angle of view of the object detection camera.

[0140] The second method is a method of determining the direction of the moving object 505 based on the relative position of the pan angle. An OSD display is performed on the adjacent camera whose pan angle relative to the object detection camera is closest. This allows the adjacent camera that was capturing an area close to the object detection camera to capture the surroundings. While the pan angle has been used as an example, the tilt angle may also be used to calculate the relative distance between the image capturing units and determine the direction of the moving object 505 based on that relative distance. In this case, the CPU 180 controls the image capturing unit 120 or 140 adjacent to the image capturing unit 110 that captured the image 501 in which the moving object 505 was detected to superimpose an arrow indicating the direction of the moving object 505 on the image 502 or 504 of the image capturing unit 120 or 140 that captures the image capturing area 602 or 604 that is closer to the image capturing area 601 of the image capturing unit 110 that captured the image 501 in which the moving object 505 was detected.

[0141] The third method is a method of determining the adjacent camera on which to display the OSD based on the position of the detection area frame 506 within the angle of view of the object detection camera. For example, in FIG. 5B, if the image 501 is divided into two in the pan angle direction, and the detection area frame 506 is on the right side of the image 501, the OSD display is performed on the image 502 of the image capture unit 120 located to the right of the image capture unit 110. Conversely, if the detection area frame 506 is on the left side of the image 501, the OSD display is performed on the image 504 of the image capture unit 140 located to the left of the image capture unit 110. Furthermore, if the image 501 is divided into three in the pan direction, and the detection area frame 506 is located in the center, the OSD display may be performed on either adjacent camera. If the moving object 505 is detected in the right area of the image 501 in which the moving object 505 is detected, the CPU 180 controls the image 502 of the image capture unit 120 located to the right of the image capture unit 110 that captured the image 501 in which the moving object 505 is detected to be superimposed with the image 502. Furthermore, when moving object 505 is detected in the left region of image 501 in which moving object 505 is detected, CPU 180 controls so that arrow 507 indicating the direction of moving object 505 is superimposed on image 504 of imaging unit 140 adjacent to the left of imaging unit 110 that captured image 501 in which moving object 505 is detected. This allows OSD display to be performed on the adjacent camera that can more easily capture the area around the object detection region.

[0142] The fourth method is a method of determining the position based on the arrangement of images from the object detection camera and the adjacent camera. For example, in FIG. 5(b), four images from each imaging unit are displayed in a square-shaped pattern, so that image 502 is the only image arranged adjacent to image 501. Therefore, an OSD display is performed on the adjacent camera (imaging unit 120) capturing image 502. The CPU 180 controls the display of image 502 from one of the imaging units 120 and 140 adjacent to both sides of the imaging unit 110 that captured image 501 in which the moving object 505 was detected, in a position adjacent to the image 501 in which the moving object 505 was detected. The CPU 180 also controls the display of image 504 from the other imaging unit 140, from the imaging units 120 and 140 adjacent to both sides of the imaging unit 110 that captured image 501 in which the moving object 505 was detected, in a position that is not adjacent to the image 501 in which the moving object 505 was detected. The CPU 180 controls the display so that an arrow 508 indicating the direction of the moving object 505 is superimposed on the image 502 of one of the imaging units 120. This allows the user to look at only the side-by-side images, improving visibility.

[0143] As described above, by narrowing down the image displayed on the OSD to one adjacent camera, the user does not need to decide which adjacent camera they want to focus on, improving user operability.

[0144] <Method of determining whether to display OSD based on relative distance> The image capturing apparatus 100 may determine whether to display the OSD based on the relative position of the object detection camera and the adjacent camera. If the relative distance is smaller (closer) than a threshold, the OSD is displayed.

[0145] The CPU 180 controls the display so that arrows 507 and 508 indicating the direction of the moving object 505 are superimposed when the distance between the imaging unit 110 that captured the image 501 in which the moving object 505 is detected and the adjacent imaging units 120 and 140 is shorter than a threshold value.

[0146] This allows the user to know that the adjacent camera is close enough when the arrows 507 and 508 are displayed on the OSD.

[0147] <Number of imaging units> Although the configuration with four imaging units has been described, the present invention can be applied to an imaging device 100 having at least two imaging units including an object detection camera and an adjacent camera. Furthermore, it is preferable to have four or more imaging units.

[0148] <highlight> By highlighting the images from the object detection camera or the adjacent camera, it is possible to make it easier for the user to see the image that should be focused on. Highlighting refers to adding color to the frames of images 501, 502, and 503 in Figure 5(b) or displaying them larger.

[0149] The CPU 180 controls to highlight both or either of the image 501 in which the moving object 505 is detected and the images 502 and 504 of the adjacent image capturing units 120 and 140 .

[0150] <Electronic zoom> Although an example in which the zoom driving unit is driven when the zoom magnification is changed has been shown, the present invention can also be applied to electronic zoom using digital processing.

[0151] (Second embodiment) In the first embodiment, a method for performing image processing in the image processing unit 151 of the imaging device 100 has been described. In the second embodiment, a case where at least a part of the functions of the image processing unit 151 is in the client device 200 will be described. For example, object detection may be performed in the client device 200, and image processing for OSD display may be performed in the client device 200. Differences between the second embodiment and the first embodiment will be described below.

[0152] Fig. 20(a) is a flowchart showing a control method for the imaging device 100 according to the second embodiment, and Fig. 20(b) is a flowchart showing a control method for the client device 200 according to the second embodiment. The flowchart in Fig. 20(a) is realized by the CPU 180 of the imaging device 100 executing a program loaded in RAM. The flowchart in Fig. 20(b) is realized by the CPU 280 of the client device 200 executing a program loaded in RAM. Here, the client device 200 is an example of a control device. The multiple imaging units 110, 120, 130, and 140 are provided outside the client device 200.

[0153] Steps S401 to S405 in Figures 20(a) and (b) correspond to steps S401 to S405 in Figure 4. The imaging device 100 performs the processes of steps S2001, S2004, and S405. The client device 200 performs the processes of steps S2002, S401 to S404, and S2003.

[0154] In step S2001, CPU 180 of imaging device 100 transmits the captured images of each imaging unit 110, 120, 130, and 140 in frame units of the captured image to client device 200 via network I / F 183. Furthermore, CPU 180 may transmit information such as the pan angle, tilt angle, zoom magnification, rotation angle, and focus position of each imaging unit 110, 120, 130, and 140 in frame units of the captured image to client device 203 as needed.

[0155] In step S2002, the CPU 280 of the client device 200 receives the captured images of each of the imaging units 110, 120, 130, and 140 on a frame-by-frame basis from the imaging device 100 via the network I / F 283. The CPU 280 may also receive information such as the pan angle, tilt angle, zoom magnification, rotation angle, and focus position of each of the imaging units 110, 120, 130, and 140 from the imaging device 100 on a frame-by-frame basis.

[0156] In step S401, the CPU 280 detects an object from images captured by the multiple imaging units 110, 120, 130, and 140, and determines whether or not the object has been detected. For example, as shown in FIG. 5(b), the CPU 280 detects a moving object 505 in an image 501 captured by the imaging unit 110 as the object.

[0157] If an object is not detected, the process returns to step S2002. If an object is detected, the process proceeds to step S402.

[0158] In step S402, the CPU 280 calculates the position of the pan driving section of the driving section of the imaging section 110, 120, 130, or 140 that has detected the object.

[0159] In step S403, CPU 280 controls display unit 203 to display an OSD indicating the direction in which the object was detected on the images captured by the imaging unit (object detection camera) that detected the object and the imaging unit (adjacent camera) that is adjacent to the imaging unit, as shown in Fig. 5(b). The OSD indicating the direction in which the object was detected is, for example, arrows 507 and 508 in Fig. 5(b).

[0160] For example, CPU 280 functions as a control unit and controls display unit 203 to display image 501 in which moving body (object) 505 is detected, and images 502 and 504 of imaging units 120 and 140 adjacent to imaging unit 110 that captured image 501 in which moving body 505 is detected, and image 503. Furthermore, CPU 280 controls display unit 203 to superimpose arrows (images) 507 and 508 indicating the direction of moving body 505 on images 502 and 504 of adjacent imaging units 120 and 140.

[0161] In step S404, CPU 280 determines whether the user has operated (clicked or tapped) arrow 507 or 508 displayed on the OSD on client device 200. If the user has not operated, the process returns to step S404 and enters a standby state. If the user has operated, the process proceeds to step S2003.

[0162] In step S2003, CPU 280 transmits control signals such as the pan angle, tilt angle, zoom magnification, rotation angle, and focus position of imaging units 110, 120, 130, and 140 to imaging device 100 via network I / F 283 to drive the adjacent camera whose arrow has been operated in a direction (the direction of the arrow) that approaches the object detection camera. For example, CPU 280 transmits control signals to realize imaging areas 601, 602, etc. of imaging units 110, 120, 130, and 140 as shown in FIGS. 9(a) to 9(f).

[0163] That is, when the arrow 507 or 508 is operated by the control signal, the CPU 280 instructs the imaging area 602 or 504 of the imaging unit 120 or 140 corresponding to the image 502 or 504 in which the operated arrow 507 or 508 is displayed to move closer to the imaging area 601 of the imaging unit 110 that captured the image 501 in which the moving object 505 is detected.

[0164] Specifically, when arrow 507 or 508 is operated, CPU 280 instructs that the pan angle of imaging unit 120 or 140 corresponding to image 502 or 504 in which the operated arrow 507 or 508 is displayed should approach the pan angle of imaging unit 110 that captured image 501 in which moving object 505 is detected.

[0165] Furthermore, when the arrow 507 or 508 is operated, the CPU 280 instructs that at least one of the tilt angle, zoom magnification, focus position, and rotation angle of the imaging unit 120 or 140 corresponding to the image 502 or 504 in which the operated arrow 507 or 508 is displayed be the same as that of the imaging unit 110 that captured the image 501 in which the moving object 505 is detected.

[0166] Furthermore, when the arrow 507 or 508 is operated, the CPU 280 instructs that at least one of the exposure, gain, shutter speed, white balance, and gamma of the imaging unit 120 or 140 corresponding to the image 502 or 504 in which the operated arrow 507 or 508 is displayed be made the same as that of the imaging unit 110 that captured the image 501 in which the moving object 505 is detected.

[0167] In step S2004, the CPU 180 of the imaging device 100 determines whether or not control signals for the pan angle, tilt angle, zoom magnification, rotation angle, focus position, etc. of the imaging units 110, 120, 130, and 140 have been received from the client device 200 via the network I / F 183. If no control signals have been received, the process returns to step S2001. If any control signals have been received, the process proceeds to step S405.

[0168] In step S405, based on the received control signal, CPU 180 controls the pan angle, tilt angle, zoom magnification, rotation angle, focus position, etc. of imaging units 110, 120, 130, and 140. Specifically, CPU 180 drives the pan drive unit, etc. of the adjacent camera whose arrow has been operated in a direction (the direction of the arrow) that moves closer to the object detection camera.

[0169] (Third embodiment) In the first embodiment, the OSD display when an object is detected and the method of driving the pan drive unit and the like when the arrow on the OSD display is operated are described. In the third embodiment, the arrangement of images displayed when an object is detected is described with reference to Figs. 11 to 16. In the third embodiment, by rearranging the images from the object detection camera and the adjacent camera so that they are arranged side by side, a highly visible image is displayed that makes it easy for the user to grasp the positional relationship of the imaging units.

[0170] An example of rearranging the images 501 to 504 will be described with reference to Fig. 11 and Fig. 12. Fig. 11(a) and (b) are diagrams showing a display example in which the images 501 to 504 according to the third embodiment are shifted one by one. Fig. 11(a) and (b) show an example in which the rearrangement of the displayed images 501 to 504 is rearranged compared to Fig. 5(a) and (b).

[0171] Fig. 11(b) shows a case where arrow 507 in Fig. 5(b) is operated and imaging unit 140 is performing complementary imaging for imaging unit 110. Fig. 11(b) explains how to rearrange images when there are four imaging units 110, 120, 130, and 140 and four images 501 to 504 are displayed in a cross-shaped pattern.

[0172] The imaging unit 140 moves closer to the imaging unit 110 to complement the area on the left side of the image 501 on the paper. Therefore, by arranging the image 504 side by side on the left side of the image 501, the user can intuitively grasp the positional relationship. Therefore, Fig. 11(b) shows an example in which the positions of the images 501 to 504 are shifted one by one with respect to Fig. 5(b), and the image 504 is arranged side by side on the left side of the image 501.

[0173] Also, although Fig. 11(b) shows an example in which the positions of the images 501 to 504 are shifted one by one, other methods may be used. Figs. 12(a) and (b) show a display example in which the positions of the images 501 to 504 are swapped. Figs. 12(a) and (b) show a display example in which the positions of the images 501 to 504 are swapped and shifted.

[0174] In Figure 12(b), the images 501 and 502 in Figure 5(b) are rearranged, and further the images 502 and 503 are rearranged. Even in this way, it is possible to arrange image 504 horizontally to the left of image 501, as in Figure 11(b).

[0175] 5(b), the CPU 180 controls the image 502 of one of the imaging units 120 and 140 adjacent on both sides to the imaging unit 110 that captured the image 501 in which the moving object 505 was detected, so as to be displayed in a position adjacent in the horizontal direction to the image 501 in which the moving object 505 was detected. The CPU 180 also controls the image 504 of the other imaging unit 140 of the imaging units 120 and 140 adjacent on both sides to the imaging unit 120, so as to be displayed in a position that is not adjacent in the horizontal direction to the image 501 in which the moving object 505 was detected.

[0176] Thereafter, when the arrow 507 displayed on the image 504 of the other imaging unit 140 is operated, the CPU 180 controls the image 504 of the other imaging unit 140 to be displayed in a position laterally adjacent to the image 501 in which the moving object 505 is detected, as shown in Fig. 11 or 12. Then, the CPU 180 controls the image 502 of one imaging unit 120 to be displayed in a position not laterally adjacent to the image 501 in which the moving object 505 is detected.

[0177] 11 and 12 illustrate the case where the arrow 507 in FIG. 5(b) is operated, but the case before the operation may also be the case. For example, if there is a driving range limitation and there is only one adjacent camera, the images of the corresponding adjacent camera are arranged side by side so that they match the positional relationship of the complementary shooting. Furthermore, the images of the object detection camera and one of the two adjacent cameras whose shooting areas are located close to each other may be arranged side by side.

[0178] Next, an example of arranging images 502 and 504 from adjacent cameras on either side of an image 501 from an object detection camera will be described with reference to Figures 13(a) and (b). Figures 13(a) and (b) are diagrams showing a display example in which images 502 and 504 from adjacent cameras are arranged on either side of an image 501 from an object detection camera.

[0179] Fig. 13(b) shows an example in which, compared to Fig. 5(b), the arrangement has been changed so that after object detection, images 502 and 504 from the adjacent cameras are positioned on either side of image 501 from the object detection camera. As with Figs. 11 and 12, the arrangement of images 501 to 504 is determined by the positional relationship between the detection camera and the adjacent cameras.

[0180] 5(b), when images 501 to 504 are arranged in a square pattern, not only are the positions of images 501 to 504 changed, but the sizes or aspect ratios of images 501, 502, 503, and 504 are also changed so that they fit on display unit 203. Since the camera in which an object is detected attracts the most attention, it is desirable that image 501 not be made smaller in size than the other images 502 to 504. Furthermore, image 503 is an image in which no object is detected, and it may be made smaller in size or may be made invisible.

[0181] Next, a case where a vertically captured image is displayed will be described with reference to FIGS.

[0182] With reference to Figures 14(a) and (b), a case where image 501 is displayed in portrait orientation with the image rotated 90 degrees will be described. Figures 14(a) and (b) are examples of display layouts for portrait orientation. Figure 14(b) shows an example in which all images 501 to 504 are arranged horizontally, whereas images 501 to 504 are arranged in a square pattern in Figure 5(b), and all images have a vertically long layout space.

[0183] The image capturing unit 110 is rotated by 90 degrees and captures an image 501 with a portrait angle of view. The other image capturing units 120, 130, and 140 are not rotated and capture images 502 to 504 with a landscape angle of view.

[0184] Image 501 is displayed tilted by 90 degrees in accordance with the rotation angle of imaging unit 110. When the display range of display unit 203 is horizontal, image 501 can be displayed larger than when images are arranged in a square pattern (horizontal orientation is prioritized) by arranging image 501 so that it is vertically long relative to the rotated image, as shown in Fig. 14(b).

[0185] Next, a display rearrangement method and an adjacent camera drive method when the object detection camera is performing rotational shooting will be described with reference to Figures 15(a) and (b). Figures 15(a) and (b) are diagrams showing display examples of a rearrangement method and an adjacent camera drive method during rotational shooting.

[0186] In FIG. 15(b), the display order of images 501 to 504 has been rearranged compared to FIG. 14(b). Images 502 and 504 from adjacent cameras are arranged on either side of image 501 from the object detection camera. Furthermore, in FIG. 15(b), as shown in FIG. 9(f), the adjacent camera is rotated by 90 degrees in the same way as the object detection camera. At this time, images 502 and 504 from the adjacent camera are also displayed tilted by 90 degrees, in the same way as image 501 from the object detection camera. This allows the images to be viewed side by side, making it easier for the user to complement and view the images around the object.

[0187] Next, the timing for rearranging the displayed images will be described with reference to Fig. 16. Fig. 16 is a flowchart for rearranging images. This flowchart is implemented by having CPU 180 of imaging device 100 execute a program loaded in RAM. Fig. 16 adds steps S1601 and S1602 for rearranging the display to Fig. 4. The timing for rearranging the images may be the timing after an object is detected in step S401, as in step S1601, or the timing after an arrow on the OSD display is operated and driven, as in step S1602.

[0188] After step S402, in step S1601, CPU 180 rearranges the display of images 501 to 504 on display unit 203, for example, as shown in Figures 13 to 15. Thereafter, the process proceeds to step S403.

[0189] After step S404, in step S1602, the CPU 180 rearranges the display of the images 501 to 504 on the display unit 203, for example, as shown in Fig. 11 or 12. Thereafter, the process proceeds to step S405.

[0190] As described above, after the moving object 505 is detected in step S401, or after the arrow 507 or 508 is operated in step S404, the CPU 180 changes the arrangement of the images 501 to 504 of the multiple imaging units 110, 120, 130, and 140 displayed on the display unit 203.

[0191] As in the second embodiment, the processes of steps S1601 and S1602 may be performed by the CPU 280 of the client device 200.

[0192] 14 and 15 show examples of vertically shot images aligned with the rotated images, but the images may be arranged in a square pattern regardless of whether they are rotated or not.Also, all images may be arranged side by side with the horizontal aspect ratio.

[0193] (Fourth embodiment) In the fourth embodiment, a case where a detected subject moves and the camera that detects the subject changes will be described with reference to Figures 17(a) and 17(b). Figures 17(a) and 17(b) are diagrams showing an example in which a subject moves according to the fourth embodiment.

[0194] 17(b), in contrast to FIG. 8(b), shows a case where moving object 505 that was being photographed in the photographing area of imaging unit 110 moves into the photographing area of imaging unit 120. At this time, the imaging unit 120 becomes an object detection camera, and imaging unit 110 and imaging unit 130 become adjacent cameras, changing their roles.

[0195] When the moving object 505 moves from the image 501 of the imaging unit 110 in which the moving object 505 was detected to the image 502 of another imaging unit 120, the object detection unit 161 detects the moving object 505 in the image 502 of the other imaging unit 120.

[0196] The CPU 180 controls the other imaging unit 120 so that arrows (images) 806 and 807 indicating the direction of the moving object 505 are superimposed on the images 501 and 503 of the imaging units 110 and 130 adjacent to the imaging unit 120.

[0197] Since image capturing unit 110 is adjacent to image capturing unit 120, which has become an object detection camera, it is desirable to maintain the positional relationship between images 501 to 504. For this reason, an end button 805 and an arrow 806 are displayed on image 501.

[0198] Since the imaging unit 140 is positioned opposite the imaging unit 120 that is detecting the object, the drive unit may be driven to the position before the complementary image capture. Here, an example is shown in which an end button 804 is displayed on the image 504 so that the user can operate it and move it.

[0199] Since the image capturing unit 130 becomes the new adjacent camera, an arrow 806 is displayed on the image 503 .

[0200] In this way, by changing the display method depending on the imaging unit capturing the subject, even if the subject moves, it is possible to capture a complementary image of the area around the detected object.

[0201] In the fourth embodiment, similarly to the second embodiment, the client device 200 may perform the processing of the fourth embodiment.

[0202] As described above, according to the first to fourth embodiments, each of the imaging units 110, 120, 130, and 140 can be driven independently on the common circumference 102. The imaging system 190 can easily capture an image of the periphery of the imaging unit that detects an object, thereby improving operability.

[0203] (Other embodiments) The present disclosure can also be realized by a process in which a program that realizes one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device read and execute the program. The present disclosure can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0204] Although the preferred embodiments have been described above, the present invention is not limited to these embodiments and various modifications and changes are possible within the scope of the gist thereof.

[0205] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) a detection unit that detects an object from each of the images captured by the plurality of imaging units; a control unit that controls a display unit to display an image in which the object is detected and an image of an imaging unit adjacent to the imaging unit that captured the image in which the object is detected, and to superimpose an image related to the object on the image of the adjacent imaging unit; A control device comprising: (Configuration 2) The control device according to configuration 1, characterized in that the control unit controls the image of the object to be superimposed on the images of the imaging units adjacent to both sides of the imaging unit that captured the image in which the object was detected. (Configuration 3) the object is a moving object, the detection unit detects a moving direction of the moving object; The control device according to configuration 1, characterized in that the control unit controls the image of the imaging unit that captured the image in which the moving body was detected to be superimposed with an image indicating the direction of the moving body as an image related to the object on the image of the imaging unit that is located in the direction of movement of the moving body, among the imaging units that are adjacent to both sides of the imaging unit that captured the image in which the moving body was detected. (Configuration 4) The control device according to configuration 1, characterized in that the control unit controls to superimpose an image indicating the direction of the object as an image related to the object on an image of an imaging unit that captures an imaging area closer to the imaging area of the imaging unit that captured the image in which the object was detected, out of the imaging units on either side of the imaging unit that captured the image in which the object was detected. (Configuration 5) The control unit when the object is detected in a right region of the image in which the object is detected, control is performed so that an image indicating the direction of the object is superimposed on an image of an imaging unit adjacent to the right of the imaging unit that captured the image in which the object is detected, as an image related to the object; The control device according to configuration 1, characterized in that when the object is detected in the left region of the image in which the object is detected, an image indicating the direction of the object is superimposed on the image of the imaging unit adjacent to the left of the imaging unit that captured the image in which the object is detected, as an image related to the object. (Configuration 6) The control unit controlling the image of one of the imaging units adjacent to both sides of the imaging unit that captured the image in which the object was detected to be displayed at a position laterally adjacent to the image in which the object was detected; controlling the image of the other of the imaging units adjacent to both sides of the imaging unit that captured the image in which the object was detected to be displayed in a position that is not laterally adjacent to the image in which the object was detected; 2. The control device according to configuration 1, wherein the control device controls to superimpose an image indicating a direction of the object as an image relating to the object on the image from one of the imaging units. (Configuration 7) One of the imaging units adjacent to both sides of the imaging unit that captured the image in which the object was detected is not movable on the same circumference in a direction approaching the imaging unit that captured the image in which the object was detected, and When the other imaging unit of the imaging units adjacent to both sides of the imaging unit that captured the image in which the object was detected is movable on the same circumference in a direction approaching the imaging unit that captured the image in which the object was detected, 2. The control device according to configuration 1, wherein the control unit controls the display of an image relating to the object, the image indicating the direction of the object, in a superimposed manner on the image captured by the other imaging unit. (Configuration 8) 8. The control device according to any one of configurations 1 to 7, wherein the image relating to the object is an arrow. (Configuration 9) 9. The control device according to any one of configurations 1 to 8, wherein the image relating to the object is a first graphical user interface that can be operated by a user. (Configuration 10) The control device described in configuration 9, characterized in that when the first graphical user interface is operated, the control unit instructs the imaging unit to move the shooting area corresponding to the image in which the operated first graphical user interface is displayed closer to the shooting area of the imaging unit that captured the image in which the object is detected. (Configuration 11) The control device described in configuration 10, characterized in that when the first graphical user interface is operated, the control unit instructs the pan angle of the imaging unit corresponding to the image in which the operated first graphical user interface is displayed to approach the pan angle of the imaging unit that captured the image in which the object is detected. (Configuration 12) The control device described in configuration 11, characterized in that when the first graphical user interface is operated, the control unit instructs the imaging unit corresponding to the image in which the operated first graphical user interface is displayed to change at least one of the tilt angle, zoom magnification, focus position, and rotation angle to be the same as those of the imaging unit that captured the image in which the object is detected. (Configuration 13) The control device according to any one of configurations 10 to 12, wherein when the first graphical user interface is operated, the control unit instructs the imaging unit corresponding to the image in which the operated first graphical user interface is displayed to have at least one of exposure, gain, shutter speed, white balance, and gamma be the same as those of the imaging unit that captured the image in which the object is detected. (Configuration 14) The control unit controlling the image of one of the imaging units adjacent to both sides of the imaging unit that captured the image in which the object was detected to be displayed at a position adjacent in the horizontal direction to the image in which the object was detected; controlling the image of the other of the imaging units adjacent to both sides of the imaging unit that captured the image in which the object was detected to be displayed in a position that is not laterally adjacent to the image in which the object was detected; A control device according to any one of configurations 10 to 13, characterized in that when the first graphical user interface displayed on the image of the other imaging unit is operated, the control device controls the image of the other imaging unit to be displayed in a position horizontally adjacent to the image in which the object is detected, and controls the image of one imaging unit to be displayed in a position that is not horizontally adjacent to the image in which the object is detected. (Configuration 15) The control device according to any one of configurations 10 to 14, characterized in that the control unit changes the arrangement of images of the multiple imaging units displayed on the display unit after the object is detected or after the first graphical user interface is operated. (Configuration 16) the object is a moving object, when the moving object moves from an image of an imaging unit in which the moving object was detected to an image of another imaging unit, the detection unit detects the moving object in the image of the other imaging unit; The control device according to any one of configurations 1 to 15, characterized in that the control unit controls the image of the imaging unit adjacent to the other imaging unit to be superimposed with an image indicating the direction of the moving body as an image related to the object. (Configuration 17) The control unit When the first graphical user interface is operated, controlling the display unit to display a second graphical user interface that is operable by the user; The control device according to any one of configurations 10 to 15, characterized in that when the second graphical user interface is operated, the control device instructs the position of the imaging area of the imaging unit corresponding to the operated first graphical user interface to be restored to its original position. (Configuration 18) The control device according to any one of configurations 1 to 17, characterized in that when the shooting area of the imaging unit that captured the image in which the object was detected moves, the control unit instructs the adjacent imaging unit to also move in conjunction with the shooting area of the imaging unit that captured the image in which the object was detected. (Configuration 19) The control device according to any one of configurations 1 to 18, characterized in that when the distance between the imaging unit that captured the image in which the object was detected and the adjacent imaging unit is shorter than a threshold, the control unit controls to superimpose an image indicating the direction of the object as an image related to the object. (Configuration 20) The control device according to any one of configurations 1 to 19, characterized in that the control unit controls to highlight both or either the image in which the object is detected and the image of the adjacent imaging unit. (Configuration 21) 21. The control device according to any one of configurations 1 to 20, wherein the plurality of imaging units are built into the control device. (Configuration 22) 21. The control device according to any one of configurations 1 to 20, wherein the plurality of imaging units are provided outside the control device. (Configuration 23) 23. The control device according to any one of configurations 1 to 22, wherein the plurality of imaging units are capable of independently changing pan angles on the same circumference. (Method 1) a detection step of detecting an object from each of the images captured by the plurality of imaging units; a control step of controlling a display unit to display an image in which the object is detected and an image from an imaging unit adjacent to the imaging unit that captured the image in which the object is detected, and to superimpose an image related to the object on the image from the adjacent imaging unit; A control method for a control device, comprising: (Program 1) A program for causing a computer to function as the control device according to any one of configurations 1 to 23. [Explanation of symbols]

[0206] 100 imaging device; 110, 120, 130, 140 imaging unit; 113, 123, 133, 143 drive unit; 114, 124, 134, 144 pan drive unit; 115, 125, 135, 145 zoom drive unit; 116, 126, 136, 146 tilt drive unit; 117, 127, 137, 147 rotation drive unit; 118, 128, 138, 148 focus drive unit; 151 image processing unit; 152, 202 control unit; 153, 201 communication unit; 154, 205 recording unit; 200 client device; 203 display unit; 204 instruction unit

Claims

1. a detection unit that detects an object from each of the images captured by the plurality of imaging units; a control unit that controls a display unit to display an image in which the object is detected and an image of an imaging unit adjacent to the imaging unit that captured the image in which the object is detected, and to superimpose an image related to the object on the image of the adjacent imaging unit; A control device comprising:

2. The control device according to claim 1 , wherein the control unit controls the display of an image relating to the object to be superimposed on images of the imaging units adjacent to both sides of the imaging unit that captured the image in which the object was detected.

3. the object is a moving object, the detection unit detects a moving direction of the moving object; The control device described in claim 1, characterized in that the control unit controls the image of the object to be superimposed on the image of the imaging unit located in the direction of movement of the moving body, among the imaging units adjacent to either side of the imaging unit that captured the image in which the moving body was detected, to display an image indicating the direction of the moving body.

4. The control device according to claim 1, characterized in that the control unit controls the display of an image indicating the direction of the object as an image related to the object, superimposed on an image of the imaging unit that captures the imaging area closer to the imaging area of the imaging unit that captured the image in which the object was detected, out of the imaging units on either side of the imaging unit that captured the image in which the object was detected.

5. The control unit when the object is detected in a right region of the image in which the object is detected, control is performed so that an image indicating the direction of the object is superimposed on an image of an imaging unit adjacent to the right of the imaging unit that captured the image in which the object is detected, as an image related to the object; The control device described in claim 1, characterized in that when the object is detected in the left area of the image in which the object is detected, an image indicating the direction of the object is superimposed on the image of the imaging unit adjacent to the left of the imaging unit that captured the image in which the object was detected, as an image related to the object.

6. The control unit controlling the image of one of the imaging units adjacent to both sides of the imaging unit that captured the image in which the object was detected to be displayed at a position laterally adjacent to the image in which the object was detected; controlling the image of the other of the imaging units adjacent to both sides of the imaging unit that captured the image in which the object was detected to be displayed in a position that is not laterally adjacent to the image in which the object was detected; 2. The control device according to claim 1, wherein the control device controls to superimpose an image indicating a direction of the object on the image captured by one of the imaging units as an image relating to the object.

7. One of the imaging units adjacent to both sides of the imaging unit that captured the image in which the object was detected is not movable on the same circumference in a direction approaching the imaging unit that captured the image in which the object was detected, and When the other imaging unit of the imaging units adjacent to both sides of the imaging unit that captured the image in which the object was detected is movable on the same circumference in a direction approaching the imaging unit that captured the image in which the object was detected, The control device according to claim 1 , wherein the control unit controls the display so that an image indicating a direction of the object is superimposed on the image captured by the other imaging unit as an image relating to the object.

8. The control device according to claim 1 , wherein the image relating to the object is an arrow.

9. The control device according to claim 1 , wherein the image of the object is a first graphical user interface that can be operated by a user.

10. 10. The control device according to claim 9, wherein, when the first graphical user interface is operated, the control unit instructs the imaging unit to move a shooting area corresponding to the image in which the operated first graphical user interface is displayed closer to a shooting area of the imaging unit that captured the image in which the object is detected.

11. 11. The control device according to claim 10, wherein, when the first graphical user interface is operated, the control unit instructs the pan angle of the imaging unit corresponding to the image in which the operated first graphical user interface is displayed to approach the pan angle of the imaging unit that captured the image in which the object is detected.

12. 12. The control device according to claim 11, wherein, when the first graphical user interface is operated, the control unit instructs the imaging unit corresponding to the image in which the operated first graphical user interface is displayed to change at least one of a tilt angle, a zoom magnification, a focus position, and a rotation angle to be the same as those of the imaging unit that captured the image in which the object is detected.

13. 11. The control device according to claim 10, wherein, when the first graphical user interface is operated, the control unit instructs the imaging unit corresponding to the image in which the operated first graphical user interface is displayed to have at least one of exposure, gain, shutter speed, white balance, and gamma set to be the same as those of the imaging unit that captured the image in which the object is detected.

14. The control unit controlling the image of one of the imaging units adjacent to both sides of the imaging unit that captured the image in which the object was detected to be displayed at a position laterally adjacent to the image in which the object was detected; controlling the image of the other of the imaging units adjacent to both sides of the imaging unit that captured the image in which the object was detected to be displayed in a position that is not laterally adjacent to the image in which the object was detected; The control device described in claim 10, characterized in that when the first graphical user interface displayed on the image of the other imaging unit is operated, the control device controls the image of the other imaging unit to be displayed in a position horizontally adjacent to the image in which the object is detected, and controls the image of the one imaging unit to be displayed in a position that is not horizontally adjacent to the image in which the object is detected.

15. 11. The control device according to claim 10, wherein the control unit changes an arrangement of the images of the plurality of imaging units displayed on the display unit after the object is detected or after the first graphical user interface is operated.

16. the object is a moving object, when the moving object moves from an image of an imaging unit in which the moving object was detected to an image of another imaging unit, the detection unit detects the moving object in the image of the other imaging unit; The control device according to claim 1 , wherein the control unit controls the display so that an image indicating the direction of the moving body is superimposed on the image of the imaging unit adjacent to the other imaging unit as an image related to the object.

17. The control unit When the first graphical user interface is operated, control is performed on the display unit to display a second graphical user interface that is operable by the user; 11. The control device according to claim 10, wherein, when the second graphical user interface is operated, an instruction is given to return the position of the imaging area of the imaging unit corresponding to the operated first graphical user interface to its original position.

18. The control device according to claim 1, wherein when the photographing area of the imaging unit that captured the image in which the object was detected moves, the control unit instructs the adjacent imaging unit to also move in conjunction with the photographing area of the imaging unit that captured the image in which the object was detected.

19. The control device according to claim 1, characterized in that the control unit controls the display of an image indicating the direction of the object to be superimposed as an image related to the object when the distance between the imaging unit that captured the image in which the object was detected and the adjacent imaging unit is shorter than a threshold value.

20. The control device according to claim 1 , wherein the control unit controls to highlight both or either one of the image in which the object is detected and the image captured by the adjacent imaging unit.

21. The control device according to claim 1 , wherein the plurality of image capturing units are built into the control device.

22. The control device according to claim 1 , wherein the plurality of image capturing units are provided outside the control device.

23. The control device according to claim 1 , wherein the pan angles of the plurality of imaging units on the same circumference can be changed independently.

24. a detection step of detecting an object from each of the images captured by the plurality of imaging units; a control step of controlling a display unit to display an image in which the object is detected and an image from an imaging unit adjacent to the imaging unit that captured the image in which the object is detected, and to superimpose an image related to the object on the image from the adjacent imaging unit; A control method for a control device, comprising:

25. A program for causing a computer to function as the control device according to any one of claims 1 to 23.

Images