Control device, imaging system, control method, and control program

Abstract

The invention provides a control device, an imaging system, a control method, and a control program capable of imaging in a manner that a moving body does not deviate from the field angle of the imaging device. A control device (60) is provided with a CPU6 (0A) for controlling a camera (10) and a swing mechanism (16) that swings the camera (10). A CPU 6 (0A) detects a moving body from a captured image in which captured image data acquired by a camera (10) has been output, acquires first position information relating to the position of the moving body in the captured image, acquires second position information relating to the turning position of a turning mechanism (16), and performs turning control of the turning mechanism (16) on the basis of the first position information and the second position information. The moving body is included in the imaging range of the camera (10). The detection of the moving body is performed even during the rotation operation of the rotation mechanism (16).

Description

Technical Field

[0001] The invention relates to a control device, a camera system, a control method and a control program. Background Art

[0002] Patent document 1 describes an automatic tracking photography system, which includes a control terminal, wherein the control terminal has: a target position prediction unit, which predicts the target position after a control delay time; and a control amount calculation unit, which calculates the control amount of camera translation and tilt required to move the center of the photographic image to the predicted target position. The control terminal remotely controls the camera to automatically track the target while shooting.

[0003] Patent document 2 describes an automatic tracking device, which includes: a position storage device, which converts the position of the tracking point of an object of the tracking object at different times into a position on an absolute coordinate using the translation angle / tilt angle obtained from a rotation device, and then stores the converted position in a time series; a speed calculation device and a position estimation device, which estimate the moving speed and the position of the tracking point at the next moment based on the position of the tracking point on the absolute coordinate stored in the time series; and a rotation control device, which calculates the translation / tilt control value that can capture the position of the tracking point at the next moment in the center of the image, and performs feedback control on the rotation device.

[0004] Patent document 3 describes a remote control pan-tilt system, which measures the time from when a pan-tilt controller sends a virtual signal to the pan-tilt until the pan-tilt controller receives a response signal from the pan-tilt, and sets a delay waiting time for the pan-tilt controller to wait for receiving a response signal from the pan-tilt based on the time, thereby setting a delay waiting time with high reliability that corresponds to the actual line conditions.

[0005] Previous technical literature

[0006] Patent Literature

[0007] Patent Document 1: International Publication No. 2016 / 151925

[0008] Patent Document 2: Japanese Patent Application Publication No. 2001-285850

[0009] Patent Document 3: Japanese Patent Application Publication No. 2001-333320 Summary of the invention

[0010] Technical issues to be solved by the invention

[0011] One embodiment of the technology according to the present invention provides a control device, an imaging system, a control method, and a control program capable of imaging a moving object without deviating from the field of view of an imaging device.

[0012] Means for solving technical problems

[0013] (1) A control device comprising a processor for controlling an imaging device and a rotating device for rotating the imaging device, wherein:

[0014] The above processor performs the following processing:

[0015] detecting a moving object from a captured image outputted from captured image data acquired by the imaging device;

[0016] acquiring first position information related to the position of the moving object in the camera image;

[0017] Acquiring second position information related to the rotation position of the above-mentioned rotating device;

[0018] Based on the first position information and the second position information, the rotation device is controlled so that the moving object is included in the imaging range of the imaging device;

[0019] The above detection is performed even during the rotation operation of the above-mentioned rotation device.

[0020] (2) The control device according to (1), wherein:

[0021] The above detection is performed in parallel with the above swing control.

[0022] (3) The control device according to (1), wherein:

[0023] The imaging device performs imaging control related to the imaging range.

[0024] (4) The control device according to (1), wherein:

[0025] The detection is performed during a period from the start to the end of the swing control.

[0026] (5) The control device according to (4), wherein:

[0027] The end of the slewing control is a state in which the change of the captured image due to the slewing control is terminated.

[0028] (6) The control device according to (1), wherein:

[0029] The detection is performed during the period between the control commands for the swing control.

[0030] (7) The control device according to any one of (1) to (6), wherein:

[0031] The above detection and the above swing control are performed in parallel and repeatedly.

[0032] (8) The control device according to (7), wherein:

[0033] The detection is performed every time the captured image data is acquired by the imaging device.

[0034] (9) The control device according to any one of (1) to (8), wherein:

[0035] The above processor performs the following processing:

[0036] Obtaining a third motion vector which is a motion vector of the moving object in a space where the moving object exists, based on a first motion vector which is a motion vector of the moving object in the imaging range and a second motion vector which is a motion vector of the imaging range based on the rotation control;

[0037] acquiring third position information related to the position of the moving object based on the third motion vector; and

[0038] The rotation of the above-mentioned rotating device is controlled based on the above-mentioned third position information.

[0039] (10) The control device according to (9), wherein:

[0040] The above processor performs the following processing:

[0041] The first motion vector and the second motion vector are acquired from the captured image.

[0042] (11) The control device according to (10), wherein:

[0043] The above processor performs the following processing:

[0044] The first motion vector and the second motion vector are acquired based on the feature points of the captured image.

[0045] (12) The control device according to (11), wherein:

[0046] The above processor performs the following processing:

[0047] The second movement vector is acquired based on the feature point determined by the historical information of the slewing control.

[0048] (13) The control device according to (9), wherein:

[0049] The above processor performs the following processing:

[0050] The second movement vector is acquired using the control command of the swing control and the history information of the swing control.

[0051] (14) The control device according to any one of (1) to (13), wherein:

[0052] The above processor performs the following processing:

[0053] The second swivel control of the swivel device is performed at a point in time before the change of the captured image based on the first swivel control of the swivel device is completed.

[0054] (15) The control device according to (14), wherein:

[0055] The above processor performs the following processing:

[0056] The second swing control is performed based on a fourth motion vector of the imaging range based on the control value of the first swing control and a fifth motion vector of the imaging range based on the captured image.

[0057] (16) The control device according to any one of (1) to (15), wherein:

[0058] The above processor performs the following control:

[0059] The rotation speed of the rotation device is determined according to the imaging conditions of the imaging device.

[0060] (17) The control device according to any one of (1) to (16), wherein:

[0061] The camera device and the rotating device are connected via a network.

[0062] (18) A camera system comprising:

[0063] Camera device;

[0064] A rotating device for rotating the camera device; and

[0065] A control device controls the camera device and the rotating device.

[0066] The above control device performs the following processing:

[0067] detecting a moving object from a captured image outputted from captured image data acquired by the imaging device;

[0068] acquiring first position information related to the position of the moving object in the camera image;

[0069] Acquiring second position information related to the rotation position of the above-mentioned rotating device;

[0070] Based on the first position information and the second position information, the rotation device is controlled so that the moving object is included in the imaging range of the imaging device;

[0071] The above detection is performed even during the rotation operation of the above-mentioned rotation device.

[0072] (19) A control method is performed by a control device including a processor for controlling an imaging device and a rotating device for rotating the imaging device, wherein:

[0073] The above processor performs the following processing:

[0074] detecting a moving object from a captured image outputted from captured image data acquired by the imaging device;

[0075] acquiring first position information related to the position of the moving object in the camera image;

[0076] Acquiring second position information related to the rotation position of the above-mentioned rotating device;

[0077] Based on the first position information and the second position information, the rotation device is controlled so that the moving object is included in the imaging range of the imaging device;

[0078] The above detection is performed even during the rotation operation of the above-mentioned rotation device.

[0079] (20) A control program for a control device having a processor for controlling an imaging device and a rotating device for rotating the imaging device, the control program causing the processor to execute the following processing:

[0080] detecting a moving object from a captured image outputted from captured image data acquired by the imaging device;

[0081] acquiring first position information related to the position of the moving object in the camera image;

[0082] Acquiring second position information related to the rotation position of the above-mentioned rotating device;

[0083] Based on the first position information and the second position information, the rotation device is controlled so that the moving object is included in the imaging range of the imaging device;

[0084] The above detection is performed even during the rotation operation of the above-mentioned rotation device.

[0085] Effects of the Invention

[0086] According to the present invention, it is possible to provide a control device, an imaging system, a control method, and a control program that can capture an image without causing a moving object to deviate from the angle of view of an imaging device. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 This is a diagram showing an example of an imaging system 1 equipped with a control device 60 according to the present embodiment.

[0088] Figure 2 This is a diagram showing an example of rotation of the camera 10 in the pitch direction by the rotation mechanism 16 .

[0089] Figure 3 FIG. 1 is a diagram showing an example of rotation of the camera 10 in the deflection direction by the rotation mechanism 16.

[0090] Figure 4 It is a block diagram showing an example of the configuration of the optical system and the electrical system of the camera 10 .

[0091] Figure 5 It is a diagram showing an example of the configuration of the electric system of the turning mechanism 16 and the management device 11 .

[0092] Figure 6 This is a diagram showing an example of management processing in a conventional imaging system as a reference.

[0093] Figure 7 It is a diagram showing an example of management processing of the imaging system 1 according to the present embodiment.

[0094] Figure 8 This is a diagram showing an example of movement position prediction based on a movement vector of a moving object.

[0095] Fig. 9 FIG. 1 is a diagram showing an example of movement position prediction based on feature points specified by history information.

[0096] Fig.10 It is a diagram showing a modification of the management process of the imaging system 1 according to the present embodiment.

[0097] Fig.11 This is a diagram showing an example of a manner in which the information processing program is installed in the control device 60 of the management device 11 from a storage medium storing the information processing program for management control. DETAILED DESCRIPTION

[0098] Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings.

[0099] <Image capture system of embodiment>

[0100] Figure 1FIG. 1 is a diagram showing an example of an imaging system 1 equipped with a control device according to the present embodiment. Figure 1 As shown, as an example, the imaging system 1 includes a camera 10 and a management device 11. The camera 10 and the management device 11 are connected via a network 20. The camera 10 is an example of an imaging device in the present invention.

[0101] The camera 10 is a camera for photographing a moving object such as a drone, a vehicle, or a person. The camera 10 uses a camera capable of telephoto photography, an ultra-high-resolution camera, or the like. In addition, the camera 10 may also use a wide-angle camera. The camera 10 is set via a rotation mechanism 16 described later, and photographs a photographing object as a subject. The camera 10 sends a photographed image obtained by photographing and photographing information related to the photographing to the management device 11 via a network 20.

[0102] The management device 11 includes a display 13a, a keyboard 13b, a mouse 13c, and a secondary storage device 14. Examples of the display 13a include a liquid crystal display, a plasma display, an organic EL (Electro-Luminescence) display, and a CRT (Cathode Ray Tube) display.

[0103] An example of the secondary storage device 14 is a hard disk drive (HDD). The secondary storage device 14 is not limited to a HDD, and may be any nonvolatile memory such as a flash memory, a solid state drive (SSD), or an electrically erasable and programmable read-only memory (EEPROM).

[0104] The management device 11 receives the captured image or the captured image information transmitted from the camera 10 , and displays the received captured image or the captured image information on the display 13 a or stores the received captured image or the captured image information in the secondary storage device 14 .

[0105] The management device 11 performs imaging control based on the shooting of the camera 10. For example, the management device 11 performs imaging control by communicating with the camera 10 via the network 20. The imaging control is to set imaging parameters for the camera 10 to perform imaging in the camera 10 and control the camera 10 to perform imaging. The imaging parameters include parameters related to exposure and parameters of zoom position, etc.

[0106] Furthermore, the management device 11 controls the rotation mechanism 16 to control the shooting direction (pan or tilt) of the camera 10. For example, the management device 11 sets the rotation direction, rotation amount, rotation speed, etc. of the camera 10 according to the operation of the keyboard 13b or the mouse 13c or the touch operation on the screen of the display 13a. The management device 11 is connected to the rotation mechanism 16 via the network 20. The management device 11 controls the rotation mechanism 16 via the network 20.

[0107] <Rotation of Camera 10 by Rotation Mechanism 16>

[0108] Figure 2 This is a diagram showing an example of rotation of the camera 10 in the pitch direction by the rotation mechanism 16 . Figure 3 1 is a diagram showing an example of rotation of the camera 10 in the yaw direction by the rotation mechanism 16. The camera 10 is mounted on the rotation mechanism 16. The rotation mechanism 16 can change the imaging direction of the camera 10 by rotating the camera 10. The rotation mechanism 16 is an example of a rotation device in the present invention.

[0109] Specifically, if Figure 2 As shown in FIG. 1 , as an example, the rotation mechanism 16 is capable of rotating the camera 10 in a rotation direction (pitch direction) that intersects the yaw direction and has the pitch axis PA as the center axis, and in a rotation direction (pitch direction) that intersects the yaw direction and Figure 3 As an example, a double-axis rotating mechanism is shown which rotates in the rotation direction (deflection direction) with the deflection axis YA as the central axis. In addition, in the rotating mechanism 16 involved in this embodiment, an example of a double-axis rotating mechanism is shown, but the technology of the present invention is not limited to this, and it can be a three-axis rotating mechanism or a single-axis rotating mechanism.

[0110] <Configuration of the Optical and Electrical Systems of Camera 10>

[0111] Figure 4 1 is a block diagram showing an example of the structure of the optical system and the electrical system of the camera 10. Figure 4 As shown, as an example, the camera 10 includes an optical system 15 and an imaging element 25. The imaging element 25 is located at the rear section of the optical system 15. The optical system 15 includes an objective lens 15A and a lens group 15B. The objective lens 15A and the lens group 15B are arranged in the order of the objective lens 15A and the lens group 15B along the optical axis OA of the optical system 15 from the object subject side (object side) to the light receiving surface 25A side (image side) of the imaging element 25. The lens group 15B includes an anti-vibration lens 15B1, a focusing lens (not shown) and a zoom lens 15B2. The zoom lens 15B2 is supported so as to be movable along the optical axis OA by a lens actuator 21 described later. The anti-vibration lens 15B1 is supported so as to be movable in a direction orthogonal to the optical axis OA by a lens actuator 17 described later.

[0112] The zoom lens 15B2 increases the focal length and sets the camera 10 to the telephoto side, so the angle of view decreases (the imaging range becomes narrower). The zoom lens 15B2 decreases the focal length and sets the camera 10 to the wide-angle side, so the angle of view increases (the imaging range becomes wider).

[0113] In addition, the optical system 15 may include various lenses not shown in the figure in addition to the objective lens 15A and the lens group 15B. In addition, the optical system 15 may include an aperture. The positions of the lenses, lens groups, and apertures included in the optical system 15 are not limited. Figure 4 The technology of the present invention is also applicable to positions different from the positions shown.

[0114] The anti-vibration lens 15B1 is movable in a direction perpendicular to the optical axis OA, and the zoom lens 15B2 is movable along the optical axis OA.

[0115] The optical system 15 includes lens actuators 17 and 21. The lens actuator 17 causes a force to be applied to the anti-vibration lens 15B1 to cause the force to be varied in a direction perpendicular to the optical axis of the anti-vibration lens 15B1. The lens actuator 17 is controlled by an OIS (Optical Image Stabilizer) driver 23. By driving the lens actuator 17 under the control of the OIS driver 23, the position of the anti-vibration lens 15B1 varies in a direction perpendicular to the optical axis OA.

[0116] The lens actuator 21 causes a force for moving along the optical axis OA of the optical system 15 to act on the zoom lens 15B2. The lens actuator 21 is controlled by the lens driver 28. The position of the zoom lens 15B2 moves along the optical axis OA by driving the lens actuator 21 under the control of the lens driver 28. The position of the zoom lens 15B2 moves along the optical axis OA, whereby the focal length of the camera 10 changes.

[0117] In addition, when the outline of the camera image is, for example, a rectangle having short sides in the pitch axis PA direction and long sides in the yaw axis YA direction, the field angle in the pitch axis PA direction is narrower than the field angle in the yaw axis YA direction and narrower than the diagonal field angle.

[0118] By the optical system 15 configured in this way, light representing the imaging target region is imaged on the light receiving surface 25A of the imaging element 25 , and the imaging target region is captured by the imaging element 25 .

[0119] The vibrations applied to the camera 10 include, for example, the vibrations caused by the passing of cars, the vibrations caused by wind, and the vibrations caused by road construction outdoors, and for example, the vibrations caused by the operation of an air conditioner and the vibrations caused by people coming in and out indoors. Therefore, the camera 10 is shaken due to the vibrations applied to the camera 10 (hereinafter, also simply referred to as "vibrations").

[0120] In addition, in the present embodiment, "jitter" refers to a phenomenon in which the image of the object subject on the light receiving surface 25A of the imaging element 25 in the camera 10 changes due to a change in the positional relationship between the optical axis OA and the light receiving surface 25A. In other words, "jitter" can also be said to be a phenomenon in which the optical image obtained by imaging on the light receiving surface 25A changes due to the tilt of the optical axis OA caused by the vibration applied to the camera 10. The change of the optical axis OA refers to, for example, the tilt of the optical axis OA relative to the reference axis (for example, the optical axis OA before the jitter occurs). Hereinafter, the jitter caused by vibration is also referred to as "jitter".

[0121] The shaking is included in the captured image as a noise component and affects the image quality of the captured image. Therefore, in order to remove the noise component caused by the shaking and included in the captured image, the camera 10 includes a lens-side shaking correction mechanism 29, an imaging element-side shaking correction mechanism 45, and an electronic shaking correction unit 33 for correcting the shaking.

[0122] The lens-side shake correction mechanism 29 and the imaging element-side shake correction mechanism 45 are mechanical shake correction mechanisms. The mechanical shake correction mechanism is a mechanism that corrects shake by applying power generated by a driving source such as a motor (for example, a voice coil motor) to a shake correction element (for example, the anti-vibration lens 15B1 and / or the imaging element 25) to move the shake correction element in a direction perpendicular to the optical axis of the imaging optical system.

[0123] Specifically, the lens-side shake correction mechanism 29 is a mechanism that corrects the shake by applying power generated by a driving source such as a motor (for example, a voice coil motor) to the anti-vibration lens 15B1, thereby moving the anti-vibration lens 15B1 in a direction perpendicular to the optical axis of the imaging optical system. The imaging element-side shake correction mechanism 45 is a mechanism that corrects the shake by applying power generated by a driving source such as a motor (for example, a voice coil motor) to the imaging element 25, thereby moving the imaging element 25 in a direction perpendicular to the optical axis of the imaging optical system. The electronic shake correction unit 33 performs image processing on the captured image according to the amount of shake, thereby correcting the shake. That is, the shake correction unit (shake correction component) performs shake correction mechanically or electronically through a hardware structure and / or a software structure. Here, mechanical shake correction refers to shake correction achieved by mechanically moving shake correction elements such as the anti-vibration lens 15B1 and / or the imaging element 25 using power generated by a driving source such as a motor (e.g., a voice coil motor), and electronic shake correction refers to shake correction achieved, for example, by performing image processing using a processor.

[0124] like Figure 4 As shown, the lens-side image stabilization mechanism 29 includes, as an example, an anti-vibration lens 15B1 , a lens actuator 17 , an OIS driver 23 , and a position sensor 39 .

[0125] As the shake correction method by the lens-side shake correction mechanism 29, various well-known methods can be adopted. In the present embodiment, as the shake correction method, a method is adopted in which the anti-vibration lens 15B1 is moved according to the shake amount detected by the shake amount detection sensor 40 (described later) to correct the shake. Specifically, the anti-vibration lens 15B1 is moved in the direction to eliminate the shake only by the amount to eliminate the shake, thereby performing the shake correction.

[0126] The anti-vibration lens 15B1 is provided with a lens actuator 17. The lens actuator 17 is a displacement mechanism equipped with a voice coil motor, and drives the voice coil motor to move the anti-vibration lens 15B1 in a direction perpendicular to the optical axis of the anti-vibration lens 15B1. In addition, here, as the lens actuator 17, a displacement mechanism equipped with a voice coil motor is used, but the technology of the present invention is not limited to this, and other power sources such as a stepping motor or a piezoelectric element can also be used instead of the voice coil motor.

[0127] The lens actuator 17 is controlled by the OIS driver 23. By driving the lens actuator 17 under the control of the OIS driver 23, the position of the anti-vibration lens 15B1 is mechanically changed within a two-dimensional plane perpendicular to the optical axis OA.

[0128] The position sensor 39 detects the current position of the anti-vibration lens 15B1 and outputs a position signal indicating the detected current position. Here, as an example of the position sensor 39, a device including a Hall element is used. Here, the current position of the anti-vibration lens 15B1 refers to the current position within the two-dimensional plane of the anti-vibration lens. The two-dimensional plane of the anti-vibration lens refers to a two-dimensional plane perpendicular to the optical axis of the anti-vibration lens 15B1. In addition, in the present embodiment, as an example of the position sensor 39, a device including a Hall element is used, but the technology of the present invention is not limited to this, and a magnetic sensor or an optical sensor may be used instead of the Hall element.

[0129] The lens-side shake correction mechanism 29 corrects the shake by moving the anti-vibration lens 15B1 in at least one of the pitch axis PA direction and the yaw axis YA direction within the actual shooting range. That is, the lens-side shake correction mechanism 29 corrects the shake by moving the anti-vibration lens 15B1 within the two-dimensional plane of the anti-vibration lens by an amount of movement corresponding to the amount of shake.

[0130] The imaging element side shake correction mechanism 45 includes the imaging element 25 , a BIS (Body Image Stabilizer) driver 22 , an imaging element actuator 27 , and a position sensor 47 .

[0131] As with the shake correction method by the lens-side shake correction mechanism 29, the shake correction method by the imaging element-side shake correction mechanism 45 can also adopt various well-known methods. In the present embodiment, as the shake correction method, a method of correcting the shake by moving the imaging element 25 according to the shake amount detected by the shake amount detection sensor 40 is adopted. Specifically, the shake correction is performed by moving the imaging element 25 in the direction to cancel the shake by the amount to cancel the shake.

[0132] An imaging element actuator 27 is mounted on the imaging element 25. The imaging element actuator 27 is a displacement mechanism equipped with a voice coil motor, and by driving the voice coil motor, the imaging element 25 is moved in a vertical direction relative to the optical axis of the anti-vibration lens 15B1. In addition, here, as the imaging element actuator 27, a displacement mechanism equipped with a voice coil motor is used, but the technology of the present invention is not limited to this, and other power sources such as a stepping motor or a piezoelectric element can be applied instead of the voice coil motor.

[0133] The imaging element actuator 27 is controlled by the BIS driver 22. By driving the imaging element actuator 27 under the control of the BIS driver 22, the position of the imaging element 25 is mechanically changed in a direction perpendicular to the optical axis OA.

[0134] The position sensor 47 detects the current position of the imaging element 25 and outputs a position signal indicating the detected current position. Here, as an example of the position sensor 47, a device including a Hall element is used. Here, the current position of the imaging element 25 refers to the current position within the two-dimensional plane of the imaging element. The two-dimensional plane of the imaging element refers to a two-dimensional plane perpendicular to the optical axis of the anti-vibration lens 15B1. In addition, in the present embodiment, as an example of the position sensor 47, a device including a Hall element is used, but the technology of the present invention is not limited to this, and a magnetic sensor or an optical sensor may be used instead of the Hall element.

[0135] The camera 10 includes a computer 19, a DSP (Digital Signal Processor) 31, an image memory 32, an electronic shake correction unit 33, a communication I / F 34, a shake amount detection sensor 40, and a UI (User Interface) system device 43. The computer 19 includes a memory 35, a storage device 36, and a CPU (Central Processing Unit) 37.

[0136] The imaging element 25, DSP 31, image memory 32, electronic shake correction unit 33, communication I / F 34, memory 35, storage device 36, CPU 37, shake amount detection sensor 40, and UI system device 43 are connected to the bus 38. In addition, the OIS driver 23 is also connected to the bus 38. Figure 4 In the example shown, for convenience of illustration, one bus is shown as the bus 38, but a plurality of buses may be provided. The bus 38 may be a serial bus or a parallel bus such as a data bus, an address bus, and a control bus.

[0137] The memory 35 temporarily stores various information and is used as a working memory. As an example of the memory 35, a RAM (Random Access Memory) can be cited, but it is not limited to this and can be other types of storage devices. Various programs for the camera 10 are stored in the storage device 36. The CPU 37 reads various programs from the storage device 36 and executes the read various programs on the memory 35, thereby controlling the entire camera 10. As the storage device 36, for example, a flash memory, an SSD, an EEPROM, or an HDD can be cited. In addition, for example, various non-volatile memories such as a magnetoresistive memory and a ferroelectric memory can be used instead of the flash memory or in combination with the flash memory.

[0138] The imaging element 25 is a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The imaging element 25 captures the target subject at a predetermined frame rate under the instruction of the CPU 37. The "predetermined frame rate" mentioned here refers to, for example, tens of frames per second to hundreds of frames per second. In addition, a control device (imaging element control device) can also be built into the imaging element 25 itself. In this case, the imaging element control device performs detailed control inside the imaging element 25 according to the shooting instruction output by the CPU 37. In addition, the imaging element 25 can capture the target subject at a predetermined frame rate under the instruction of the DSP 31. In this case, the imaging element control device performs detailed control inside the imaging element 25 according to the shooting instruction output by the DSP 31. In addition, the DSP 31 is sometimes also referred to as an ISP (Image Signal Processor).

[0139] The light receiving surface 25A of the imaging element 25 is formed by a plurality of photosensitive pixels (not shown) arranged in a matrix. In the imaging element 25, each photosensitive pixel is exposed, and photoelectric conversion is performed for each photosensitive pixel. The charge obtained by photoelectric conversion for each photosensitive pixel is an analog imaging signal representing the object. Here, as the plurality of photosensitive pixels, a plurality of photoelectric conversion elements having sensitivity to visible light (for example, a photoelectric conversion element having a color filter) are used. In the imaging element 25, as the plurality of photoelectric conversion elements, a photoelectric conversion element having sensitivity to R (red) light (for example, a photoelectric conversion element having an R color filter corresponding to R), a photoelectric conversion element having sensitivity to G (green) light (for example, a photoelectric conversion element having a G color filter corresponding to G), and a photoelectric conversion element having sensitivity to B (blue) light (for example, a photoelectric conversion element having a B color filter corresponding to B) are used. In the camera 10, shooting based on visible light (for example, light on the short wavelength side of about 700 nanometers or less) is performed by using these photosensitive pixels. However, the present embodiment is not limited thereto, and photography based on infrared light (for example, light with a longer wavelength than about 700 nanometers) can be performed. In this case, as a plurality of photosensitive pixels, a plurality of photoelectric conversion elements having sensitivity to infrared light can be used. In particular, for photography related to SWIR (Short-wavelength infrared), for example, an InGaAs sensor and / or a Type-II quantum well (T2SL; Simulation of Type-IIQuantum Well) sensor can be used.

[0140] The imaging element 25 performs signal processing such as A / D (Analog / Digital) conversion on the analog imaging signal and generates a digital imaging signal, that is, a digital image. The imaging element 25 is connected to the DSP 31 via the bus 38 and outputs the generated digital image to the DSP 31 via the bus 38 in units of frames.

[0141] In addition, here, as an example of the imaging element 25, a CMOS image sensor is cited for explanation, but the technology of the present invention is not limited to this, and a CCD (Charge Coupled Device) image sensor can be applied as the imaging element 25. In this case, the imaging element 25 is connected to the bus 38 via an AFE (Analog Front End) not shown in the figure built into the CCD driver, and the AFE generates a digital image by performing signal processing such as A / D conversion on the analog camera signal obtained by the imaging element 25, and outputs the generated digital image to the DSP 31. The CCD image sensor is driven by a CCD driver built into the AFE. Of course, the CCD driver can be set separately.

[0142] The DSP 31 performs various digital signal processing on the digital image. The various digital signal processing refers to, for example, demosaic processing, noise removal processing, grayscale correction processing, and color correction processing. The DSP 31 outputs the digital image after the digital signal processing to the image memory 32 for each frame. The image memory 32 stores the digital image from the DSP 31.

[0143] The shake amount detection sensor 40 is a device including a gyro sensor, for example, and detects the shake amount of the camera 10. In other words, the shake amount detection sensor 40 detects the shake amount for each of a pair of axial directions. The gyro sensor detects the respective axes (reference numerals 14 and 15 ) around the pitch axis PA, the yaw axis YA, and the roll axis RA (axis parallel to the optical axis OA). Figure 1 The shake amount detection sensor 40 converts the rotational shake amount around the pitch axis PA and the rotational shake amount around the yaw axis YA detected by the gyro sensor into the shake amount in a two-dimensional plane parallel to the pitch axis PA and the yaw axis YA, thereby detecting the shake amount of the camera 10.

[0144] Here, a gyro sensor is cited as an example of the shake amount detection sensor 40, but this is only an example, and the shake amount detection sensor 40 may be an acceleration sensor. The acceleration sensor detects the shake amount in a two-dimensional plane parallel to the pitch axis PA and the yaw axis YA. The shake amount detection sensor 40 outputs the detected shake amount to the CPU 37.

[0145] Here, an example of a form in which the shake amount is detected by a physical sensor such as the shake amount detection sensor 40 is given, but the technology of the present invention is not limited to this. For example, a motion vector obtained by comparing the captured images before and after the time series stored in the image memory 32 can be used as the shake amount. In addition, the shake amount to be finally used can be derived from the shake amount detected by the physical sensor and the motion vector obtained by image processing.

[0146] The CPU 37 acquires the shake amount detected by the shake amount detection sensor 40 and controls the lens-side shake correction mechanism 29, the imaging element-side shake correction mechanism 45, and the electronic shake correction section 33 according to the acquired shake amount. The shake amount detected by the shake amount detection sensor 40 is used in the shake correction performed by each of the lens-side shake correction mechanism 29 and the electronic shake correction section 33.

[0147] The electronic shake correction unit 33 is a device including an ASIC (Application Specific Integrated Circuit) and corrects the shake by performing image processing on the captured image in the image memory 32 according to the shake amount detected by the shake amount detection sensor 40 .

[0148] In addition, here, as the electronic jitter correction unit 33, a device including an ASIC is exemplified, but the technology of the present invention is not limited to this. For example, it can be a device including an FPGA (Field Programmable Gate Array) or a PLD (Programmable Logic Device). And, for example, the electronic jitter correction unit 33 can be a device including multiple of ASIC, FPGA and PLD. And, as the electronic jitter correction unit 33, a computer including a CPU, a storage device and a memory can be used. The CPU can be single or multiple. And the electronic jitter correction unit 33 can be implemented by a combination of hardware structure and software structure.

[0149] The communication I / F 34 is, for example, a network interface, and controls the transmission of various information between the camera 10 and the management device 11 via the network 20. The network 20 is, for example, a WAN (Wide Area Network) such as the Internet or a LAN (Local Area Network). The communication I / F 34 performs communication between the camera 10 and the management device 11.

[0150] The UI system device 43 includes a receiving device 43A and a display 43B. The receiving device 43A is, for example, a hardware key or a touch panel, and receives various instructions from the user. The CPU 37 acquires various instructions received by the receiving device 43A and operates according to the acquired instructions.

[0151] The display 43B displays various information under the control of the CPU 37. Examples of the various information displayed on the display 43B include the contents of various instructions received by the receiving device 43A and captured images.

[0152] <Structure of the electrical system of the rotary mechanism 16 and the management device 11>

[0153] Figure 5 1 is a diagram showing an example of the structure of the electrical system of the rotary mechanism 16 and the management device 11. Figure 5 As shown, the swivel mechanism 16 includes a yaw axis swivel mechanism 71 , a pitch axis swivel mechanism 72 , motors 73 , 74 , drivers 75 , 76 , and communication I / Fs 79 , 80 , as an example.

[0154] The yaw axis rotation mechanism 71 rotates the camera 10 in the yaw direction. The motor 73 generates power by being driven under the control of the driver 75. The yaw axis rotation mechanism 71 rotates the camera 10 in the yaw direction by receiving the power generated by the motor 73. The pitch axis rotation mechanism 72 rotates the camera 10 in the pitch direction. The motor 74 generates power by being driven under the control of the driver 76. The pitch axis rotation mechanism 72 rotates the camera 10 in the pitch direction by receiving the power generated by the motor 74.

[0155] The communication I / Fs 79 and 80 are, for example, network interfaces, and control the transmission of various information with the management device 11 via the network 20. The network 20 is, for example, a WAN such as the Internet or a LAN. The communication I / Fs 79 and 80 perform communication between the rotation mechanism 16 and the management device 11.

[0156] like Figure 5 As shown, as an example, the management device 11 includes a display 13a, a secondary storage device 14, a control device 60, a receiving device 62, and communication I / Fs 66, 67, and 68. The control device 60 includes a CPU 60A, a storage device 60B, and a memory 60C. The CPU 60A is an example of a processor in the present invention.

[0157] Each of the receiving device 62, the display 13a, the secondary storage device 14, the CPU 60A, the storage device 60B, the memory 60C, and the communication I / F 66 is connected to the bus 70. Figure 5In the example shown, for convenience of illustration, one bus is shown as the bus 70, but a plurality of buses may be provided. The bus 70 may be a serial bus or a parallel bus including a data bus, an address bus, a control bus, and the like.

[0158] The memory 60C temporarily stores various information and is used as a working memory. As an example of the memory 60C, a RAM can be cited, but it is not limited to this and can be other types of storage devices. Various programs for the management device 11 (hereinafter referred to as "management device programs") are stored in the storage device 60B.

[0159] The CPU 60A reads out the management device program from the storage device 60B and executes the read management device program on the memory 60C, thereby controlling the entire management device 11. The management device program includes the control program in the present invention.

[0160] The communication I / F 66 is, for example, a network interface. The communication I / F 66 is communicably connected to the communication I / F 34 of the camera 10 via the network 20, and controls the transmission of various information with the camera 10. The communication I / Fs 67 and 68 are, for example, network interfaces. The communication I / F 67 is communicably connected to the communication I / F 79 of the rotation mechanism 16 via the network 20, and controls the transmission of various information with the yaw axis rotation mechanism 71. The communication I / F 68 is communicably connected to the communication I / F 80 of the rotation mechanism 16 via the network 20, and controls the transmission of various information with the pitch axis rotation mechanism 72.

[0161] The CPU 60A receives captured images, captured image information, and the like from the camera 10 via the communication I / F 66 and the communication I / F 34. The CPU 60A controls the image capturing operation of the image capturing object by the camera 10 via the communication I / F 66 and the communication I / F 34.

[0162] The CPU 60A controls the driver 75 and the motor 73 of the rotation mechanism 16 via the communication I / F 67 and the communication I / F 79, thereby controlling the rotation operation of the yaw axis rotation mechanism 71. Furthermore, the CPU 60A controls the driver 76 and the motor 74 of the rotation mechanism 16 via the communication I / F 68 and the communication I / F 80, thereby controlling the rotation operation of the pitch axis rotation mechanism 72.

[0163] The CPU 60A detects a moving object from a captured image that outputs captured image data acquired from the camera 10, and acquires first position information related to the position of the moving object in the captured image. The CPU 60A acquires second position information related to the rotation position of the rotating mechanism 16 that rotates the camera 10 from the rotating mechanism 16. The CPU 60A performs rotation control of the rotating mechanism 16 based on the first position information related to the position of the moving object and the second position information related to the rotation position of the rotating mechanism 16 so that the moving object is included in the imaging range of the camera 10. The CPU 60A performs detection of a moving object from a captured image even during the rotation operation of the rotating mechanism 16. That is, the CPU 60A performs detection of a moving object from a captured image in parallel with the rotation control of the rotating mechanism 16.

[0164] The CPU 60A performs detection of a moving object from a camera image during a period including from the start of the slewing control for rotating the slewing mechanism 16 to the end thereof. The end of the slewing control of the slewing mechanism 16 is a state in which the change of the camera image based on the slewing control is terminated. The CPU 60A performs detection of a moving object from a camera image during a period from after a control command for slewing the slewing mechanism 16 is output until the next control command is output. The CPU 60A repeatedly performs detection of a moving object from a camera image in parallel with the slewing control of the slewing mechanism 16. The CPU 60A performs detection of a moving object from a camera image every time a camera image captured by the camera 10 is acquired.

[0165] The receiving device 62 is, for example, a keyboard 13b, a mouse 13c, or a touch panel of the display 13a, and receives various instructions from the user. The CPU 60A obtains various instructions received by the receiving device 62 and operates according to the instructions obtained. For example, when the receiving device 62 receives processing content for the camera 10 and / or the rotating mechanism 16, the CPU 60A operates the camera 10 and / or the rotating mechanism 16 according to the instruction content received by the receiving device 62.

[0166] The display 13a displays various information under the control of the CPU 60A. Examples of the various information displayed on the display 13a include the contents of various instructions received by the receiving device 62 and the captured images or captured information received by the communication I / F 66. The CPU 60A displays the contents of various instructions received by the receiving device 62 and the captured images or captured information received by the communication I / F 66 on the display 13a.

[0167] The secondary storage device 14 is, for example, a nonvolatile memory, and stores various information under the control of the CPU 60A. Examples of the various information stored in the secondary storage device 14 include a captured image or captured information received by the communication I / F 66. The CPU 60A stores the captured image or captured information received by the communication I / F 66 in the secondary storage device 14.

[0168] <Processing of Management Device in Conventional Camera System>

[0169] Figure 6 This figure shows, as a reference, an example of management processing in which the CPU 60A of the management device 11 monitors a moving object by rotating and controlling the rotating mechanism 16 based on a captured image captured by the camera 10 in a conventional camera system. As described above, the camera 10 and the rotating mechanism 16 are connected to the management device 11 via the network 20 so as to be communicable.

[0170] The camera 10 is mounted on the rotating mechanism 16 and is set to face a predetermined imaging area as an imaging target. In the camera 10, for example, the zoom position of the zoom lens is set to the wide-angle end. Figure 6 As shown, the data of the captured image captured by the camera 10 is recorded in the memory 35 of the computer 19 mounted on the camera 10 for each frame, and is transmitted to the management device 11 via the network 20 .

[0171] The sent camera image of the camera 10 is displayed, for example, on the display 13a of the management device 11. When the camera image of the camera 10 is displayed on the display 13a, for example, when a moving object is detected in the camera image, the CPU 60A of the management device 11 starts the tracking monitoring process of the moving object in which the detected moving object is tracked as a monitoring object.

[0172] Furthermore, for example, a user (manager) may be in front of the management device 11 and observe the state of the camera image of the camera 10 displayed on the display 13a. The CPU 60A of the management device 11 may, for example, start a tracking monitoring process of tracking a camera object (for example, a moving object) specified by a camera object specifying operation from the user, while the camera image of the camera 10 is displayed on the display 13a. Specifically, when a moving object is captured in the camera image displayed on the display 13a, for example, when the user performs a touch operation on the moving object, the CPU 60A may start a tracking monitoring process of tracking the moving object specified by the touch operation as a moving object to be monitored.

[0173] The CPU 60A reads the captured image data sent from the camera 10, and records the read captured image data in the memory 60C mounted on the control device 60 or the secondary storage device 14. In addition, the CPU 60A performs a moving object detection 111a for detecting a moving object in the captured image based on the captured image of the output captured image data, for example, the captured image of the captured image data read by the input 110a.

[0174] The CPU 60A obtains the moving body position information related to the position of the moving body detected in the moving body detection 111a. For example, the CPU 60A can calculate the relationship between the coordinates of each position (point) on the camera image displayed on the display 13a and the translation / tilt value of the slewing mechanism 16 relative to the coordinates based on the size and position relationship of the camera image and each position (point). The calculated translation / tilt value is associated with the coordinates of each position (point) and stored as corresponding information in the memory 60C or the secondary storage device 14. The CPU 60A obtains the translation / tilt value corresponding to the position of the specified moving body as the moving body position information based on the pre-calculated corresponding information.

[0175] The CPU 60A acquires the swivel mechanism position information about the current swivel position of the swivel mechanism 16 that swivels the camera 10 that is photographing the moving object. The CPU 60A acquires the current pan / tilt value of the swivel mechanism 16 as the swivel mechanism position information.

[0176] The CPU 60A first performs the rotation control of the rotation mechanism 16 based on the moving object position information and the rotation mechanism position information so that the moving object detected by the moving object detection 111a is displayed in the central area of ​​the camera image taken by the camera 10. The CPU 60A sends information such as the translation / tilt value of the rotation mechanism 16 for displaying the moving object in the central area of ​​the camera image to the rotation mechanism 16 via the network 20 as a control instruction 121a for the rotation mechanism 16.

[0177] The rotating mechanism 16 starts a rotating operation 161 a in response to a control command 121 a sent from the CPU 60A of the management device 11 , and rotates the camera 10 in the commanded direction.

[0178] When the swivel operation 161a of the swivel mechanism 16 is started, the CPU 60A performs a swivel mechanism stop waiting 122a process to wait until the swivel mechanism 16 is swung to a commanded predetermined swivel position (pan / tilt value).

[0179] When the swing operation 161a of the swing mechanism 16 is completed and the swing mechanism stop waiting process 122a is terminated, the CPU 60A performs an image delay waiting process 123a to wait for the end of the image delay caused by transmitting the captured image via the network 20. The image delay time is set according to the environment of the network 20 and the like.

[0180] When the image delay is terminated and the image delay waiting 123a processing is completed, the CPU 60A performs the next moving body detection 111b for detecting a moving body in the camera image. The moving body detection 111b is started, for example, after a predetermined time (for example, 300 milliseconds) has passed from the time point of the last moving body detection 111a. The camera image used in the moving body detection 111b is, for example, a camera image of the camera image data read in by reading in the most recent camera image data at a time point after a predetermined time has passed from the last moving body detection 111a. In the example shown in the figure, the camera image used in the moving body detection 111b is a camera image of the camera image data read in by reading in 110j.

[0181] The CPU 60A acquires the moving object position information about the position of the moving object detected by the moving object detection 111b. The process of acquiring the moving object position information is the same as the process of acquiring the first position information of the moving object from the captured image read by the reading 110a.

[0182] The CPU 60A acquires slewing mechanism position information (pan / tilt value) regarding the current slewing position of the slewing mechanism 16 that slews the camera 10 .

[0183] The CPU 60A performs the rotation control of the rotation mechanism 16 based on the acquired moving object position information and the acquired rotation mechanism position information so that the moving object detected by the moving object detection 111b is displayed in the central area of ​​the camera image taken by the camera 10. The CPU 60A transmits information such as the translation / tilt value of the rotation mechanism 16 for displaying the moving object in the central area of ​​the camera image to the rotation mechanism 16 via the network 20 as a control instruction 121b for the rotation mechanism 16.

[0184] The rotating mechanism 16 starts a rotating operation 161 b in response to a control command 121 b sent from the management device 11 , and rotates the camera 10 in the commanded direction.

[0185] When the swivel operation 161b of the swivel mechanism 16 is started, the CPU 60A performs a swivel mechanism stop waiting process 122b to wait until the swivel mechanism 16 is swiveled to a commanded predetermined swivel position (translation / tilt value).

[0186] In conventional camera systems, for example, such a process is repeatedly performed to control the slewing mechanism 16 to monitor the moving object. As in conventional camera systems, when the next moving object detection 111b is started after a predetermined time has passed since the last moving object detection 111a, even if the slewing control based on the last control command 121a is completed, the next control command 121b is often not output immediately. Therefore, for example, if the moving speed of the moving object is fast, the moving object may deviate from the field of view of the camera 10 before the next moving object detection 111b, and the moving object cannot be accurately monitored.

[0187] <Processing of Management Device in Imaging System 1 of the Present Embodiment>

[0188] On the other hand, in the imaging system 1 of the present embodiment, the following management process is performed in order to image the moving object so that it does not deviate from the angle of view of the camera 10 . Figure 7 FIG. 1 is a diagram showing an example of management processing of the imaging system 1 according to the present embodiment. Figure 7 As shown, in the case of the camera system 1 of this embodiment, the CPU 60A of the management device 11 performs moving body detection 111a to 111n on the camera images read from the camera 10 by the reading 110a to 110n. The CPU 60A obtains moving body position information related to the position of the moving body detected in the moving body detection 111a to 111n. The method of obtaining the moving body position information is the same as the method of obtaining the moving body position information in the conventional camera system. The moving body position information is an example of the first position information in the present invention.

[0189] Furthermore, the CPU 60A performs predictions 131a to 1311 of the moving position of the moving object based on the moving position information detected in each moving object detection 111a to 111n, and sequentially acquires moving position information related to the position of the moving object. For example, the prediction 131a is performed by calculating the movement (speed and direction) of the moving object based on the moving position information detected in the moving object detection 111a and the moving position information detected in the moving object detection 111b, and the moving position information of the moving object at a time point earlier than the next moving object detection 111c is acquired. Furthermore, the prediction 131b is performed by calculating the movement (speed and direction) of the moving object based on the moving position information predicted in the prediction 131a and the moving position information detected in the moving object detection 111c, and the moving position information of the moving object at a time point earlier than the next moving object detection 111d is acquired. Similarly, prediction 131c is performed based on the moving object position information predicted in prediction 131b and the moving object position information detected in moving object detection 111d, and moving object position information of the moving object at a time point earlier than the next moving object detection 111e is acquired.

[0190] The CPU 60A acquires slewing mechanism position information (pan / tilt value) regarding the current slewing position of the slewing mechanism 16 that slews the camera 10. The slewing mechanism position information is an example of the second position information in the present invention.

[0191] The CPU 60A first performs the rotation control of the rotation mechanism 16 based on the moving object position information obtained by the moving object detection 111a and the above-mentioned current rotation mechanism position information, so that the moving object detected by the moving object detection 111a is displayed in the central area of ​​the camera image taken by the camera 10. The CPU 60A sends information such as the translation / tilt value of the rotation mechanism 16 for displaying the moving object in the central area of ​​the camera image to the rotation mechanism 16 via the network 20 as a control instruction 121a for the rotation mechanism 16.

[0192] The swivel mechanism 16 starts a swivel motion 161a according to the control command 121a sent from the management device 11, and swivels the direction of the camera 10 to the commanded direction. In the imaging system 1 of the present embodiment, moving object detection 111b to 111i for the camera image read from the camera 10 is repeatedly performed in parallel with the swivel control of the swivel mechanism 16. For example, moving object detection 111b to 111f for the camera image read from the camera 10 is performed even during the swivel motion 161a of the swivel mechanism 16.

[0193] If the rotation action 161a of the rotation mechanism 16 is started, the CPU 60A performs the stop waiting 122a processing of the rotation mechanism, which waits until the rotation mechanism 16 rotates to the specified rotation position (translation / tilt value) instructed. The termination of the stop waiting 122a of the rotation mechanism can be detected based on the end notification signal of the rotation action 161a sent from the rotation mechanism 16 side. In addition, the time required for the sending process of the control command from the management device 11 to the rotation mechanism 16 and the sending process of the completion notification signal from the rotation mechanism 16 to the management device 11 is very short, so the overall processing time of the management device 11 is less affected. In addition, the time until the rotation of the rotation mechanism 16 ends can be calculated, for example, based on the angle and speed at which the rotation mechanism 16 is rotated, so this time can be set as the time of the stop waiting 122a processing of the rotation mechanism.

[0194] If the rotation action 161a of the rotation mechanism 16 is completed and the stop waiting 122a processing of the rotation mechanism is terminated, the CPU 60A performs the image delay waiting 123a processing of waiting for the delay of the image generated by sending the camera image via the network 20 to terminate. Whether the delay of the image is terminated can be determined by analyzing the camera image read in through 110a~110n. The CPU 60A makes a determination based on, for example, the background part captured in the read camera image (for example, the background feature points reflected on the background of the moving object). If the rotation action 161a of the rotation mechanism 16 stops, the background feature points in the camera image will become a stopped state in the camera image as it stops. The CPU 60A determines that the delay of the image has terminated based on the stopping of the background feature points, and calculates the image delay time based on the stopping of the rotation action 161a of the rotation mechanism 16 and the stopping of the background feature points in the camera image. The image delay time is a period from the time when the swivel operation of the swivel mechanism 16 stops until the time when the background feature points in the captured image stop.

[0195] When the image delay ends and the image delay waiting 123a processing ends, the CPU 60A obtains the moving body position information of the moving body predicted by the most recent prediction 131h for the end point. In addition, the moving body position information predicted by the prediction 131h is the moving body position information predicted based on the moving body position information predicted in the prediction 131g and the moving body position information detected in the moving body detection 111i.

[0196] Furthermore, the CPU 60A acquires slewing mechanism position information (pan / tilt value) regarding the current slewing position of the slewing mechanism 16 for slewing the camera 10 .

[0197] The CPU 60A performs the slewing control of the slewing mechanism 16 based on the moving body position information obtained by the prediction 131h and the above-mentioned current slewing mechanism position information, so that the imaging range of the camera 10 changes toward the predicted moving position of the moving body. The CPU 60A sends information such as the pan / tilt value of the slewing mechanism 16 for photographing the moving body at the predicted moving position to the slewing mechanism 16 via the network 20 as a control command 121b for the slewing mechanism 16. In the imaging system 1 of the present embodiment, the CPU 60A performs the moving body detection 111b to 111i for the captured image read from the camera 10 between the control command 121a for slewing the slewing mechanism 16 and the next control command 121b. For example, the CPU 60A performs the moving body detection 111b to 111i during the period from the start of the slewing control, i.e., the sending of the control command 121a, to the end of the slewing control, i.e., the end of the image delay wait 123a.

[0198] The rotating mechanism 16 starts a rotating action 161b according to a control instruction 121b sent from the management device 11 so that the moving object of the monitored object continues to be included in the camera range of the camera 10, that is, the camera 10 continues to photograph the moving object and the direction of the camera 10 is rotated in the instructed direction.

[0199] When the swivel operation 161b of the swivel mechanism 16 is started, the CPU 60A performs a swivel mechanism stop waiting process 122b to wait until the swivel mechanism 16 is swiveled to a commanded predetermined swivel position (translation / tilt value).

[0200] As described above, the CPU 60A of the management device 11 performs the moving body detection 111a to 111n in the camera image read from the camera 10 for each read image. Therefore, the moving path of the moving body can be detected accurately, and the moving body can be photographed in a manner that does not deviate from the camera range of the camera 10. In addition, the CPU 60A repeatedly performs the moving body detection 111a to 111n in the camera image in parallel with the rotation control of the rotation mechanism 16. Therefore, the movement of the moving body can be accurately recognized, and the moving body can be photographed in a manner that does not deviate from the camera range of the camera 10.

[0201] <Motion prediction based on the motion vector of the moving object>

[0202] Figure 8 FIG. 1 is a diagram showing an example of predicting the moving position of a moving object based on the moving vector of the moving object in the management device 11 of the camera system 1. Figure 8 As shown, the CPU 60A of the management device 11 performs moving object detection 111a on the image captured by the camera 10 read in the reading 110a.

[0203] The CPU 60A performs the rotation control of the rotation mechanism 16 based on the moving object position information obtained by the moving object detection 111a and the current rotation mechanism position information so that the moving object detected by the moving object detection 111a is displayed in the central area of ​​the camera image taken by the camera 10. Specifically, the CPU 60A sends the pan / tilt value of the rotation mechanism 16 for displaying the moving object in the central area of ​​the camera image as the control command 121a to the rotation mechanism 16. When the rotation action 161a of the rotation mechanism 16 based on the control command 121a is started, the CPU 60A performs the stop waiting 122a processing of the rotation mechanism until the rotation action 161a is terminated.

[0204] The CPU 60A performs a swivel motion vector estimation 171a process of the swivel mechanism 16 based on the camera image of the camera 10 read in by the read-in 110a and the camera image of the camera 10 read in by the read-in 110b, and obtains the swivel motion vector. The swivel motion vector is a vector indicating the movement of the camera range of the camera 10 based on the swivel control of the swivel mechanism 16. The CPU 60A obtains the swivel motion vector from the camera image of the camera 10. For example, the CPU 60A obtains the swivel motion vector based on feature points such as the background portion in the camera image. The swivel motion vector is an example of the second motion vector in the present invention.

[0205] The CPU 60A performs a moving object detection 111b on the captured image of the camera 10 read in the reading 110b. The CPU 60A obtains an image motion vector indicating the movement of the moving object in the image of the camera range of the camera 10 based on the moving object position information detected by the moving object detection 111b and the moving object position information detected by the moving object detection 111a. The CPU 60A obtains the image motion vector from the captured image of the camera 10. For example, the CPU 60A obtains the image motion vector based on the feature points of the moving object in the captured image. The image motion vector is an example of the first motion vector in the present invention.

[0206] The CPU 60A performs a moving body motion vector estimation 181a process of the moving body based on the on-image motion vector acquired by the moving body detection 111b and the rotation motion vector acquired by the rotation motion vector estimation 171a, and acquires the moving body motion vector. The moving body motion vector is a vector indicating the movement of the moving body in the space where the moving body exists. The moving body motion vector is an example of the third motion vector in the present invention.

[0207] The CPU 60A calculates the movement (speed and direction) of the moving object based on the moving object movement vector obtained by the moving object movement vector estimation 181a and the moving object position information detected by the moving object detection 111a and executes the prediction 131a to obtain the moving object position information of the moving object. Furthermore, the CPU 60A executes the prediction 131b based on the moving object movement vector obtained by the moving object movement vector estimation 181b and the moving object position information obtained by the prediction 131a to obtain the moving object position information of the moving object. Similarly, the moving object position information of the moving object is obtained in sequence, for example, Figure 7 Similarly, the control command 121b is outputted to perform the slewing control of the slewing mechanism 16 based on the moving body position information obtained by the prediction 131h. The moving body position information obtained by the predictions 131a to 131d based on the moving body movement vector (third movement vector) is an example of the third position information in the present invention.

[0208] In addition, Figure 8The mark for image delay waiting is omitted, but for example, the time of image delay waiting can be estimated based on the relationship between information related to the rotation control of the rotation mechanism 16 and the rotation movement vector obtained by the rotation movement vector estimation processing of the rotation mechanism 16.

[0209] The CPU 60A of the management device 11 in this example performs a moving body motion vector estimation process of the moving body based on the on-image motion vector (first motion vector) acquired by the moving body detection and the rotation motion vector (second motion vector) acquired by the rotation motion vector estimation, and acquires the moving body motion vector (third motion vector). According to this structure, the moving body motion vector in the space where the moving body exists can be appropriately acquired, and the moving body can be photographed in a manner that does not deviate from the imaging range of the camera 10.

[0210] <Swing control based on feature points determined by historical information>

[0211] Fig. 9 FIG. 1 is a diagram showing an example of predicting the moving position of a moving object based on feature points specified by historical information in the management device 11 of the camera system 1. Fig. 9 As shown, the CPU 60A of the management device 11 performs a moving body detection 111a on the camera image of the camera 10 read by the reading 110a. The CPU 60A performs a rotation control of the rotation mechanism 16 based on the moving body position information obtained by the moving body detection 111a and the current rotation mechanism position information, so that the moving body detected by the moving body detection 111a is displayed in the central area of ​​the camera image taken by the camera 10.

[0212] The CPU 60A sends the pan / tilt value of the swivel mechanism 16 for displaying the moving object in the central area of ​​the captured image as the control command 121a to the swivel mechanism 16. For example, when the moving object is detected in the lower right direction relative to the central area of ​​the captured image when viewed from the front of the display 13a, the CPU 60A sends the pan / tilt value for rotating the swivel mechanism 16 in the lower right direction as the control command 121a to the swivel mechanism 16. The "lower right direction" for rotating the swivel mechanism 16 is an example of the history information in the present invention.

[0213] CPU60A and the above Figure 8 Similarly to the case described in , based on the camera image of the camera 10 read in the input 110a and the camera image of the camera 10 read in the input 110b, the swivel motion vector estimation 171a of the swivel mechanism 16 is performed to obtain the swivel motion vector (second motion vector). In addition, the swivel motion vector estimation 171b, 171c, 171d are performed in the same manner to obtain the swivel motion vectors respectively.

[0214] In addition, the CPU 60A can also obtain the slewing motion vector based on the control command 121a and the historical information of the slewing control. That is, when the current control command has occurred in the past, the slewing motion vector can be obtained from the historical information of the past control command. In this way, by using the historical information of the slewing control, the slewing motion vector can be obtained even without using the camera image.

[0215] For example, it is assumed as follows: among the respective rotational motion vectors acquired by the rotational motion vector estimation 171a to 171d, Fig. 9 As shown, the rotational motion vectors obtained by the rotational motion vector estimations 171a and 171b are rotational motion vectors without movement (zero vectors), and the rotational motion vectors obtained by the rotational motion vector estimations 171c and 171d are rotational motion vectors in the lower right direction.

[0216] In this case, the CPU 60A adopts the swivel motion vector that matches the swivel direction of the swivel mechanism 16 sent by the control instruction 121a, that is, the lower right direction, in the moving body motion vector estimation process, and does not adopt the swivel motion vector that does not match in the moving body motion vector estimation process. That is, the CPU 60A adopts the swivel motion vector of the lower right direction obtained by the swivel motion vector estimation 171c and 171d in the moving body motion vector estimation process, and does not adopt the swivel motion vector of no movement obtained by the swivel motion vector estimation 171c and 171d in the moving body motion vector estimation process.

[0217] The CPU 60A of the management device 11 in this example adopts the swivel motion vector that matches the historical information related to the swivel control of the swivel mechanism 16 in the moving body motion vector estimation process, and does not adopt the swivel motion vector that does not match the historical information related to the swivel control of the swivel mechanism 16 in the moving body motion vector estimation process. According to this structure, the swivel motion vector in the moving body motion vector estimation process can be appropriately acquired, and the moving body can be photographed in a manner that does not deviate from the imaging range of the camera 10.

[0218] <Modification of Processing of Management Device in Imaging System 1 of the Present Embodiment>

[0219] Fig.10 FIG. 2 is a diagram showing a modified example of the management process of the imaging system 1 according to the present embodiment. Fig.10 As shown in the example of the modified example of the management processing, the CPU 60A of the management device 11 performs the moving body detection 111a to 111n on the camera images read in from the camera 10 through the reading 110a to 110n, respectively, and obtains the moving body position information related to the position of the moving body detected in each moving body detection 111a to 111n. Figure 7 Same management process as described in .

[0220] In addition, the CPU 60A executes the predictions 131a to 1311 of the moving position of the moving object based on the moving object position information detected in each of the moving object detections 111a to 111n and sequentially acquires the moving object position information related to the position of the moving object. Figure 7 Same management process as described in .

[0221] In addition, the fact that the CPU 60A acquires the slewing mechanism position information (translation / tilt value) related to the current slewing position of the slewing mechanism 16 is also related to the present invention. Figure 7 Same management process as described in .

[0222] Furthermore, first, the CPU 60A performs the rotation control of the rotation mechanism 16 based on the moving body position information and the rotation mechanism position information, and sends the control instruction 121a to the rotation mechanism 16 so that the moving body detected by the moving body detection 111a is displayed in the central area of ​​the camera image. Figure 7 Same management process as described in .

[0223] In addition, the rotating mechanism 16 starts the rotating action 161a according to the control command 121a sent from the CPU 60A of the management device 11, and rotates the camera 10 in the commanded direction. Figure 7 Same management process as described in .

[0224] Furthermore, when the swivel mechanism 16 starts its swivel motion 161a, the CPU 60A performs a swivel mechanism stop waiting 122a process to wait until the swivel mechanism 16 is swiveled to the commanded predetermined swivel position (translation / tilt value). Figure 7 Same management process as described in .

[0225] However, in the modified example of the management process, the CPU 60A does not execute the image delay waiting 123a process for waiting for the end of the delay of the image generated by transmitting the camera image via the network 20 after the rotation operation 161a of the rotation mechanism 16 is completed and the stop waiting 122a process of the rotation mechanism is completed. Figure 7 The management process for executing the image delay waiting 123a process described in is different.

[0226] When the slewing action 161a of the slewing mechanism 16 is completed and the stop waiting 122a process of the slewing mechanism is terminated, the CPU 60A obtains the moving body position information of the moving body predicted by the most recent prediction 131e for the termination time point. In addition, the moving body position information predicted by the prediction 131e is the moving body position information predicted based on the moving body position information predicted in the prediction 131d and the moving body position information detected in the moving body detection 111f.

[0227] Furthermore, the CPU 60A acquires slewing mechanism position information (pan / tilt value) regarding the current slewing position of the slewing mechanism 16 for slewing the camera 10 .

[0228] The CPU 60A performs the rotation control of the rotation mechanism 16 based on the moving body position information obtained by the prediction 131e and the above-mentioned current rotation mechanism position information, so that the imaging range of the camera 10 changes toward the predicted moving position of the moving body. The CPU 60A sends information such as the translation / tilt value of the rotation mechanism 16 for photographing the moving body at the predicted moving position to the rotation mechanism 16 via the network 20 as a control instruction 121b for the rotation mechanism 16.

[0229] In the modified example of the management process, the CPU 60A executes the swivel control of the swivel mechanism 16 based on the next control command 121b at a point before the change of the camera image is terminated in the swivel control of the swivel mechanism 16 based on the control command 121a (without executing the image delay waiting process). The swivel control of the swivel mechanism 16 based on the control command 121a is an example of the first swivel control in the present invention. The swivel control of the swivel mechanism 16 based on the control command 121b is an example of the second swivel control in the present invention.

[0230] The rotation mechanism 16 starts the rotation action 161b according to the control command 121b so that the specified moving object continues to be included in the imaging range of the camera 10, that is, the orientation of the camera 10 is rotated in the instructed direction so that the camera 10 continues to image the specified moving object. Figure 7 Furthermore, if the rotation action 161b of the rotation mechanism 16 is started, the CPU 60A performs the stop waiting process 122b of the rotation mechanism, which is also the same as Figure 7 Same management process as described in .

[0231] The CPU 60A of the management device 11 in this modified example outputs the next control instruction 121b without executing the image delay waiting 123a processing for waiting for the image delay to terminate after the stop waiting 122a processing of the slewing mechanism based on the previous control instruction 121a is terminated. According to this structure, the tracking performance of the camera 10 to the moving object is improved, and it is easy to prevent the moving object from deviating from the camera range of the camera 10.

[0232] <Swing Control Based on Modification of Movement Vector>

[0233] The following describes the swing control (second swing control) of the swing mechanism 16 based on the control command of the above-described modification using the movement vector of the moving body.

[0234] The CPU 60A obtains the previous rotation movement vector indicating the movement of the imaging range of the camera 10 based on the control value in the previous rotation control of the rotation mechanism 16. The previous rotation movement vector is the movement amount of the imaging range predetermined in the previous rotation control. In addition, the CPU 60A performs the rotation movement vector estimation processing of the rotation mechanism 16 based on the camera image read from the camera 10, and obtains the next rotation movement vector indicating the movement of the imaging range of the camera 10. The next rotation movement vector is the actual movement amount of the imaging range. The CPU 60A performs the next rotation control of the rotation mechanism 16 based on the previous rotation movement vector and the next rotation movement vector. The previous rotation control of the rotation mechanism 16 is an example of the first rotation control in the present invention. The next rotation control of the rotation mechanism 16 is an example of the second rotation control in the present invention. The previous rotation movement vector is an example of the fourth movement vector in the present invention. The next rotation movement vector is an example of the fifth movement vector in the present invention.

[0235] As described above, in the modified example, the CPU 60A does not execute the delayed waiting process of the image when the stop waiting process of the slewing mechanism is terminated, but acquires the predicted moving body position information of the moving body and performs the next slewing control of the slewing mechanism 16 without executing the delayed waiting process of the image. Therefore, in the camera image read into the management device 11, in the last slewing movement vector in the last slewing control, the amount of the slewing movement vector corresponding to the stop waiting period of the slewing mechanism has moved, but the amount of the slewing movement vector corresponding to the delayed waiting period of the image has not moved completely.

[0236] Therefore, as an actual action, the rotation action of the rotation mechanism 16 has been terminated, but since an image delay has occurred, the background portion (rotation mechanism 16) is still in a state of movement as the camera image displayed on the management device 11. Therefore, the CPU 60A obtains the next movement vector by considering (subtracting) the amount of the rotation movement vector that has not yet completely moved in the previous movement vector in the previous rotation control, and performs the next rotation control based on the subtracted next movement vector. In this way, the movement vector in the next rotation control can be appropriately obtained, and the moving object can be photographed in a manner that does not deviate from the camera range of the camera 10.

[0237] Hereinafter, a modification example of the control by the CPU 60A will be described.

[0238] <First Modification>

[0239] The CPU 60A can determine the rotation speed of the rotation mechanism 16 according to the imaging condition of the camera 10, for example. The imaging condition of the camera 10 refers to, for example, the shutter speed of the camera 10. Specifically, the CPU 60A controls the rotation operation of the rotation mechanism 16 so as not to exceed the upper limit value of the rotation speed of the rotation mechanism 16 set according to the shutter speed of the camera 10.

[0240] Thus, for example, even when the shutter speed of the camera 10 is increased, the occurrence of shaking of the captured image can be suppressed, and the extraction of feature points of the captured image or the detection of a moving object can be appropriately performed. Furthermore, the CPU 60A can stop the detection of a moving object in the captured image when the rotation speed of the rotation mechanism 16 becomes greater than a predetermined speed relative to the shutter speed of the camera 10. Thus, the extraction of feature points or the detection of a moving object in a shaking captured image can be stopped, and the computing resources of the device can be reduced.

[0241] <Second Modification>

[0242] exist Figure 7 In the processing of the management device in this embodiment described in , the predictions 131a to 1311 of the moving positions of the moving objects are respectively performed based on the moving object position information detected by the moving object detection 111a to 111n, but the present invention is not limited to this. Figure 7 The moving object position information of the moving object used to output the control command 121b is based on the position information of the prediction 131h.

[0243] Therefore, for example, even if the position information of the moving object is not predicted in the predictions 131a to 131g, as long as the position information of the moving object is obtained in the moving object detections 111a to 111i, the moving object position information of the moving object can be predicted in the prediction 131h based on the position information. Figure 7 In the process of predicting the moving position of the moving object, the prediction processes 131a to 131g are omitted. Thus, the processing efficiency of the CPU 60A of the management device 11 can be improved.

[0244] <Third Modification>

[0245] The CPU 60A can change the imaging range (zoom) of the camera 10 according to the length of the image delay waiting time. For example, the CPU 60A can change the zoom of the camera 10 to the telephoto side when the image delay waiting time is long. In addition, the CPU 60A can change the imaging range (zoom) of the camera 10 according to the size of the moving body motion vector of the moving body. For example, the CPU 60A can change the zoom of the camera 10 to the telephoto side when the moving body motion vector is large. As a result, the moving body as the imaging object is not easy to deviate from the imaging range of the camera 10.

[0246] <Fourth Modification>

[0247] like Figure 8As described in the above, the rotational movement vector of the rotation mechanism 16 is obtained based on feature points such as the background portion of the moving object in the camera image. However, when the background portion that becomes a feature point in the camera image cannot be detected, for example, when the background portion of the moving object is a clear sky, the rotational movement vector of the rotation mechanism 16 may not be obtained based on the feature points.

[0248] In this case, the CPU 60A can change the processing content of the slewing control according to whether the slewing motion vector can be obtained. For example, the CPU 60A can switch to the following mode: when the slewing motion vector can be obtained, the slewing control with image delay waiting process is executed; when the slewing motion vector cannot be obtained, the slewing control without image delay waiting process is executed. In this way, the moving object as the imaging object is not easy to deviate from the imaging range of the camera 10.

[0249] <Fifth Modification>

[0250] like Figure 8 As described in the above, the CPU 60A obtains the moving body motion vector of the moving body based on the moving body motion vector on the image obtained by the moving body detection and the rotational motion vector of the rotation mechanism 16 obtained by the rotational motion vector estimation, and predicts the moving position of the moving body based on the moving body motion vector. However, the size of the moving body motion vector is not necessarily constant, and it also changes depending on the surrounding conditions. Therefore, the size of the moving vector on the image and the moving body motion vector also changes in many cases.

[0251] Therefore, when the randomness of the moving vector of the moving object is small, for example, when the moving object moves close to a constant speed motion, the CPU 60A predicts the moving position of the moving object based on the acquired moving vector of the moving object. On the other hand, when the randomness of the moving vector of the moving object is large, the CPU 60A can reduce the reference ratio of the acquired moving vector of the moving object to predict the moving position of the moving object. Thus, the predictability of the moving position of the moving object based on the moving vector of the moving object is improved.

[0252] <Storage Medium of Information Processing Program>

[0253] In the above management controls, an example is given in which the information processing program of each embodiment is stored in the storage device 60B of the management device 11, and the CPU 60A of the management device 11 executes the information processing program through the memory 60C. However, the technology of the present invention is not limited to this.

[0254] Fig.11 1 is a diagram showing an example of a method in which an information processing program is installed in the control device 60 of the management device 11 from a storage medium storing the information processing program for management control. Fig.11As shown, as an example, the information processing program 221 can be stored in a non-temporary storage medium, that is, a storage medium 220. Fig.11 In the illustrated example, the information processing program 221 stored in the storage medium 220 is installed in the control device 60 , and the CPU 60A executes the above-described processes according to the information processing program 221 .

[0255] Various embodiments have been described above, but the present invention is certainly not limited to these examples. Obviously, those skilled in the art can think of various variations or modifications within the scope described in the technical solution, and it should be understood that these variations or modifications also naturally fall within the technical scope of the present invention. Furthermore, the various constituent elements in the above embodiments can be arbitrarily combined within the scope of the purpose of the invention.

[0256] In addition, this application is based on the Japanese patent application (Japanese Patent Application No. 2022-147975) filed on September 16, 2022, and the contents thereof are incorporated herein by reference.

[0257] Explanation of symbols

[0258] 1-camera system, 10-camera, 11-management device, 13a, 43B-display, 13b-keyboard, 13c-mouse, 14-secondary storage device, 15-optical system, 15B-lens group, 15B1-anti-vibration lens, 15B2

[0259] - zoom lens, 16 - rotation mechanism, 17, 21 - lens actuator, 19 - computer, 20 - network, 22, 23, 75, 76 - driver, 22 - BIS driver, 23 - OIS driver, 25 - imaging element, 25A - light receiving surface, 27 - imaging element actuator, 28 - lens driver, 29, 45 - correction mechanism, 31 - DSP, 32 - image memory, 33 - correction unit, 34, 66 to 68, 79, 80 - communication I / F, 35, 60C - memory, 36, 60B - storage device, 37, 60A - CPU, 38, 70 - bus, 39, 47 - position sensor, 40-quantity detection sensor, 43-system device, 43-UI system device, 43A, 62-receiving device, 60-control device, 71-yaw axis rotation mechanism, 72-pitch axis rotation mechanism, 73, 74-motor, 110a~110n-reading, 111a~111n-moving body detection, 121a, 121b-control instruction, 122b-stop waiting processing, 131a~131l-prediction, 161a, 161b-rotation action, 171a~171d-rotation movement vector estimation, 181a, 181b-moving body movement vector estimation, 220-storage medium, 221-information processing program.

Claims

1. A control device comprising a processor for controlling an imaging device and a rotating device for rotating the imaging device, wherein: The processor performs the following processing: detecting a moving object from a captured image outputting captured image data acquired by the imaging device; acquiring first position information related to the position of the moving object in the camera image; Acquiring second position information related to the rotation position of the rotation device; According to the first position information and the second position information, the rotation device is controlled so that the moving object is included in the imaging range of the imaging device; The detection is performed even during the swinging motion of the swing device.

2. The control device according to claim 1, wherein: The detection is performed in parallel with the swing control.

3. The control device according to claim 1, wherein: The imaging device performs imaging control related to the imaging range.

4. The control device according to claim 1, wherein: The detection is performed during a period including from the start to the end of the swing control.

5. The control device according to claim 4, wherein: The end of the slewing control is a state in which the change of the captured image based on the slewing control is terminated.

6. The control device according to claim 1, wherein: The detection is performed during a period between control commands for the swing control.

7. The control device according to claim 1, wherein: The detection and the swing control are performed in parallel and repeatedly.

8. The control device according to claim 7, wherein: The detection is performed every time the captured image data is acquired by the image pickup device.

9. The control device according to claim 1, wherein: The processor performs the following processing: Acquire a third motion vector which is a motion vector of the moving object in a space where the moving object exists, based on a first motion vector which is a motion vector of the moving object in the imaging range and a second motion vector which is a motion vector of the imaging range based on the rotation control; acquiring third position information related to the position of the moving object based on the third motion vector; and The rotation of the rotation device is controlled based on the third position information.

10. The control device according to claim 9, wherein: The processor performs the following processing: The first motion vector and the second motion vector are acquired from the captured image.

11. The control device according to claim 10, wherein: The processor performs the following processing: The first motion vector and the second motion vector are acquired based on feature points of the captured image.

12. The control device according to claim 11, wherein: The processor performs the following processing: The second movement vector is acquired based on the feature point determined by the history information of the slewing control.

13. The control device according to claim 9, wherein: The processor performs the following processing: The second movement vector is acquired using the control command of the swing control and the history information of the swing control.

14. The control device according to claim 1, wherein: The processor performs the following processing: The second swivel control of the swivel device is performed at a point in time before the change of the captured image based on the first swivel control of the swivel device is completed.

15. The control device according to claim 14, wherein: The processor performs the following processing: The second swing control is performed based on a fourth motion vector of the imaging range based on the control value of the first swing control and a fifth motion vector of the imaging range based on the captured image.

16. The control device according to claim 1, wherein: The processor performs the following control: The rotation speed of the rotation device is determined according to the imaging condition of the imaging device.

17. The control device according to any one of claims 1 to 16, wherein: The camera device and the rotating device are connected via a network.

18. A camera system, comprising: Camera device; A rotating device for rotating the camera device; and A control device controls the camera device and the rotating device, The control device performs the following processing: detecting a moving object from a captured image outputting captured image data acquired by the imaging device; acquiring first position information related to the position of the moving object in the camera image; Acquiring second position information related to the rotation position of the rotation device; According to the first position information and the second position information, the rotation device is controlled so that the moving object is included in the imaging range of the imaging device; The detection is performed even during the swinging motion of the swing device.

19. A control method, which is performed by a control device having a processor for controlling a camera device and a rotating device for rotating the camera device, wherein the processor performs the following processing: detecting a moving object from a captured image outputting captured image data acquired by the imaging device; acquiring first position information related to the position of the moving object in the camera image; Acquiring second position information related to the rotation position of the rotation device; According to the first position information and the second position information, the rotation device is controlled so that the moving object is included in the imaging range of the imaging device; The detection is performed even during the swinging motion of the swing device.

20. A control program for a control device having a processor for controlling an imaging device and a rotating device for rotating the imaging device, the control program causing the processor to execute the following processing: detecting a moving object from a camera image that outputs camera image data acquired by the imaging device; acquiring first position information related to the position of the moving object in the camera image; Acquiring second position information related to the rotation position of the rotation device; According to the first position information and the second position information, the rotation device is controlled so that the moving object is included in the imaging range of the imaging device; The detection is performed even during the swinging motion of the swing device.

Images